Dear NEPS-users, dear colleagues,
I sometimes struggle to find the variables that I look for. While https://variablesearch.lifbi.de/ is very helpful indeed, I was looking for a way to semantically search for variables. Meaning, not having to know the exact wording. Simply feeding the publicly available PDF codebook into a LLM/chatbot is not feasible because it is too large for most models (especially local ones) to keep in memory/context. Hence, this script follows a two-layered logic. It creates a compact csv file that contains labels and variable names (basically a codebook light) which is semantically embedded and a look-up csv with additional information (basically the codebook). The latter is only filled, if using a real SUF in place of the Data Structure File.
Disclaimer: This is not supposed to replace any services by LIfBi or NEPS and is completely unofficial. Use at your own risk.
Comments are very welcome!