python string eval string got error in dict

Question

In [33]: def select_func(df): ...: df=df.copy() ...: rules=["df['Age']<60"] ...: s={ f"n{i}" :eval(r) for i,r in enumerate(rules) } ...: return df In [34]: select_func(df=df1) --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-34-4cc0569a2417> in <module> ----> 1 select_func(df=df1) <ipython-input-33-0613bd096f62> in select_func(df) 2 df=df.copy() 3 rules=["df['Age']<60"] ----> 4 s={ f"n{i}" :eval(r) for i,r in enumerate(rules) } 5 return df 6 <ipython-input-33-0613bd096f62> in <dictcomp>(.0) 2 df=df.copy() 3 rules=["df['Age']<60"] ----> 4 s={ f"n{i}" :eval(r) for i,r in enumerate(rules) } 5 return df 6 <string> in <module> NameError: name 'df' is not defined df1=pd.DataFrame({'Name':['JOHN','ALLEN','BOB','NIKI','CHARLIE','CHANG'], 'Age':[35,42,63,29,47,51], 'Salary_in_1000':[100,93,78,120,64,115], 'FT_Team':['STEELERS','SEAHAWKS','FALCONS','FALCONS','PATRIOTS','STEELERS']}) 

It is strange that eval() in dict got an error. Strangely enough, it works without dict.

• Note：df1 is a must. I do not want to only use df as an input variable. df1 is also necessary for my application.

• My Goal is that creating columns telling why the row is not selected, which refers this post. But I don't want to create n1,n2,…… to refer each condition. Instead, I want to using raw conditions as variable.

• env

  Python 3.7.9 | packaged by conda-forge | (default, Dec 9 2020, 20:36:16) [MSC v.1916 64 bit (AMD64)] Type 'copyright', 'credits' or 'license' for more information IPython 7.32.0 -- An enhanced Interactive Python. Type '?' for help.

Answer 1

The minimal example

The minimal example of your problem is:

m1=1 def outer_fun(m): commands = ["m"] s = {i: eval(r) for i, r in enumerate(commands)} outer_fun(m1) 

Which gives:

Traceback (most recent call last): File "/tmp/pycharm_project_53/eval.py", line 7, in <module> outer_fun(m1) File "/tmp/pycharm_project_53/eval.py", line 5, in outer_fun s = {i: eval(r) for i, r in enumerate(commands)} File "/tmp/pycharm_project_53/eval.py", line 5, in <dictcomp> s = {i: eval(r) for i, r in enumerate(commands)} File "<string>", line 1, in <module> NameError: name 'm' is not defined 

Simplify the dict comprehension operation

The dict comprehension makes the problem complex and conceals the essence, let's get rid of it.

1. According to the PEP 274, the expression in you question is equivalent to s = dict([(i, eval(r)) for i, r in enumerate(commands)]), where the [(i, eval(r)) for i, r in enumerate(commands)] causes the problem.
2. I think the list comprehesion used above will use lambda function.You can get rid of the list comprehension, perhpas get something like s = list(map(lambda i, r: (i, eval(r)), range(len(commands)), commands))(EDIT: this is not accurate, see my question Is list comprehension implemented via map and lambda function?, they are not the same, but they work in a similar way; Perhaps one can still uses the lambda to understand the behavior of list comprehension if doesn't want to refer to python assembly). And just applying the lambda function (lambda i, r: (i, eval(r)))(0, "m") will result in problems!
3. Using an explict function definition to replace the lambda function that cause the problem. which gives:

m1=1 def outer_fun(m): commands = ["m"] # All of the following lines fails: # s = {i: eval(r) for i, r in enumerate(commands)} # s = dict([(i, eval(r)) for i, r in enumerate(commands)]) # s = [(i, eval(r)) for i, r in enumerate(commands)] # s = list(map(lambda i, r: (i, eval(r)), range(len(commands)), commands)) # (lambda i, r: (i, eval(r)))(0, "m") def inner_func(i, r): return (i, eval(r)) inner_func(0, "m") outer_fun(m1) 

And it is just like:

m1=1 def outer_fun(m): def inner_func(): return (0, eval("m")) inner_func() outer_fun(m1) 

A most simplified version

A most simplified version will be:

m1=1 def outer_fun(m): def inner_func(): eval("m") inner_func() outer_fun(m1) 

Which give:

Traceback (most recent call last): File "/tmp/pycharm_project_53/eval.py", line 7, in <module> outer_fun(m1) File "/tmp/pycharm_project_53/eval.py", line 6, in outer_fun inner_func() File "/tmp/pycharm_project_53/eval.py", line 5, in inner_func eval("m") File "<string>", line 1, in <module> NameError: name 'm' is not defined 

Why the "most simplified version" raise exception

You can refer to the doc of eval. If arguments globals and locals are not passed to the eval, the eval is executed with the globals and locals in the environment where eval() is called. However, m is not the local variable of inner_func, nor a global variable(but m1 is a global variable).

Another thing that may be insteresting is that if you replace the eval('m') with print(m), the inner_func can print without any problem, this is because m is the local variable of the outer_func that the inner_func can access.

You can check the globals and locals with the function globals() and locals():

m1=1 def outer_fun(m): print("local in outer:") print(locals()) print("global in outer:") print(globals()) def inner_func(): print("local in inner:") print(locals()) print("global in inner:") print(globals()) eval("m") inner_func() outer_fun(m1) 

Which gives:

local in outer: {'m': 1} global in outer: {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7f0d85ad0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__file__': '/tmp/pycharm_project_53/11.py', '__cached__': None, 'm1': 1, 'outer_fun': <function outer_fun at 0x7f0d85ad6280>} local in inner: {} global in inner: {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7f0d85ad0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__file__': '/tmp/pycharm_project_53/11.py', '__cached__': None, 'm1': 1, 'outer_fun': <function outer_fun at 0x7f0d85ad6280>} 

How to fix

You should store the globals and locals, and then pass them to the eval.

m1=1 def outer_fun(m): commands = ["m"] globs = globals() locs = locals() s = {i: eval(r, globs, locs) for i, r in enumerate(commands)} print(s) outer_fun(m1) 

Futher reading

These questions may be related:

1. NameError using eval inside dictionary comprehension
2. Python Name Error in dict comprehension when called inside a function
3. eval fails in list comprehension