Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Parser] Expression tree is not properly generated in FormattedValue #641

Open
akshanshbhatt opened this issue Jun 20, 2022 · 1 comment · May be fixed by #2566
Open

[Parser] Expression tree is not properly generated in FormattedValue #641

akshanshbhatt opened this issue Jun 20, 2022 · 1 comment · May be fixed by #2566
Labels
Parser Issues or improvements related to parser

Comments

@akshanshbhatt
Copy link
Collaborator

akshanshbhatt commented Jun 20, 2022

FormattedValue field of a fstring does not generate a proper expression tree for the expression inside it.

Example -

$ cat examples/expr2.py
f'''
a is {a}
b is {b}
sum is {a + b}
'''
$ lpyast examples/expr2.py
(Module [(Expr (JoinedStr [(ConstantStr "
a is " ()) (FormattedValue (Name a Load) -1 ()) (ConstantStr "
b is " ()) (FormattedValue (Name b Load) -1 ()) (ConstantStr "
sum is " ()) (FormattedValue (BinOp (Name a Load) Add (Name b Load)) -1 ()) (ConstantStr "
" ())]))] [])
$ lpynewpar examples/expr2.py
(Module [(Expr (JoinedStr [(ConstantStr "
a is " ()) (FormattedValue (Name a Load) -1 ()) (ConstantStr "
b is " ()) (FormattedValue (Name b Load) -1 ()) (ConstantStr "
sum is " ()) (FormattedValue (Name a + b Load) -1 ()) (ConstantStr "
" ())]))] [])
@certik
Copy link
Contributor

certik commented Jun 20, 2022

Currently a + b is a string, and we put it into a Name. Rather, we will need to parse this string properly, using the parser itself, to obtain (BinOp (Name a Load) Add (Name b Load)). It would be nice to reuse the parser that we have. This expression I assume can be arbitrarily complicated, such as a+sin(x)+f*10**y - g(a, b, c, d = x). I don't know if our parser has some global state or not. If not, then it should be possible to instantiate it and call it on this string to obtain AST, while parsing the main file. If there is a global state, then probably the best way is to store a "flag" that the generated AST will have to be visited after wards. Then we write an AST visitor, that goes into each FormattedValue and calls the parser on the string to get BinOp. This walker will slow things down, so we only want to call it if the "flag" is on, which will only happen if there is an expression like "a+b". Most use cases are just simple variables I think, and those we can parse already.

@Thirumalai-Shaktivel Thirumalai-Shaktivel added the Parser Issues or improvements related to parser label Jun 20, 2022
@mrdaybird mrdaybird linked a pull request Mar 2, 2024 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Parser Issues or improvements related to parser
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants