Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

test_simple_fstring and test_fstring_with_args fail with Python 3.12 beta #646

Open
tjanez opened this issue Jul 4, 2023 · 2 comments · May be fixed by #656
Open

test_simple_fstring and test_fstring_with_args fail with Python 3.12 beta #646

tjanez opened this issue Jul 4, 2023 · 2 comments · May be fixed by #656

Comments

@tjanez
Copy link
Contributor

tjanez commented Jul 4, 2023

Fedora is rebuilding all Python packages for the upcoming Python 3.12 upgrade in Fedora 39.

Pydocstyle's test suite is run as part of the RPM build process and there are 2 failures (which were not present before):

============================= test session starts ==============================
platform linux -- Python 3.12.0b3, pytest-7.3.2, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /builddir/build/BUILD/pydocstyle-6.3.0
configfile: pyproject.toml
collecting ... collected 214 items
src/tests/parser_test.py::test_function PASSED                           [  0%]

... trimmed ...

src/tests/test_utils.py::test_strip_non_alphanumeric PASSED              [100%]
=================================== FAILURES ===================================
_____________________________ test_simple_fstring ______________________________
    def test_simple_fstring():
        """Test parsing of a function with a simple fstring as a docstring."""
        parser = Parser()
        code = CodeSnippet("""\
            def do_something(pos_param0, pos_param1, kw_param0="default"):
                f\"""Do something.\"""
                return None
        """)
        module = parser.parse(code, 'file_path')
        assert module.is_public
        assert module.dunder_all is None
    
        function, = module.children
        assert function.name == 'do_something'
        assert function.decorators == []
        assert function.children == []
>       assert function.docstring == 'f"""Do something."""'
E       assert None == 'f"""Do something."""'
E        +  where None = Function(name='do_something', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=3, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=4, decorators=[], docstring=None, children=[Function(name='do_something', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=3, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=4, decorators=[], docstring=None, children=[...], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='')], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='').docstring
src/tests/parser_test.py:69: AssertionError
____________________________ test_fstring_with_args ____________________________
    def test_fstring_with_args():
        """Test parsing of a function with an fstring with args as a docstring."""
        parser = Parser()
        code = CodeSnippet("""\
            foo = "bar"
            bar = "baz"
            def do_something(pos_param0, pos_param1, kw_param0="default"):
                f\"""Do some {foo} and some {bar}.\"""
                return None
        """)
        module = parser.parse(code, 'file_path')
        assert module.is_public
        assert module.dunder_all is None
    
        function, = module.children
        assert function.name == 'do_something'
        assert function.decorators == []
        assert function.children == []
>       assert function.docstring == 'f"""Do some {foo} and some {bar}."""'
E       assert None == 'f"""Do some {foo} and some {bar}."""'
E        +  where None = Function(name='do_something', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=3, end=5, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=1, end=6, decorators=[], docstring=None, children=[Function(name='do_something', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=3, end=5, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=1, end=6, decorators=[], docstring=None, children=[...], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='')], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='').docstring
src/tests/parser_test.py:100: AssertionError
======================== 2 failed, 212 passed in 18.93s ========================

This is Fedora's build task in Koji.

Full RPM build log

@sigmavirus24
Copy link
Member

This is because Py3.12 changed how strings are tokenized and broke most things. I suspect there hasn't been enough time for this to be updated. It's been tricky to do in other projects I've seen attempt it as well

@mgorny
Copy link
Contributor

mgorny commented Oct 2, 2023

Ping. 3.12.0 is just around the corner, and this is now blocking other packages that are otherwise ready for 3.12 already.

parona-source added a commit to parona-source/pydocstyle that referenced this issue Nov 1, 2023
* fstring don't emit tk.STRING in python3.12, reattach the now separate
  tokens and pass them to Docstring preserving previous behavior.

Closes: PyCQA#646
Signed-off-by: Alfred Wingate <[email protected]>
@parona-source parona-source linked a pull request Nov 1, 2023 that will close this issue
2 tasks
parona-source added a commit to parona-source/pydocstyle that referenced this issue Nov 1, 2023
* fstring don't emit tk.STRING in python3.12, reattach the now separate
  tokens and pass them to Docstring preserving previous behavior.

Closes: PyCQA#646
Signed-off-by: Alfred Wingate <[email protected]>
parona-source added a commit to parona-source/pydocstyle that referenced this issue Nov 1, 2023
* fstring don't emit tk.STRING in python3.12, reattach the now separate
  tokens and pass them to Docstring preserving previous behavior.

Closes: PyCQA#646
Signed-off-by: Alfred Wingate <[email protected]>
parona-source added a commit to parona-source/pydocstyle that referenced this issue Nov 1, 2023
* fstring don't emit tk.STRING in python3.12, reattach the now separate
  tokens and pass them to Docstring preserving previous behavior.

Closes: PyCQA#646
Signed-off-by: Alfred Wingate <[email protected]>
parona-source added a commit to parona-source/pydocstyle that referenced this issue Nov 2, 2023
* fstrings are broken into several distinct token in py3.12, reattach
  them together as a singular string to preserve previous behavior.

Closes: PyCQA#646
Signed-off-by: Alfred Wingate <[email protected]>
parona-source added a commit to parona-source/pydocstyle that referenced this issue Nov 2, 2023
* fstrings are broken into several distinct token in py3.12, reattach
  them together as a singular string to preserve previous behavior.

Closes: PyCQA#646
Signed-off-by: Alfred Wingate <[email protected]>
parona-source added a commit to parona-source/pydocstyle that referenced this issue Nov 2, 2023
* fstrings are broken into several distinct tokens in py3.12, reattach
  them together as a singular string to preserve previous behavior.

Closes: PyCQA#646
Signed-off-by: Alfred Wingate <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants