Skip to content

gh-96670: Raise SyntaxError when parsing NULL bytes #97594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 27, 2022
Merged

gh-96670: Raise SyntaxError when parsing NULL bytes #97594

merged 1 commit into from
Sep 27, 2022

Conversation

pablogsal
Copy link
Member

@pablogsal pablogsal commented Sep 27, 2022 •
edited by bedevere-bot
Loading

@pablogsal pablogsal force-pushed the gh-96670 branch 5 times, most recently from 97a8f83 to cb89392 Compare September 27, 2022 12:25
gvanrossum
gvanrossum reviewed Sep 27, 2022
View reviewed changes
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's up with python3.12.abi.new having 22,644 additions?

gvanrossum
gvanrossum reviewed Sep 27, 2022
View reviewed changes
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG except for one word in an error message, I think.

@pablogsal
Copy link
Member Author

What's up with python3.12.abi.new having 22,644 additions?

Wow, that's some error. Let me clean that up

lysnikolaou
lysnikolaou reviewed Sep 27, 2022
View reviewed changes
@gvanrossum
Copy link
Member

Ugh, force-push. :-(

gvanrossum
gvanrossum reviewed Sep 27, 2022
View reviewed changes
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see nothing else wrong, let's wait for Lysandros' LGTM.

lysnikolaou
lysnikolaou approved these changes Sep 27, 2022
View reviewed changes
Copy link
Member

@lysnikolaou lysnikolaou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well!

@mdboom
Copy link
Contributor

mdboom commented Sep 27, 2022 •
edited
Loading

I was going to ask "doesn't this break things for encodings where NULLs are normal, like UTF-16", but then realized we never supported that anyway:

PEP 253: "It does not include encodings which use two or more bytes for all characters like e.g. UTF-16. The reason for this is to keep the encoding detection algorithm in the tokenizer simple."

In that sense, we get a better error message for UTF-16 files with this PR than before. Prior, it would silently fail (behave as if an empty file), and with this PR:

  File "x.py", line 1
    #
SyntaxError: source code cannot contain null bytes

@pablogsal pablogsal merged commit aab01e3 into python:main Sep 27, 2022
@pablogsal pablogsal deleted the gh-96670 branch September 27, 2022 22:23
@pablogsal
Copy link
Member Author

Landing, thank you everyone for your great review and comments! And also thank @mdboom for checking the UTF-16, that was a good consideration :)

P.S. Sorry for the force push :S

@vstinner vstinner mentioned this pull request Sep 28, 2022
Closed
@bskinn bskinn mentioned this pull request Apr 24, 2023
Closed
lysnikolaou added a commit to lysnikolaou/cpython that referenced this pull request May 5, 2023
@lysnikolaou @pablogsal
[3.11] pythongh-96670: Raise SyntaxError when parsing NULL bytes (pyt…
e6d28ba 
…honGH-97594).

(cherry picked from commit aab01e3)

Co-authored-by: Pablo Galindo Salgado <[email protected]>
@bedevere-bot
Copy link

GH-104195 is a backport of this pull request to the 3.11 branch.

pablogsal pushed a commit that referenced this pull request May 7, 2023
@lysnikolaou
[3.11] gh-96670: Raise SyntaxError when parsing NULL bytes (GH-97594) (…
a09d390 
…#104195)
@mkniewallner mkniewallner mentioned this pull request Jun 18, 2023
Merged
4 tasks
@bdraco bdraco mentioned this pull request May 12, 2024
Closed
This was referenced Jul 30, 2024
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gvanrossum gvanrossum gvanrossum left review comments

@lysnikolaou lysnikolaou lysnikolaou approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@pablogsal @gvanrossum @mdboom @bedevere-bot @lysnikolaou