[security] CVE-2022-0391: urllib.parse should sanitize urls containing ASCII newline and tabs.

[orsenthil]

BPO	43882
Nosy	@gpshead, @orsenthil, @vstinner, @ned-deily, @OddBloke, @ambv, @mgorny, @apollo13, @mlissner, @pablogsal, @miss-islington, @tirkarthi, @felixxm, @sethmlarson
PRs	bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs. #25595 [3.9] bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs. (GH-25595) #25725 [3.8] bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs. (GH-25595) #25726 [3.7] bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs. (GH-25595) #25727 [3.6] bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs. (GH-25595) #25728 [3.9] bpo-43882 Remove the newline, and tab early. From query and fragments. #25853 bpo-43882 Remove the newline, and tab early. From query and fragments. #25921 [3.7] bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs. #25923 [3.6] bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs #25924 [3.10] bpo-43882 Remove the newline, and tab early. From query and fragments. (GH-25921) #25936 [3.7] bpo-43882 - Mention urllib.parse changes in Whats New section for 3.7.11 #26267 [3.6] bpo-43882 - Mention urllib.parse changes in Whats New section for 3.6.14 #26268 [3.10] bpo-43882 - Mention urllib.parse changes in Whats new section. #26275 [3.9] bpo-43882 - Mention urllib.parse changes in Whats new section. #26276 [3.8] bpo-43882 - Mention urllib.parse changes in Whats new section. #26277

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/orsenthil'
closed_at = <Date 2021-06-02.01:26:09.260>
created_at = <Date 2021-04-18.19:37:00.259>
labels = ['type-security', '3.8', '3.9', '3.10', '3.11', '3.7', 'library']
title = '[security] CVE-2022-0391: urllib.parse should sanitize urls containing ASCII newline and tabs.'
updated_at = <Date 2022-02-09.11:40:17.243>
user = 'https://github.com/orsenthil'

bugs.python.org fields:

activity = <Date 2022-02-09.11:40:17.243>
actor = 'felixxm'
assignee = 'orsenthil'
closed = True
closed_date = <Date 2021-06-02.01:26:09.260>
closer = 'gregory.p.smith'
components = ['Library (Lib)']
creation = <Date 2021-04-18.19:37:00.259>
creator = 'orsenthil'
dependencies = []
files = []
hgrepos = []
issue_num = 43882
keywords = ['patch']
message_count = 47.0
messages = ['391343', '391352', '391426', '391859', '392334', '392338', '392611', '392781', '392808', '392835', '392873', '392926', '392944', '392971', '392995', '393009', '393025', '393030', '393033', '393034', '393039', '393049', '393107', '393108', '393136', '393139', '393142', '393144', '393146', '393149', '393150', '393198', '393203', '393205', '393207', '393211', '393997', '394056', '394057', '394058', '394062', '394112', '394113', '396628', '412688', '412705', '412821']
nosy_count = 14.0
nosy_names = ['gregory.p.smith', 'orsenthil', 'vstinner', 'ned.deily', 'odd_bloke', 'lukasz.langa', 'mgorny', 'apollo13', 'Mike.Lissner', 'pablogsal', 'miss-islington', 'xtreak', 'felixxm', 'sethmlarson']
pr_nums = ['25595', '25725', '25726', '25727', '25728', '25853', '25921', '25923', '25924', '25936', '26267', '26268', '26275', '26276', '26277']
priority = 'high'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue43882'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

[orsenthil]

A security issue was reported by Mike Lissner wherein an attacker was able to use \r\n in the url path, the urlparse method didn't sanitize and allowed those characters be present in the request.

In [9]: from urllib.parse import urlsplit
In [10]: urlsplit("java\nscript:alert('bad')")
Out[10]: SplitResult(scheme='', netloc='', path="java\nscript:alert('bad')", query='', fragment='')

Firefox and other browsers ignore newlines in the scheme. From
the browser console:

> new URL("java\nscript:alert(bad)")
<< URL { href: "javascript:alert(bad)", origin: "null", protocol:
"javascript:", username: "", password: "", host: "", hostname: "", port: "", pathname: "alert(bad)", search: ""

Mozilla Developers informed about the controlling specification for URLs is in fact defined by the "URL Spec"
from WHATWG which updates RFC 3986 and specifies that tabs and newlines
should be stripped from the scheme.

See: https://url.spec.whatwg.org/#concept-basic-url-parser

That link defines an automaton for URL parsing. From that link, steps 2 and 3 of scheme parsing read:

If input contains any ASCII tab or newline, validation error.
3. Remove all ASCII tab or newline from input.

---

urlparse module behavior should be updated, and an ASCII tab or newline should be removed from the url (sanitized) before it is sent to the request, as WHATWG spec.

[tirkarthi]

See also a related issue to sanitise newline on other helper functions https://bugs.python.org/issue30713

See also discussion and compatibility on disallowing control characters : https://bugs.python.org/issue30458