mailbox._mboxMMDF.get_message throws away From envelope

[bpoaugust]

BPO	31524
Nosy	@warsaw, @bitdancer

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-09-19.23:27:27.205>
labels = ['easy', 'type-bug', 'expert-email', '3.10', '3.11', '3.9']
title = 'mailbox._mboxMMDF.get_message throws away From envelope'
updated_at = <Date 2022-01-16.19:05:51.256>
user = 'https://bugs.python.org/bpoaugust'

bugs.python.org fields:

activity = <Date 2022-01-16.19:05:51.256>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['email']
creation = <Date 2017-09-19.23:27:27.205>
creator = 'bpoaugust'
dependencies = []
files = []
hgrepos = []
issue_num = 31524
keywords = ['easy']
message_count = 5.0
messages = ['302570', '302575', '302577', '302578', '302582']
nosy_count = 3.0
nosy_names = ['barry', 'r.david.murray', 'bpoaugust']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31524'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

• #gh-75705: Set unixfrom envelope in mailbox._mboxMMDF #107117
• [3.12] GH-gh-75705: Set unixfrom envelope in mailbox._mboxMMDF (GH-107117) #115098
• [3.11] GH-gh-75705: Set unixfrom envelope in mailbox._mboxMMDF (GH-107117) #115099

[bpoaugust]

https://github.com/python/cpython/blob/master/Lib/mailbox.py#L778

The code here reads the first line, but fails to save it as the unixfrom line.

Alternatively perhaps it should reset the file back to the start so the message factory has sight of the envelope.

The end result is that the envelope is lost.

[bitdancer]

It looks like it is saving it (the set_from line). Do you have a test that proves otherwise?

[bpoaugust]

It is not saving the unix from line.

#!/usr/bin/env python3

with open("test.mbox",'w') as f:
    f.write("From sender@invalid Thu Nov 17 00:49:30 2016\n")
    f.write("Subject: Test\n")
    f.write("\n")
    f.write("\n")

import mailbox
messages = mailbox.mbox("test.mbox")
for msg in messages:
    print(msg.get('subject'))
    print(msg.get_from())
    print(msg.get_unixfrom())

[bpoaugust]

I believe that setting the file back to the start is probably the best solution.

The message as provided by e.g. postfix will include the From header and the parser is able to deal with that successfully, so I'm not sure why the mbox reader removes it before calling the parser. It only really needs to drop the trailing newline to ensure that the parser gets the same data.

[bitdancer]

It just needs to call set_unixfrom as well as set_from. I don't know why the MMDFMessage tracks it separately, but I'm sure the author had a reason that seemed good at the time :)