Description
I’d like to report what appears to be a bug in the rust compiler which can be reproduced by using the reqwest crate, where it fails to include the http version string in the request and it only affects binaries that were:
- built targeting x86_64-apple-darwin
- using release mode
- using a macOS Ventura (13.x) SDK
The bug itself has nothing to do with the reqwest crate (looking at the code, it appears that this behaviour should be impossible) and specifically related to the built binary.
this bug does exist in release binaries cross-compiled from linux to macOS when using the macOS 13 (Ventura) SDK with osxcross.
the bug does not exist when creating a debug build or when cross-compiling from an AppleSilicon machine → intel and does not exist when performing a release build on intel → intel when the OS is not Ventura.
We tracked this down to being triggered by opt-level = 2
this bug does not exist in rustc 1.85.0 and appears to only affect 1.86.0 (and also occurs in nightly rust).
I have a reduction:
add reqwest with --features=blocking and this main.rs:
use reqwest::{blocking::Client, Method};
fn main() {
Client::new().request(Method::GET, "http://localhost:8888/hello").send().unwrap();
}in another window, start a nc server with: nc -l localhost 8888
then cargo run and you should see:
GET /hello HTTP/1.1
accept: */*
host: localhost:8888
If you then start a new nc server and cargo run --release you will see:
GET /hello
accept: */*
host: localhost:8888
(note the missing HTTP/1.1 on the first line)
when running this code in a debugger, I can see that the slice that’s supposed to add the version number to the request buffer has a length of 0, so it contains no data:
Process 79729 stopped
* thread #2, name = 'reqwest-internal-sync-runtime', stop reason = step in
frame #0: 0x00000001000b380f reqwest-test`_$LT$hyper..proto..h1..role..Client$u20$as$u20$hyper..proto..h1..Http1Transaction$GT$::encode::h33cc17a167a456d1 at spec_extend.rs:61:18 [opt]
58 #[track_caller]
59 fn spec_extend(&mut self, iterator: slice::Iter<'a, T>) {
60 let slice = iterator.as_slice();
-> 61 unsafe { self.append_elements(slice) };
62 }
63 }
Target 0: (reqwest-test) stopped.
(lldb) p self
(alloc::vec::Vec<unsigned char, alloc::alloc::Global> *) $0 = 0x0000000100504b90
(lldb) p self[0]
(alloc::vec::Vec<unsigned char, alloc::alloc::Global>) $1 = size=11 {
[0] = 'G'
[1] = 'E'
[2] = 'T'
[3] = ' '
[4] = '/'
[5] = 'h'
[6] = 'e'
[7] = 'l'
[8] = 'l'
[9] = 'o'
[10] = ' '
}
(lldb) p slice
(*const [u8]) $2 = {
data_ptr = 0x0000000000000000
length = 0
}
a note from someone on my team who continued to look into it more deeply:
“it looks like it's computing the correct string HTTP/1.1 but for some odd reason it's computing a length of 0. as best I can tell it uses one lookup table of string pointers to find the string, and it uses another lookup table of 4-byte offsets that get added to the lookup table's base address to produce a new pointer, which looks like it's supposed to be the end of the string. Unfortunately that second lookup table has 3 identical values in it, meaning it will produce the correct end pointer for HTTP/1.0 but it produces the start pointer for HTTP/1.1, and so it ends up calculating a length of 0”
This always happens no matter what version of reqwest is used, and it seems to be caused by the rustc version.
I hope this is enough information.