regression: code is SIGSEGVing after LLVM lifetime intrinsics were implemented

[farcaller]

Previously valid code is now behaving wrong.

  pub fn new<'a>(cx: &'a ExtCtxt, tts: &[TokenTree]) -> Parser<'a> {
    let sess = cx.parse_sess();
    let ttsvec = tts.iter().map(|x| (*x).clone()).collect();   // [1]
    // let ttsvec = Vec::from_slice(tts);   // [2]

    debug!("tts: {}", tts);
    debug!("ttsvec: {}", ttsvec);   // [3]

    let mut reader = box lexer::new_tt_reader(
        &sess.span_diagnostic, None, ttsvec) as Box<lexer::Reader>;

    let tok0 = reader.next_token();
    let token = tok0.tok;
    let span = tok0.sp;


    Parser {
      sess: sess,
      reader: reader,

      token: token,
      span: span,

      last_token: None,
      last_span: span,
    }
  }

Here's the problem description: parser cannot parse input, as it's hitting wrong tokens. Apparently, a bunch of tokens in input are lost. The problem is in line [1], which works as expected when replaced by line [2]. The problem is getting even worse, as line [3] SIGSEGVs the application.

If line [1] is replaced by line [2], the code fails further, as I described in #15931. I think it relates to the same problem (parser reports bad token which isn't even in the input!). It kind of feels like memory corruption (in rust, heh).

Digging deeper, it seems that it doesn't crash at debug!() point, it's just debug!() somehow altering the program state to allow it to crash further on. gdb can't unwind the stack for some reason, lldb shows this:

  * frame #0: 0x000000010235f522 libsyntax-4e7c5e5c.dylib`cell::Cell$LT$T$GT$::get::h16703176363586027891 + 66
    frame #1: 0x0000000000000018
    frame #2: 0x000000010238e6c4 libsyntax-4e7c5e5c.dylib`rc::RcBoxPtr::strong::h13767585679411698615 + 84
    frame #3: 0x00000001023f64af libsyntax-4e7c5e5c.dylib`rc::RcBoxPtr::inc_strong::h7981967022436922026 + 95
    frame #4: 0x00000001023f6436 libsyntax-4e7c5e5c.dylib`rc::Rc$LT$T$GT$.Clone::clone::h3247237882429377366 + 54
    frame #5: 0x00000001023f59bd libsyntax-4e7c5e5c.dylib`ast::TokenTree...std..clone..Clone::clone::hc9d3937696cc261fCkl + 221
    frame #6: 0x00000001024c0dd7 libsyntax-4e7c5e5c.dylib`ext::tt::transcribe::tt_next_token::he37abde7b9ce15f4Rfb + 2679
    frame #7: 0x00000001024c010d libsyntax-4e7c5e5c.dylib`parse::lexer::TtReader$LT$$x27a$GT$.Reader::next_token::hea6b26d7f0c328024mG + 93
    frame #8: 0x0000000106a65e29 libmacro_platformtree.dylib`parser::Parser$LT$$x27a$GT$::bump::hb11efbf8fc1eeb58Xnc + 249
    frame #9: 0x0000000106a5a843 libmacro_platformtree.dylib`parser::Parser$LT$$x27a$GT$::parse_platformtree::h55536ab21adb63f1CJb + 1235

I did a bisect yesterday, which points at 92d1f15 as the first bad commit.

I will try to provide a smaller test case, but so far I had no luck, all the pieces extracted from the original crate seem to work as expected.

Please advise, we're kind of stuck with this breakage.

[farcaller]

Further on, this happens only with --opt-level 2.

[farcaller]

Can't find the relevant chunk in IR, here's the full dump (with -O2, i.e. when it crashes): https://gist.github.com/farcaller/49cd31d3f95126c6bf99

[alexcrichton]

cc @dotdash

[dotdash]

@farcaller Could you provide the IR from -O2 before optimizations, i.e. from the .no-opt.bc file that's generated when using -C save-temps?

[farcaller]

http://vfp.in/macro_platformtree.no-opt.bc.gz

[dotdash]

Thanks, but unfortunately I wasn't paying enough attention and missed the fact that this is just a library and not a program making actual use the code. For generic functions, IR is only generated when there's a concrete user of the code. So, could you please also provide the IR (optimized as well as the .no-opt.bs file) for the program uses the library and causes the crash?

[farcaller]

This is a plugin to rustc, so far I wasn't been able to reproduce the crash on something standalone.

Also I think rustc is using the dylib "as is", no?

[dotdash]

Sorry, I jumped to conclusions, the generic parameter is just a lifetime, so the above doesn't apply. Looking through zinc's Rakefile, it looks like you provided the results for the build target macro_platformtree and what I'd need is platformtree_crate.

[dotdash]

Could build that myself (had assumed that I'd need a cross-compile toolchain).

[farcaller]

ok, so you don't need anything from me now?

[dotdash]

This is a misoptimization in LLVM's SimplifyCFG pass, same bug as in #16011 which was luckily a bit easier to analyze.