Must `static DATA: UnsafeCell = expr;` still `== expr` by `main()`?

[workingjubilee]

"Okay I think we first and foremost have a Rust Abstract Machine / @rust-lang/opsem question here, not a Miri question:

If a mutable (or interior mutable) static is initialized in Rust to a certain value, then can the Rust AM assume that it does have that value when the program starts? Or is it okay to have some "before main setup" actually change the value of that static?

I would say for an immutable static, this is clearly UB -- using linker script tricks like what has been described here is just not allowed for those statics. But for statics that can be mutated, we already can't in general optimize assuming their values did not change, so... it seems reasonable to allow this?

With this way of thinking about the question, obviously Miri has no way of knowing that you are modifying the value of the static before Rust code starts running. On the Rust side you are saying this static is filled with uninit memory, but then you are making some outside-of-Rust assumptions to justify that actually when the program starts the static has a different value, in particular that the bytes now are initialized. So I don't think there is anything actionable on the Miri side here, and the discussion (with the clarified question) should move to https://github.com/rust-lang/unsafe-code-guidelines/ ."

Originally posted by @RalfJung in rust-lang/miri#2807 (comment)

[digama0]

I think that mutable statics still need to have the values they were promised to have in the initializer unless an AM-visible write occurred to them. That write might have been caused by FFI or asm!, but if the value isn't there at lang_start then it's not clear where the value goes at all, in which case a compiler which just ignores all static mut initializers would be conforming and that's not ok.

Regarding the linked thread, I think the appropriate model is to use an extern static and allocate/uninitialize it using a linker script, and then use as the AM invariant that when main() is called that static is initialized with a random bit pattern (not uninit). That's still not quite enough to validate the technique of signature validation since that's only probabilistically correct in the first place, but it does eliminate the need to do any volatile reads and writes and you won't get UB.

I don't see a way to make this work with a rust-allocated static without lying to the compiler to some extent. For example, we could use a mutable static (initialized to 0, say) with a link_section = ".uninit" on it, and then call an empty asm! block which has the postcondition of ensuring that the static is initialized to a random bit pattern, but in that case the AM will be in disagreement with the hardware between the start of main() and the asm! block, since the AM thinks the static is zero-initialized and the hardware static has actual data in it. (This is "fine" since the static will not be read or written to during this period.)

[Diggsey]

What if you did this:

use std::arch::asm;

static mut FOO: i32 = 0;

pub fn main() {
    unsafe {
        asm!("/* {} */", in(reg) &mut FOO);
    }
    // Rest of program...
}

The ASM block does nothing but the compiler must behave as though it may have written a value into the static. I guess this is what you were suggesting @digama0 ?

[digama0]

Yes, that is what I meant. The drawback is that the compiler is technically allowed to read FOO before the asm! block and assume that the value in the static is 0 because no threads have started yet so things should still look like the initialization values. In practice this doesn't seem to be a likely issue, since there isn't much optimization reason to do so.

(Also, we should replace main() with lang_start() in this discussion; there is some std code which runs before main which is not special in this regard. This should be fine for the present discussion since it certainly doesn't touch FOO.)

[CAD97]

It's not completely unreasonable to have a step between codegen where the initial starting parameters of the AM are serialized and the start entry point into AM execution where the host could be allowed to insert some set of AM state changes, which could include changing the value of static muts.

Notably, the host can expose some known memory locations' provenance, such that from_exposed_addr(MMIO_ADDRESS) can pick up valid provenance; I think that being valid is fairly non-controversial. If the startup state can include exposed addresses from the host in the AM ambient state, I find it more difficult to say why other host-writable AM state cannot be modified between codegen and startup.

But I still agree that the proper way to model the original use case here is almost certainly an extern static allocated by the host linker shenanigans rather than managed within the AM.

[bjorn3]

I think for as long as DATA is exported (using eg #[no_mangle]), it should be possible to write to it before main. I don't think the main function should be special cased. There is no main function for cdylib's and for cdylib's I don't see any reason to not allow writing to DATA from the user of the cdylib before the first time calling an exported function. I think a binary should be treated just like a cdylib except always exporting the main shim which calls lang_start and the user main function. If on the other hand DATA is not exported outside of the rust crate, I think it is fair to require that it has the exact same value as the initializer upon the first call to an exported function and does not get changed outside of the control of functions in the rust crate. So for example if there are no functions that modify it, ot could be moved fo read-only memory.

[digama0]

I think for as long as DATA is exported (using eg #[no_mangle]), it should be possible to write to it before main. I don't think the main function should be special cased.

I don't think this is about special casing, so much as the issue that there is no "life before main", even in the AM. As far as I am aware the AM semantics start at lang_start(). So FFI which calls lang_start() is another level of subtle compared to regular FFI.

As I mentioned before, the issue with saying that the initialization value of a static doesn't need to be in that static when the AM takes its first step (wherever we decide to put that) is that it becomes impossible for the AM to distinguish between static mut FOO: u32 = 0; and static mut FOO: u32 = 1; as declarations - these are observably equivalent and a compiler could just ignore the initializers if it wanted to. This is obviously not what we want.

The way the analogy breaks down with cdylibs is that a call to a rust function from FFI is not the AM's first step. There is still some subtlety to define what exactly was happening in the AM before the first line of rust code, but I imagine that it would be something like "arbitrary RAM-legal operations, such that the state at the call satisfies the appropriate calling convention". Since it's an unsafe interface we can freely add additional safety constraints like "and FOO should have value 0" to control the state of the machine when the function is called; for lang_start it is unclear how we could do anything like that because it's a safe function.

[RalfJung]

I think that mutable statics still need to have the values they were promised to have in the initializer unless an AM-visible write occurred to them. That write might have been caused by FFI or asm!, but if the value isn't there at lang_start then it's not clear where the value goes at all, in which case a compiler which just ignores all static mut initializers would be conforming and that's not ok.

No that's not correct. Just because the user is allowed to change the value via arbitrary means before lang_start, doesn't mean the compiler is allowed to do that.

For the same reason your "observably equivalent" argument doesn't hold up. This issue is not about giving the compiler more choice, it is purely about giving the user more choice. @CAD97 expressed it pretty well I think, it's basically a user-decided machine step that happens before the lang_start stack frame is pushed.

[digama0]

For the same reason your "observably equivalent" argument doesn't hold up. This issue is not about giving the compiler more choice, it is purely about giving the user more choice. @CAD97 expressed it pretty well I think, it's basically a user-decided machine step that happens before the lang_start stack frame is pushed.

I mean, I'm fine with building the equivalent of an empty asm! block into the language semantics for the start of the machine, but that still leaves me confused as to how this actually affects the spec. What exactly is the compiler promising when we compile a regular rust program with a static mut and no linker shenanigans? How precisely are you going to prevent a compiler that ignores initializers from being counted as conforming?

A completely user-decided machine step at program start is also a really heavy hammer. If you aren't careful, you might make it impossible for the compiler to produce a working program under those conditions since the compiler doesn't know what the user intends there.

[RalfJung]

I'm confused about why you are bringing up this question since it doesn't change with this issue at all? In the initial machine state (before that new user-defined step we are discussing) the static mut has the value given by the initializer.

But I still agree that the proper way to model the original use case here is almost certainly an extern static allocated by the host linker shenanigans rather than managed within the AM.

That seems to be hard to do without a C toolchain, or something? I don't claim to understand the usecase that triggered the discussion here.^^

[digama0]

I'm confused about why you are bringing up this question since it doesn't change with this issue at all? In the initial machine state (before that new user-defined step we are discussing) the static mut has the value given by the initializer.

This user-defined step probably needs quite some work to unpack, since it is basically what we have been calling "linker shenanigans". Somehow the spec has to interact with linker scripts, and I guess it is this code that would actually see the values of static initializers and have an opportunity to change them.

As it relates to the title question, I think the answer should still be "yes" provided there are no linker scripts (or only the default one). A program like:

static mut FOO: u8 = 0;
fn main() {
  println!("{}", unsafe { FOO });
}

with no special linking stuff should be required to print 0 (and the user should not be able to get this to print 1 by wishing on a star). Anything which changes the output of this program needs to involve changing an actual input to rustc (+ linker + dynamic linker).

[digama0]

But I still agree that the proper way to model the original use case here is almost certainly an extern static allocated by the host linker shenanigans rather than managed within the AM.

That seems to be hard to do without a C toolchain, or something? I don't claim to understand the usecase that triggered the discussion here.^^

If true, this seems like a language issue. It should be possible to declare "extern statics" with link_section using only rust and a linker script (not that I know much about the latter). I think the OP was talking about using C to allocate the static itself, which would be just as incorrect as it is in rust.

[bjorn3]

A program like:

static mut FOO: u8 = 0;
fn main() {
  println!("{}", unsafe { FOO });
}

with no special linking stuff should be required to print 0 (and the user should not be able to get this to print 1 by wishing on a star).

That is satisfied by my suggestion in #397 (comment) to require that only exported statics can be mutated by external processes. If you were to add pub or #[no_mangle] I think it should be allowed to print something else, but without both I agree that it should only print 0.

[digama0]

Ah, sorry I should have made it pub. I am talking specifically about the case where it is public but nothing is linked in which could actually do any funny business. In such a case, external processes are allowed to mutate it, but this is not the same as them having already done so. The AM sees all, and unless such a process actually runs no mutation should happen.

[bjorn3]

In case of pub for bin's specifically I can agree. But in case of #[no_mangle] you can't prove that the libc that is linked in the above program doesn't do dlsym to lookup FOO and mutate it.