OOM on cyclical network

[evetion]

Over at https://github.com/Deltares/Ribasim.jl, we're using MTK to build large hydrological models. What we see is that on some models, especially those that form a cyclic graph (but not on all of them), we run out of memory in the following code:

https://github.com/SciML/ModelingToolkit.jl/blob/a9dc1fc8d26b02e183ce5cc67e7471aae019d14b/src/structural_transformation/codegen.jl#L116...L121

The size of next_assignments keeps growing for each loop, hitting 1e9 for a system with 1290 states. Smaller models, even those with many cyclic parts converge much quicker. This behavior has also improved in the last months, now only running out of memory on the largest models.

With this serialized structurally simplified sim here to deserialize, we have the following MWE for the current MTK master branch, Julia 1.9.

using Serialization
using ModelingToolkit
using DataInterpolations
sim = deserialize("cached_simplified_sim.jls")
tspan = (1.5463008e9, 1.5513984e9)
prob = ODAEProblem(sim, [], tspan; sparse = true)  # OOM

@visr

[evetion]

An overview of the model, with names of the nodes. Nodes with a black background form a cycle, of which there are two.

[ChrisRackauckas]

We now have our first benchmarks out which shows JuliaSimCompiler solves the problem:

https://docs.sciml.ai/SciMLBenchmarksOutput/dev/ModelingToolkit/ThermalFluid/

So closing this as it's just inherent to the MTK compiler's form, with trade-offs between the two. I recommend using the IR system for large problems.

It would be nice to get this as a benchmark, though we would need to not have a serialized version if we're to test the codegen elements as well. I would entertain that over at SciMLBenchmarks.jl: we're looking for large models to test!

[visr]

Thanks for the update. We don't have any recent large models from https://github.com/Deltares/Ribasim since we moved to directly using OrdinaryDiffEq.jl, since JuliaSimCompiler wasn't an option for us.