julep: conditional modules

[vtjnash]

currently, some modules (incorrectly and unfortunately) use Pkg.installed to decide whether it should include some additional functionality. I would like to propose the following extension to module initialization that could help simplify this situation of optional dependencies:

---

Rules:

1. During module parse time, a module can declare a dependency on another module, given by name
2. Following the closing of a module (without error), and after all __init__ functions have run, any modules that declared module B require Main.C will be (re-)loaded.

Example:

module A
  module B requires Main.C
    # code in here is part of A iff Main.C is defined as a module
    # it's execution time is defined as after the call to A.__init__ and C.__init__
    using Main.C
    import ..A.something
    something(::C.CType) = 2
  end
  # code in here is always part of A
  something(::Any) = 1
end
# Accessing A.B here would throw an UndefRef-like error, with a message that B requires C
module C
  type CType end
end
# Now A.B exists, and A.something() has extra functionality

[JeffBezanson]

Using modules to delimit optional code seems like a potentially very good idea.
It's not good for it to inherently require reloading code though.

[timholy]

I was mentally going over similar ground yesterday. Would it be reasonable to declare that interacting code can't be loaded until after the types are defined?

[ivarne]

Very similar to #2025

I do not get the crucial point here. Why does anything need to be reloaded? I would assume that most of this could be solved with a defer_require(mod::String, file::String) method that defers loading of the file until a module with name mod is loaded into Main.

[vtjnash]

The comment about reloading would only be triggered in the case where the user replaced the module. However, perhaps that condition does not need to be defined.

Using modules to define the separation allows Julia to pre-compile and cache the results (in the future). Thus, it is almost just as flexible, but it can be optimized to be fast.

[StefanKarpinski]

I wonder if parametric modules are relevant here. Consider that Main.C could be viewed as a parameter to A.B – you're declaring, in effect, a module template A.B{C} where the actual C to specialize that template on is to be provided later by some other code.

[vtjnash]

@StefanKarpinski that sounds rather clever. however, who would be responsible for instantiating the necessarily templates?

[StefanKarpinski]

Julia?

[vtjnash]

I mean, how would Julia know which templates to instantiate if the user wrote:

module X{A}
end

what is A allowed to be?

[StefanKarpinski]

That would be up to the user. If they wrote using X{Foo} then A would be Foo. This would inherently allow multiple instances of modules.

[quinnj]

What's the reasoning of using modules for optional code vs. arbitrary blocks of code? Something like:

when A
    using A
    # do stuff with module A
    # define functions, call methods, optionally wrap them in a module, etc.
elsewhen B
    using B
    # if A isn't loaded, but B is; do stuff with B
else
    # fallback if A or B can't be found; could just throw a "Need A or B error"
end

There's probably some good implementation detail to require modules, but I can imagine there are a lot of cases where you really only need a few lines, so not needing the submodule would be convenient.