multi-threaded (@threads) dotcall/broadcast?

[stevengj]

It would be nice to be able to put @threads in front of a dot call, e.g. @threads X .= f.(Y), and have it call a multi-threaded version of broadcast (which would assume f is thread-safe).

[vchuravy]

I don't think this is something that should go into the @threads macro (since that is already too complex). @SimonDanisch has been working on doing this based on dispatch to a special array type in https://github.com/JuliaGPU/GPUArrays.jl/blob/master/src/backends/julia/julia.jl

The beauty of Simon's approach is that it is general and works for distributed arrays, GPU arrays, and native arrays. I consed that it might feel a bit unnatural for native arrays, and I wouldn't be opposed to add a macro that transforms X .= f.(Y) into the threaded dispatch version but I don't think that necessarily needs to be in Base.

[stevengj]

Regardless of the name of the macro, it would be nice to have something that just involved a decorator and didn't involve re-allocating or wrapping all of your arrays in some other type. This is the big distinction between threads and GPUs or distributed-memory — with threads, you don't need to decide in advance to put your data on a GPU or in another process.

[SimonDanisch]

I'd suggest having a macro similar to fastmath, that replaces the broadcast calls with e.g. threaded_broadcast. Would be nice to share the implementation between what I have and Base :)
I tried to create a minimal working version, but it seems like expand needs some extra attention to get the broadcast back.
Blocks of code expand into a Expr(:thunk, Toplevel LambdaInfo thunk), so I had to iterate through the sub expressions to make things easier.

[StefanKarpinski]

Would it be too crazy for broadcast to be implicitly parallel once threading is stable?

[martinholters]

Already, broadcast makes no guaranties concerning the order or exact number of times the given function is called (think sparse matrices), so passing non-pure functions is questionable anyway. That would be in favor of implicit parallelism. Con: When broadcasting over a collection with only few items, the complexity of the given function will decide whether it is worth any threading overhead, which cannot be easily decided by broadcast itself.

[ChrisRackauckas]

Could this get a 1.0 milestone? At least some kind of macro would be very useful, if not implicit parallelism (or a mixture: default implicit parallelism, which can be overridden with a macro). Since broadcasting is such a cool feature, this would really complete the story.

[Sacha0]

cc @lkuper

[vchuravy]

If anyone wants to tackle this (developing this outside of base at first is probably a good idea) my roadmap/ideas would be.

1. Thin wrapper type around Arrays
2. Macro that automates this wrapping
3. Co-existence with other types of parallelism. We currently have DistributedArrays as a seconder user of broadcast
   The nice thing about using a thin wrapper is that you can have DArray{Threaded{Array}} to combine distributed and threaded parallelism.
4. What happens in the case of mixed accelerated array types? e.g. A GPUArray meets a threaded array?
5. NUMA-aware parallelism. Communication across NUMA nodes is still quite expensive and memory should probably be pinned to the NUMA group that the threads belong to.

For other inspiration take a look at parallel collections in Scala http://docs.scala-lang.org/overviews/parallel-collections/overview.html (which were the inspiration for Java 8).

[SimonDanisch]

Just as an info: I'm already getting good speed ups out of @threads in my broadcast implementation in GPUArrays ;) It's still a bit crappy, but if I have some time I could polish it up and I can make a first PR for a tbroadcast or however we call it ;)

[RoyiAvital]

@ChrisRackauckas ,

I like the idea of letting the user choose.
Should be something like a Macro (Or the most efficient way to do it) with 3 states:

1. Auto (Default) - Adding heuristic to set On or OFF.
2. ON.
3. OFF.

Thank You.

[ChrisRackauckas]

It would be interesting if the heuristic could be applied to arbitrary loops via some macro as well.