Skip to content

getindex is very slow for adjoints of sparse arrays #36

Open
@Pbellive

Description

@Pbellive

Calling getindex on an adjoint of a sparse array falls back (tested on 1.2 and on master built from source today, both on Linux) to a general AbstractArray method, making it unusable for non-scalar indexing with large sparse arrays. Consider the following example:

julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i9-9820X CPU @ 3.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
julia> using SparseArrays
julia> n = 100000;m = 10000; nf = 25000;

julia> a = sprandn(n,m, 0.0005);

julia> bt = sprandn(m,n, 0.0005)';

julia> @time c = a[1:nf, :];
  0.091680 seconds (208.92 k allocations: 12.387 MiB)

julia> @time c = a[1:nf, :];
  0.007136 seconds (12 allocations: 1.976 MiB)

julia> @time c = bt[1:nf, :];
  9.522862 seconds (444.40 k allocations: 37.878 MiB, 0.55% gc time)

julia> @time c = bt[1:nf, :];
  9.280306 seconds (17 allocations: 15.308 MiB)

This was discussed last year on Discourse. That thread contains another example of the issue. A PR with a fix for sparse vectors (JuliaLang/julia#28654) was opened not long after that discussion but seems to have been abandoned. I don't have time to resurrect the PR at the moment but I thought it would be good to have an issue open to track this. Is fixing the issue on anyone's radar at the moment?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions