Performance issue w/ getiterator(::CSVFile)

[quinnj]

I noticed a weird performance issue when playing w/ CSVFile; note the following (this is all after having run the functions once):

julia> @time f = load("/Users/jacobquinn/Downloads/randoms.csv");
  0.000297 seconds (84 allocations: 5.234 KiB)

julia> @time f = IteratorInterfaceExtensions.getiterator(load("/Users/jacobquinn/Downloads/randoms.csv"));
  6.374569 seconds (36.76 M allocations: 810.608 MiB, 2.06% gc time)

julia> @time DataFrame(f);
  0.070821 seconds (532.63 k allocations: 34.434 MiB, 9.99% gc time)

The getiterator call takes that long each time it's called. I haven't had a chance to dig in further, but wanted to report.

[quinnj]

(by comparison, CSV.jl timings:

julia> @time ct = CSV.File("/Users/jacobquinn/Downloads/randoms.csv") |> DataFrame;
  0.189268 seconds (1.25 M allocations: 35.464 MiB, 8.97% gc time)
```)

[davidanthoff]

Hm, would be interesting to see what a plain TextParse.jl loading of that file gives, could you just try

using TextParse

@time csvread(filename)

? That would let us isolate whether this is a case where TextParse.jl is just slow, or whether I messed things up in the CSVFiles.jl part of things.

[davidanthoff]

Oh, and could you post the file online?

[quinnj]

Looks like it is perhaps TextParse:

julia> using TextParse

julia> @time csvread("/Users/jacobquinn/Downloads/randoms.csv");
 14.448354 seconds (51.23 M allocations: 1.511 GiB, 3.49% gc time)

julia> @time f = csvread("/Users/jacobquinn/Downloads/randoms.csv");
  6.629736 seconds (36.34 M allocations: 804.135 MiB, 2.15% gc time)

File is here: https://raw.githubusercontent.com/JuliaData/DataStreamsIntegrationTests.jl/master/test/randoms.csv

[davidanthoff]

I think that is a regression of TextParse.jl on 1.0. Here is what I get on julia 0.6:

julia> @time csvread("randoms.csv");
  0.089883 seconds (142.01 k allocations: 25.989 MiB, 6.97% gc time)

and on 1.0

julia> @time csvread("randoms.csv");
  5.367225 seconds (36.34 M allocations: 804.135 MiB, 2.68% gc time)