Filtering a copy modifies original DataFrame (and arrays if applicable)

[kescobo]

This behavior of filter!() caught me quite off-guard and took a while to debug:

julia> using DataFrames

julia> a = rand(10); b = rand(10); c = rand(10);

julia> df = DataFrame(x = a, y = b, z = c);

julia> df2 = copy(df[[2,3]]);

julia> filter!(row->row[:y] > 0.5, df2);

julia> df
10×3 DataFrame
│ Row │ x         │ y        │ z          │
│     │ Float64   │ Float64  │ Float64    │
├─────┼───────────┼──────────┼────────────┤
│ 1   │ 0.0403589 │ 0.984833 │ 0.320714   │
│ 2   │ 0.221344  │ 0.949222 │ 0.0521154  │
│ 3   │ 0.245672  │ 0.659739 │ 0.400802   │
│ 4   │ 0.463964  │ 0.87275  │ 0.610431   │
│ 5   │ 0.0797766 │ 0.841099 │ 0.650547   │
│ 6   │ 0.0193848 │ 0.991481 │ 0.00325831 │
│ 7   │ 0.929879  │ #undef   │ #undef     │
│ 8   │ 0.611218  │ #undef   │ #undef     │
│ 9   │ 0.509054  │ #undef   │ #undef     │
│ 10  │ 0.381528  │ #undef   │ #undef     │

julia> b
6-element Array{Float64,1}:
 0.9848328699008353
 0.9492215902675882
 0.6597386640528373
 0.8727504349162607
 0.8410991929844753
 0.9914814768586502

I expected filter!() on a DataFrame to only filter out rows on that DataFrame, not to modify the underlying arrays (which are shared across copies). The fix is to use deepcopy() instead.

I'm not sure if this is something that should be "fixed", I think I understand the reasoning behind it, but at the very least I thought I'd open the issue for future stumblers on.

I think this is related to #1695

[bkamins]

This is exactly #1695 😄.

There were several possible solutions, my current thinking (and I guess this is something @nalimilan tends to prefer) is that:

1. DataFrames are allowed to share underlying vectors (so we drop column "ownership" concept)
2. all ! functions do not modify underlying vectors but allocate new vectors if needed
3. the exception is setindex!, which may modify underlying vectors in some cases (we should add a clear documentation of this behavior)
4. We add functions that are allowed to modify underlying vectors in-place, but they end with !!; push!! and append!! the most important functions here
5. we add copycolumns kwarg to some functions (e.g. DataFrame, constructor, hcat) to force copying of columns on operation (by default it will be false) - this is an optional feature, but can be handy in some situations.

[kescobo]

That makes a lot of sense to me! I especially like the idea of !! functions to be really explicit that they're doing something to pay attention to. I guess your #2 would fix this issue.

[bkamins]

Additionally (these changes are complementing):

deprecate df[col] and df[cols] in favor of getcol(df, col) and select(df, cols)

In the future df[idx] will select a row and df[idxs] will select rows (but we should probably wait with this at least 6 months)

[nalimilan]

Closing in favor of #1695.