Skip to content

Segmentation fault  #146

Closed
Closed
@essenciary

Description

@essenciary

I'm importing a large amount of JSON data (some 800 MB of JSON text files), through Julia, into a SQLite DB.
It seems that I consistently get a segmentation fault at every 3rd JSON file -- so I suspect it's related to the volume of queries processed. After the crash I restart the import from the last file being processed and so on - crashing again at the 3rd file.

signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
sqlite3_errmsg at /usr/lib/libsqlite3.dylib (unknown line)
sqliteerror at /Users/adrian/.julia/v0.6/SQLite/src/SQLite.jl:25
Type at /Users/adrian/.julia/v0.6/SQLite/src/SQLite.jl:50
connect at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:67
connect at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:45 [inlined]
connection at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:51 [inlined]
##escape_column_name_unmemoized at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:145
escape_column_name at /Users/adrian/.julia/v0.6/Memoize/src/Memoize.jl:109
_collect at ./array.jl:488
escape_column_name at /Users/adrian/.julia/v0.6/SearchLight/src/model_types.jl:182
#to_fully_qualified_sql_column_name#65 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:3500
unknown function (ip: 0x12c28e5e9)
#to_fully_qualified_sql_column_name at ./<missing>:0
collect_to! at ./array.jl:508
_collect at ./array.jl:489
#to_fully_qualified_sql_column_names#62 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:3489 [inlined]
#to_fully_qualified_sql_column_names at ./<missing>:0
_to_select_part at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:305
to_select_part at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:342 [inlined]
to_select_part at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:342 [inlined]
to_find_sql at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:255
to_find_sql at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:219 [inlined]
to_find_sql at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:2713 [inlined]
find_df at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:134 [inlined]
find at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:207
unknown function (ip: 0x12c284846)
#find_one_by#34 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:391 [inlined]
#find_one_by at ./<missing>:0
unknown function (ip: 0x12c2844d2)
#find_one_by#35 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:394 [inlined]
find_one_by at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:394
get at ./nullable.jl:92 [inlined]
#update_by_or_create!!#47 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:1262
unknown function (ip: 0x12c283879)
#update_by_or_create!! at ./<missing>:0
unknown function (ip: 0x12c2832d2)
#update_by_or_create!!#48 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:1278
unknown function (ip: 0x12c282d85)
run_task! at /Users/adrian/Dropbox/Projects/hotel-hound/task/ImportHotelsTask.jl:27
jl_f__apply at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_f__apply_latest at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
macro expansion at ./util.jl:237 [inlined]
#run_task#1 at /Users/adrian/.julia/v0.6/Genie/src/Toolbox.jl:31
unknown function (ip: 0x116b1533d)
#run_task at ./<missing>:0
unknown function (ip: 0x116b15096)
do_call at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
eval at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_interpret_toplevel_expr at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_toplevel_eval_in at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
eval at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jlcall_eval_18042 at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
eval_user_input at ./REPL.jl:66
unknown function (ip: 0x116b14836)
macro expansion at ./REPL.jl:97 [inlined]
#1 at ./event.jl:73
unknown function (ip: 0x116aecdbf)
start_task at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
Allocations: 322352714 (Pool: 322326627; Big: 26087); GC: 792
bin/repl: line 2: 14383 Segmentation fault: 11  julia --color=yes --depwarn=no -q -L genie.jl -- "$@"
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, skylake)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions