Closed
Description
I'm importing a large amount of JSON data (some 800 MB of JSON text files), through Julia, into a SQLite DB.
It seems that I consistently get a segmentation fault at every 3rd JSON file -- so I suspect it's related to the volume of queries processed. After the crash I restart the import from the last file being processed and so on - crashing again at the 3rd file.
signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
sqlite3_errmsg at /usr/lib/libsqlite3.dylib (unknown line)
sqliteerror at /Users/adrian/.julia/v0.6/SQLite/src/SQLite.jl:25
Type at /Users/adrian/.julia/v0.6/SQLite/src/SQLite.jl:50
connect at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:67
connect at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:45 [inlined]
connection at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:51 [inlined]
##escape_column_name_unmemoized at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:145
escape_column_name at /Users/adrian/.julia/v0.6/Memoize/src/Memoize.jl:109
_collect at ./array.jl:488
escape_column_name at /Users/adrian/.julia/v0.6/SearchLight/src/model_types.jl:182
#to_fully_qualified_sql_column_name#65 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:3500
unknown function (ip: 0x12c28e5e9)
#to_fully_qualified_sql_column_name at ./<missing>:0
collect_to! at ./array.jl:508
_collect at ./array.jl:489
#to_fully_qualified_sql_column_names#62 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:3489 [inlined]
#to_fully_qualified_sql_column_names at ./<missing>:0
_to_select_part at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:305
to_select_part at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:342 [inlined]
to_select_part at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:342 [inlined]
to_find_sql at /Users/adrian/.julia/v0.6/SearchLight/src/database_adapters/SQLiteDatabaseAdapter.jl:255
to_find_sql at /Users/adrian/.julia/v0.6/SearchLight/src/Database.jl:219 [inlined]
to_find_sql at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:2713 [inlined]
find_df at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:134 [inlined]
find at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:207
unknown function (ip: 0x12c284846)
#find_one_by#34 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:391 [inlined]
#find_one_by at ./<missing>:0
unknown function (ip: 0x12c2844d2)
#find_one_by#35 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:394 [inlined]
find_one_by at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:394
get at ./nullable.jl:92 [inlined]
#update_by_or_create!!#47 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:1262
unknown function (ip: 0x12c283879)
#update_by_or_create!! at ./<missing>:0
unknown function (ip: 0x12c2832d2)
#update_by_or_create!!#48 at /Users/adrian/.julia/v0.6/SearchLight/src/SearchLight.jl:1278
unknown function (ip: 0x12c282d85)
run_task! at /Users/adrian/Dropbox/Projects/hotel-hound/task/ImportHotelsTask.jl:27
jl_f__apply at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_f__apply_latest at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
macro expansion at ./util.jl:237 [inlined]
#run_task#1 at /Users/adrian/.julia/v0.6/Genie/src/Toolbox.jl:31
unknown function (ip: 0x116b1533d)
#run_task at ./<missing>:0
unknown function (ip: 0x116b15096)
do_call at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
eval at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_interpret_toplevel_expr at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
jl_toplevel_eval_in at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
eval at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jlcall_eval_18042 at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
eval_user_input at ./REPL.jl:66
unknown function (ip: 0x116b14836)
macro expansion at ./REPL.jl:97 [inlined]
#1 at ./event.jl:73
unknown function (ip: 0x116aecdbf)
start_task at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib (unknown line)
Allocations: 322352714 (Pool: 322326627; Big: 26087); GC: 792
bin/repl: line 2: 14383 Segmentation fault: 11 julia --color=yes --depwarn=no -q -L genie.jl -- "$@"
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
OS: macOS (x86_64-apple-darwin14.5.0)
CPU: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
LAPACK: libopenblas64_
LIBM: libopenlibm
LLVM: libLLVM-3.9.1 (ORCJIT, skylake)
Metadata
Metadata
Assignees
Labels
No labels