Currently jemalloc is skipped if system library is specified. We should also do so when use_jemalloc is False.