Julia won't pull from https behind a corporate proxy

[mogthesprog]

Hi,

I've been experiencing some dificulty using Pkg.init() and Pkg.add() behind a corporate proxy. See below...

[31510:31508 - 0:1029] 01:16:24 [root@gb-slo-svb-0139:o +1] /opt $ julia
Version 0.5.0-dev
x86_64-redhat-linux
julia> Pkg.init()
INFO: Initializing package repository /opt/julia/lib/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, Failed to connect to github.com: Connection refused)
 [inlined code] from libgit2/error.jl:98
 in clone at libgit2/repository.jl:189
 in __clone#364__ at libgit2.jl:310
 in anonymous at pkg/dir.jl:49
 in cd at file.jl:47
 in init at pkg/dir.jl:47
 in init at pkg.jl:22
 in eval at ./boot.jl:263

julia>
[31510:31508 - 0:1030] 01:16:32 [root@gb-slo-svb-0139:o +1] /opt $ cd /tmp/
[31510:31508 - 0:1031] 01:16:38 [root@gb-slo-svb-0139:o +1] /tmp $ git clone https://github.com/JuliaLang/METADATA.jl
Cloning into 'METADATA.jl'...
remote: Counting objects: 52190, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 52190 (delta 3), reused 0 (delta 0), pack-reused 52178
Receiving objects: 100% (52190/52190), 6.68 MiB | 3.47 MiB/s, done.
Resolving deltas: 100% (13081/13081), done.
[31510:31508 - 0:1032] 01:16:52 [root@gb-slo-svb-0139:o +1] /tmp $

I've tried the git config --global https.proxy <comeproxy> and also git config --global url."git://".insteadOf https://" , i've tried configuring environment variables in the shell. I've even tried issuing Pkg.init() with the full url, replacing https with git or http. I'm a bit stuck at the moment. I can't see any reason why this shouldn't work behind our proxy.

Any ideas guys?

Cheers,

Morgan

[lobingera]

looking at the above i get the impression, you are using Pkg with libgit2, so there is a possiblity that your configuration of the command line git (calling git config) is not applied

[mogthesprog]

Thanks for the quick reply :)

Yeah I think so too... Any idea how I can change the libgit2 calls in julia? is there a Pkg method for changing the config?

[lobingera]

There seems to be some config in Libgit2 and it might be a idea to connect to this
#13879 where the topic of Libgit2 using the 'global' git config was touched.

[tkelman]

duplicate of #13472, see #13952 for a workaround. Or use release-0.4, you should only use master for submitting pull requests.

[axsk]

I am having a similar problem with master on my cluster at work:
ERROR: GitError(Code:ERROR, Class:OS, Failed to connect to github.com: No route to host)

On 0.4 the git config trick fixed the issue.
I would assume that this should not be necessary since #11312, so maybe the git:// vs https:// issue is not even the problem here?

Edit: I also tried @vtjnash's suggestion from #13952 , USE_SYSTEM_LIBGIT2=1 followed by a make clear && make but this did not help either.

[tkelman]

it's make clean not make clear, though you may need a make cleanall and/or make -C deps distclean-libgit2 to get the dependency change to propagate to the build result

[ViralBShah]

Should be fixed in #17783. Please try the latest nightlies tomorrow.

[gersonjferreira]

Is this fixed? Version 0.5.0 is now on the Ubuntu PPA and I got in trouble yesterday because I'm behind a proxy. So I had to revert to 0.4.5 version on the official Ubuntu repos and lock the version. I'll keep it lock until the PPA version gets the proxy fixed.

[ViralBShah]

Please do not use ubuntu PPAs and use the generic linux binaries instead.

[ViralBShah]

@staticfloat Do you think on the ubuntu PPA page, we can put a word about using the generic linux binaries for such cases?

Alternately, perhaps our custom PPAs could repackage all the necessary libraries and not use the system ones.

[gersonjferreira]

@ViralBShah , I don't understand this. You do announce the PPA on the Downloads section of the Julia webpage under Platform Specific Instructions. If the PPAs are not meant for the general user, this (and the reason) should be explicit somewhere. Or simply change the PPA name to Debian-like "testing" and "unstable", so that it is clear that only the generic binaries are the ones considered stable. In any case, I would like to have a stable PPA distribution available to help me keep it updated. Or you could try snap packaging... ?

[StefanKarpinski]

Unfortunately, issues with standard configurations of many libraries, combined with distro policies requiring the use of shared versions of libraries when they exist, make it essentially impossible to have a fully working Julia in most distros, including Debian and Ubuntu. We'll continue to work towards improving this situation, but don't expect it to be smooth in the near future.

[StefanKarpinski]

To get a sense of how many libraries need patches to work correctly, check out this directory of patches that we apply when building Julia's dependencies from source:

• https://github.com/JuliaLang/julia/tree/master/deps/patches

We do upstream these patches, but it takes a while for projects to review them, let alone for the fixes to make it into distros. Upstream projects also vary in how willing they are to take patches.