proxy keeps node process alive

[zkat]

There's an issue, probably with the proxy libraries make-fetch-happen is using, where the node process is kept alive if you use m-f-h's proxying agent features. This is likely because some socket somewhere isn't closing, or the libraries are doing some keepalive thing without any timeouts.

To set up a local SOCKS proxy to test with:

1. enable SSH (in OSX, System Preferences -> Sharing -> Remote login)
2. ssh -vND 8888 "$USER@localhost"

To set up a local http(s) proxy to test with:

1. install a reverse proxy, such as Squid
2. Configure and run the proxy locally

You can use PROXY={socks,http}://localhost:8888 node to have make-fetch-happen pick it up. You should be able to hit any http or https service with m-f-h after that.

[zkat]

hint: @yyjdelete suggested trying something like JoshGlazebrook/socks@c65706d as a patch. I wonder if that's really what it would take. And why it would work.

There's an issue here that @TooTallNate's proxy projects seem to be... dead? So we may need to fork, or find something different. Perhaps it's a good idea to look into switching to https://npm.im/tunnel-agent? That's what request uses so it's definitely still maintained. I don't think it supports socks proxying though.

[pi0]

Test snippet:

const fetch = require('make-fetch-happen')
process.env.PROXY='http://icanhazip.com' // Without this line works fine
fetch('http://icanhazip.com').then(console.log).catch(console.error)

[pi0]

Tried a simple attempt to catch object leaks.

const fetch = require('make-fetch-happen')
const heapdump = require('heapdump')

const dump = () => new Promise((resolve,reject) => heapdump.writeSnapshot((err, filename) => {  
  if(err) reject(err);
  resolve();
  console.log('dump written to', filename)
}))

const test = () => fetch('http://icanhazip.com').then(console.log).catch(console.error)

test().then(dump).then(() => {
	process.env.PROXY = 'http://icanhazip.com'
	test().then(dump)
});

You can compare two heap dumps. Seems HTTPProxyAgent is keeping process open.

[TooTallNate]

There's an issue here that @TooTallNate's proxy projects seem to be... dead?

Come again? Is there a pull request I need to address?

[zkat]

@TooTallNate oh! Great! Sorry, I saw that it'd been a few years since a release and didn't wanna bother you. I might end up having a pull request for you then -- just gotta figure out why using the proxies is keeping the node process alive 👍

[TooTallNate]

Sounds great! Ya all my proxy modules have mostly been completed, hence the small amount of activity. But I'm here if you need help or find bugs.

[zkat]

@TooTallNate do you have any idea where to look for why this particular bug is happening? It's definitely only when going through one of these proxies.

[TooTallNate]

process._getActiveHandles() and/or process._getActiveRequests() might shed some light, but in general I've had a hard time making heads or tails of the results in the past.

If you can put together a simplified test case I can take a look.

[zkat]

Will do! I'll push that onto the stack :)

[yyjdelete]

@zkat @TooTallNate
JoshGlazebrook/socks@c65706d is for another issue that socks-proxy can't get response for http/ws. Tested with the example code of node-socks-proxy-agent and node 7.9

And found there is already an PR for http as TooTallNate/node-socks-proxy-agent#12.

[zkat]

I'm having a hard time coming up with a repro step isolated from npm itself :( I'll have to take a look at this much later, then, unless someone else picks it up. I've really no clue right now.

[zkat]

Heads-up: The npm-side repro case is going away as of the latest beta 'cause I worked around the dangling socket issue (by calling process.exit() directly when commands finish, whereas the CLI would usually let the process exit by itself). This is still an issue for make-fetch-happen itself, though.