Non-blocking I/O, Event loops and node.js
63 likes22,889 views
This 15 minute presentation discusses non-blocking I/O, event loops, and Node.js. It builds on previous work by Ryan Dahl, explaining how threads can be expensive due to context switching and memory usage, and how Node.js uses an event-driven, non-blocking model to avoid these costs. Code examples demonstrate getting and printing a policy object, handling HTTP requests asynchronously without blocking additional connections, and using callbacks to chain asynchronous actions together.
![% node hello.js
% ab -n 1000 -c 200 http://127.0.0.1:8080/
...
Time per request: 2010.070 [ms] (mean)
min mean[+/-sd] median max
Connect: 0 2 1.9 1 8
Processing: 2001 2006 2.8 2005 2013
Waiting: 2000 2003 2.0 2003 2009
Total: 2002 2008 3.5 2006 2015
Percentage of the requests served within a certain time (ms)
50% 2006
66% 2008
75% 2012
80% 2013
90% 2013
95% 2014
98% 2014
99% 2014
100% 2015 (longest request)](https://image.slidesharecdn.com/node-js-english-091210145620-phpapp01/85/Non-blocking-I-O-Event-loops-and-node-js-42-320.jpg)
![var web = require('./web');
web.server(function (route) {
route.get("^/([a-z]+)$", function(parms, req, res) {
res.sendHeader(200, {'Content-Type':'text/plain'});
res.sendBody("Hello " + parms[0]);
res.finish();
});
}).listen(8080);](https://image.slidesharecdn.com/node-js-english-091210145620-phpapp01/85/Non-blocking-I-O-Event-loops-and-node-js-44-320.jpg)
![% node web.test.js
% ab -n 10000 -c 250 http://127.0.0.1:8080/marcus
...
Requests per second: 4851.93 [#/sec] (mean)
Time per request: 51.526 [ms] (mean)
Time per request: 0.206 [ms] (mean, across all c requests)
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 14 111.8 1 999
Processing: 4 35 15.8 31 103
Waiting: 4 35 15.8 30 103
Total: 14 49 112.5 32 1035
Percentage of the requests served within a certain time (ms)
50% 32
66% 34
75% 35
80% 36
90% 57
95% 92
98% 102
99% 986
100% 1035 (longest request)](https://image.slidesharecdn.com/node-js-english-091210145620-phpapp01/85/Non-blocking-I-O-Event-loops-and-node-js-45-320.jpg)
![var web = require('./web'),
posix = require('posix');
web.server(function (route) {
route.get("^/([a-z]+)$", function(parms, req, res) {
posix.cat(parms[0]).addCallback(function(text) {
res.sendHeader(200, {'Content-Type':'text/plain'});
res.sendBody(text);
res.finish();
});
});
}).listen(8080);](https://image.slidesharecdn.com/node-js-english-091210145620-phpapp01/85/Non-blocking-I-O-Event-loops-and-node-js-46-320.jpg)
Non-blocking I/O, Event loops and node.js
- 1. This is a 15 minute presentation I did at work recently about threads, polling and node.js. It rests heavily on the shoulders of giants, especially Ryan Dahls original node.js talk at jsconf.eu.
- 2. non-blocking I/O, event loops, javascript and node.js.
- 3. Threads suck
- 4. Why do we use threads?
- 5. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
- 6. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
- 7. Policy p = cmServer.getPolicy(...); Blocking implies multiple execution stacks. Where else to go? System.out.println(p.getContentId().toString());
- 8. How to deal with blocking? Processes No shared memory Heavy OS Threads Can share memory Relatively cheap Green threads / Co-routines
- 9. Are threads cheaper than processes?
- 10. HTTP request latency; glibc 2.3.2 on Linux 2.6 HTTP-anropslatens; glibc 2.3.2 på Linux 2.6 1 thread per call 1 process per call http://bulk.fefe.de/scalable-networking.pdf Open connections
- 11. HTTP request latency; glibc 2.3.2 on Linux 2.6 1 thread per call 1 process per call Which is the reason why we use thread pools! Open connections
- 12. Are threads cheap at all?
- 13. nginx vs apache http://blog.webfaction.com/a-little-holiday-present
- 14. nginx vs apache http://blog.webfaction.com/a-little-holiday-present ∆ = Apaches context switching
- 15. lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
- 16. lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 On a MBP, OS X 10.6 Context switch around 65 µs
- 17. 65 µs x 2 x 4000 = 520 ms
- 19. > ulimit -a | grep stack stack size (kbytes, -s) 8192
- 20. ...but at least 512+1 kb For OS X, source: ADC
- 21. nginx vs apache http://blog.webfaction.com/a-little-holiday-present
- 22. nginx vs apache http://blog.webfaction.com/a-little-holiday-present Difference: Threads and processes cost.
- 24. The cost of I/O L1-cache 3 cycles L2-cache 14 cycles RAM 250 cycles Disk 41 000 000 cycles Network 240 000 000 cycles
- 25. The weight of I/O
- 26. The weight of I/O L1-cache A big squirrel
- 27. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat
- 28. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically
- 29. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales
- 30. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales Network Belarus yearly wheat import
- 31. Servers pretty I/O-intensive = Squirrelʼs burried in whales fat and wheat
- 32. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
- 33. cmServer.getPolicy(function (Policy p) { System.out.println( p.getContentId().toString() ); });
- 34. cmServer.getPolicy(function (Policy p) { System.out.println( Sleeps until p’s available p.getContentId().toString() ); });
- 35. cmServer.getPolicy(function (Policy p) { System.out.println( Sover tills dess p finns p.getContentId().toString() ); }); // And down here we can do other stuff
- 36. Non-blocking I/O... All code is run in a single thread Functions listen on events, act and sleep
- 37. ...is pretty demanding All I/O has to be non-blocking Callbacks are pretty ugly without HOF
- 38. V8 + node.js
- 39. “a purely evented, non-blocking infrastructure to script highly concurrent programs.”
- 40. V8 Chromes JS engine JS Built to be event driven node.js Queues built on /dev/epoll amongst other things
- 41. var http = require('http'); http.createServer(function (req, res) { setTimeout(function () { res.sendHeader(200, {'Content-Type': 'text/plain'}); res.sendBody('Hello World'); res.finish(); }, 2000); }).listen(8080);
- 42. % node hello.js % ab -n 1000 -c 200 http://127.0.0.1:8080/ ... Time per request: 2010.070 [ms] (mean) min mean[+/-sd] median max Connect: 0 2 1.9 1 8 Processing: 2001 2006 2.8 2005 2013 Waiting: 2000 2003 2.0 2003 2009 Total: 2002 2008 3.5 2006 2015 Percentage of the requests served within a certain time (ms) 50% 2006 66% 2008 75% 2012 80% 2013 90% 2013 95% 2014 98% 2014 99% 2014 100% 2015 (longest request)
- 43. var stat = require('posix').stat, puts = require('sys').puts; var promise = stat('/etc/passwd'); promise.addCallback(function (s) { puts('modified: ' + s.mtime); });
- 44. var web = require('./web'); web.server(function (route) { route.get("^/([a-z]+)$", function(parms, req, res) { res.sendHeader(200, {'Content-Type':'text/plain'}); res.sendBody("Hello " + parms[0]); res.finish(); }); }).listen(8080);
- 45. % node web.test.js % ab -n 10000 -c 250 http://127.0.0.1:8080/marcus ... Requests per second: 4851.93 [#/sec] (mean) Time per request: 51.526 [ms] (mean) Time per request: 0.206 [ms] (mean, across all c requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 14 111.8 1 999 Processing: 4 35 15.8 31 103 Waiting: 4 35 15.8 30 103 Total: 14 49 112.5 32 1035 Percentage of the requests served within a certain time (ms) 50% 32 66% 34 75% 35 80% 36 90% 57 95% 92 98% 102 99% 986 100% 1035 (longest request)
- 46. var web = require('./web'), posix = require('posix'); web.server(function (route) { route.get("^/([a-z]+)$", function(parms, req, res) { posix.cat(parms[0]).addCallback(function(text) { res.sendHeader(200, {'Content-Type':'text/plain'}); res.sendBody(text); res.finish(); }); }); }).listen(8080);
- 47. throughput 80 74 68 62 56 50 1 5 13 25 50 100 250
- 48. HEALTH
- 49. Distributed ab One configuration, multiple nodes REST interface Streams JSON over HTTP Decent performance
- 50. demo!