Functional Programming with Streams in node.js
16 likes9,540 views
The document presents a discussion on functional programming with streams in Node.js, highlighting the benefits of using Node.js for building scalable network applications. It covers stream concepts, including readable and writable streams, as well as higher-order functions and their applications in processing data without buffering. The speaker also mentions ongoing work to port Node.js functionalities to the browser and invites collaboration with interested developers.
Functional Programming with Streams in node.js
- 1. Functional Programming with Streams in node.js
- 2. Who Am I? HP webOS Framework Team @CrabDude (Twitter, Github, etc…) Organizer BayNode & Dallas.node Author: trycatch – async try/catch in node.js stepup – step node.js control flow + trycatch tiki – in-browser package manager
- 3. The Problem VFS VFS VFS hermesClient hermesClient hermesClient hermesBuild 3rd Party API hermes hermesDeploy Ares (browser)
- 4. The Goal Don’t buffer any data Don’t write to disk Minimize latency Allow reuse of functionality (e.g., readdir) Functionality may be local or remote (e.g., getFolder)
- 5. Why node.js? “Node's goal is to provide an easy way to build scalable network programs.” – nodejs.org
- 6. node.js === JavaScript JavaScript is the future of the web, and code reuse is its unfair advantage.
- 8. Basics EventEmitter Stream Traditional streams (aka non-node.js lazy streams)
- 11. Streams “Streams are to time as arrays are to space.” – Jed Schmidt @ JSConf.eu/2010
- 12. Readable Streams Events: data, end, error, close Methods: pause, resume, end, destroy
- 13. Writable Streams Events: drain, error, close, pause, resume Methods: write, end, destroy
- 14. readable.pipe(writeable) Readable streams can be piped to writable streams (and vice-versa!) on(‘data’) => write() on(‘end’) => end() on(‘drain’) => resume() on(‘close’) => destroy() on(‘error’) => on(‘error’) on(‘pause’) => pause()
- 16. Lazy Streams Streams in functional programming…
- 17. Lazy Streams: Higher Order Functions A higher-order function is a function that does at least one of the following: Take one or more functions as input output a function
- 19. Got All That?
- 20. Hermes: Revisited FTP Dropbox Box.net Filesystem hermesFtp hermesDropbox hermesBox hermesFilesystem hermesBuild 3rd Party API hermesDeploy Ares (browser)
- 21. Still Got All That?
- 22. We need… We need to recursively zip a directory… Let’s start with abstract implementation non-specific functional versions of… Filter paths from readdir Filter directories from paths Filter files from paths Recursively return all file paths in a directory
- 24. We need… More “abstract implementation non-specific functional versions” (errr…) of… ls (aka readdir) getFile (aka readFile) getFolder (stream of gets)
- 26. We need… Yet More AINSFV of... Zip a stream of files Allow node.js streams to be piped to lazy streams pipe(): lazy stream => node.js stream pump(): node.js stream => lazy stream
- 28. Almost there!
- 29. Now we can… Zip a folder up using an arbitrary… getFile / readFile ls / readdir … And stream the response! Yeeeeehaw!
- 31. Sweeeeeet.
- 32. Now What? Since all the implementation specific functions are passed in, we can…
- 33. Port this to the browser!
- 34. Say whaaat? node-browserify by @Substack does a lot of this already Stream, Buffer, Path, etc… We need… Complete buffer implementation (browser has TypedArrays & ArrayBuffer) fs (browser has FileSystem API) zlib… inflate(), deflate(), gzip(), gunzip(), unzip()
- 35. Stay Tuned. I’m currently porting the fs and buffer modules to the browser, and eventually zlib. http://github.com/crabdude/fs-browserify http://github.com/crabdude/buffer-browserify http://github.com/crabdude/zlib-browserify
- 36. We’re HIRING! webOS Framework Team If you think this is pretty cool… Or you think I’M pretty cool… Stop by the Enyo booth or talk to me afterwards.
- 37. Thank you. [email protected] https://github.com/Gozala/streamer/blob/master/readme.js http://en.wikipedia.org/wiki/Higher-order_function https://github.com/mikeal/request http://felixge.s3.amazonaws.com/11/nodejs-streams.pdf https://github.com/substack/node-browserify Functional Programming in JavaScript: http://igstan.ro/posts/2011-05-02-understanding-monads-with- javascript.html http://ndc2011.macsimum.no/mp4/Day2%20Thursday/ Track6%201500-1600.mp4 http://drboolean.tumblr.com/ http://osteele.com/sources/javascript/functional/