SlideShare a Scribd company logo
Functional Programming with
     Streams in node.js
Who Am I?

  HP webOS Framework Team

  @CrabDude (Twitter, Github, etc…)

  Organizer BayNode & Dallas.node

  Author:

    trycatch – async try/catch in node.js

    stepup – step node.js control flow + trycatch

    tiki – in-browser package manager
The Problem
     VFS         VFS           VFS

hermesClient hermesClient hermesClient

              hermesBuild        3rd Party API


    hermes      hermesDeploy



   Ares
  (browser)
The Goal

  Don’t buffer any data

  Don’t write to disk

  Minimize latency

  Allow reuse of functionality (e.g., readdir)

  Functionality may be local or remote (e.g., getFolder)
Why node.js?

“Node's goal is to provide an easy way to build
  scalable network programs.”
                                    – nodejs.org
node.js === JavaScript

JavaScript is the future of the web,
  and code reuse is its unfair advantage.
Functional Programming with Streams in node.js
Basics

  EventEmitter

  Stream

  Traditional streams (aka non-node.js lazy streams)
EventEmitter: Example
EventEmitter: Extend
Streams

“Streams are to time as arrays are to space.”
              – Jed Schmidt @ JSConf.eu/2010
Readable Streams

  Events: data, end, error, close

  Methods: pause, resume, end, destroy
Writable Streams

  Events: drain, error, close, pause, resume

  Methods: write, end, destroy
readable.pipe(writeable)

Readable streams can be piped to writable streams
  (and vice-versa!)

  on(‘data’) => write()

  on(‘end’) => end()

  on(‘drain’) => resume()

  on(‘close’) => destroy()

  on(‘error’) => on(‘error’)

  on(‘pause’) => pause()
readable.pipe(writeable): Example
Lazy Streams

Streams in functional programming…
Lazy Streams: Higher Order Functions

A higher-order function is a function that does at least
  one of the following:

  Take one or more functions as input
  output a function
Lazy Streams: Lists
Got All That?
Hermes: Revisited
     FTP        Dropbox        Box.net   Filesystem

hermesFtp hermesDropbox hermesBox hermesFilesystem

               hermesBuild                 3rd Party API

                          hermesDeploy

     Ares
   (browser)
Still Got All That?
We need…

We need to recursively zip a directory…

  Let’s start with abstract implementation non-specific
   functional versions of…

    Filter paths from readdir

    Filter directories from paths

    Filter files from paths

    Recursively return all file paths in a directory
Functional Programming with Streams in node.js
We need…

  More “abstract implementation non-specific functional
   versions” (errr…) of…

    ls (aka readdir)

    getFile (aka readFile)

    getFolder (stream of gets)
Functional Programming with Streams in node.js
We need…

  Yet More AINSFV of...

    Zip a stream of files

    Allow node.js streams to be piped to lazy streams

       pipe(): lazy stream => node.js stream

       pump(): node.js stream => lazy stream
Functional Programming with Streams in node.js
Almost there!
Now we can…

Zip a folder up using an arbitrary…

  getFile / readFile

  ls / readdir



… And stream the response! Yeeeeehaw!
Functional Programming with Streams in node.js
Sweeeeeet.
Now What?

Since all the implementation specific functions
  are passed in, we can…
Port this to the browser!
Say whaaat?

node-browserify by @Substack does a lot of this already

    Stream, Buffer, Path, etc…

We need…

    Complete buffer implementation (browser has
     TypedArrays & ArrayBuffer)

    fs (browser has FileSystem API)

    zlib…

       inflate(), deflate(), gzip(), gunzip(), unzip()
Stay Tuned.

I’m currently porting the fs and buffer modules to
  the browser, and eventually zlib.

http://github.com/crabdude/fs-browserify
http://github.com/crabdude/buffer-browserify
http://github.com/crabdude/zlib-browserify
We’re HIRING!
            webOS Framework Team
If you think this is pretty cool…
   Or you think I’M pretty cool…

Stop by the Enyo booth or talk to me afterwards.
Thank you.

        adam.crabtree@palm.com
https://github.com/Gozala/streamer/blob/master/readme.js
http://en.wikipedia.org/wiki/Higher-order_function
https://github.com/mikeal/request
http://felixge.s3.amazonaws.com/11/nodejs-streams.pdf
https://github.com/substack/node-browserify

Functional Programming in JavaScript:
http://igstan.ro/posts/2011-05-02-understanding-monads-with-
  javascript.html
http://ndc2011.macsimum.no/mp4/Day2%20Thursday/
  Track6%201500-1600.mp4
http://drboolean.tumblr.com/
http://osteele.com/sources/javascript/functional/
Ad

Recommended

Functional node.js
Functional node.js
Carob Cherub
 
Node.js: CAMTA Presentation
Node.js: CAMTA Presentation
Rob Tweed
 
Introduction to performance tuning perl web applications
Introduction to performance tuning perl web applications
Perrin Harkins
 
Os Fitzpatrick Sussman Swp
Os Fitzpatrick Sussman Swp
oscon2007
 
Os Alrubaie
Os Alrubaie
oscon2007
 
Introduction to NodeJS with LOLCats
Introduction to NodeJS with LOLCats
Derek Anderson
 
Background Processing - PyCon MY 2015
Background Processing - PyCon MY 2015
Kok Hoor Chew
 
Node.js for Rubists
Node.js for Rubists
Sagiv Ofek
 
Applied Shell Scripting - stills to time-lapse
Applied Shell Scripting - stills to time-lapse
Shoaib Sufi
 
Алексей Петров "PHP at Scale: Knowing enough to be dangerous!"
Алексей Петров "PHP at Scale: Knowing enough to be dangerous!"
Fwdays
 
Александр Махомет "Beyond the code или как мониторить ваш PHP сайт"
Александр Махомет "Beyond the code или как мониторить ваш PHP сайт"
Fwdays
 
Server side scripting smack down - Node.js vs PHP
Server side scripting smack down - Node.js vs PHP
Marc Gear
 
Webdevcon Keynote hh-2012-09-18
Webdevcon Keynote hh-2012-09-18
Pierre Joye
 
Basic testing with selenium
Basic testing with selenium
Søren Lund
 
HTTP::Parser::XS - writing a fast & secure XS module
HTTP::Parser::XS - writing a fast & secure XS module
Kazuho Oku
 
Chenli linux-kerne-community
Chenli linux-kerne-community
力 陈
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
Tom Croucher
 
Searching CPAN Offline
Searching CPAN Offline
acme
 
Nginx وب سروری برای تمام فصول
Nginx وب سروری برای تمام فصول
efazati
 
A slightly advanced introduction to node.js
A slightly advanced introduction to node.js
Sudar Muthu
 
Web Development with Python and Django
Web Development with Python and Django
Michael Pirnat
 
Great Tools Heavily Used In Japan, You Don't Know.
Great Tools Heavily Used In Japan, You Don't Know.
Junichi Ishida
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
Sylvain Zimmer
 
5 things MySql
5 things MySql
sarahnovotny
 
Jruby a Pi and a database
Jruby a Pi and a database
Philipp Fehre
 
A rough guide to JavaScript Performance
A rough guide to JavaScript Performance
allmarkedup
 
OSDC 2016 - DNS for Developers by Jan-Piet Mens
OSDC 2016 - DNS for Developers by Jan-Piet Mens
NETWAYS
 
Writing native bindings to node.js in C++
Writing native bindings to node.js in C++
nsm.nikhil
 
Use of 3D Immersive Technology for the Support of Gifted Learners
Use of 3D Immersive Technology for the Support of Gifted Learners
Giftedkids.ie
 
♥♥♥
♥♥♥
jennybabe
 

More Related Content

What's hot (19)

Applied Shell Scripting - stills to time-lapse
Applied Shell Scripting - stills to time-lapse
Shoaib Sufi
 
Алексей Петров "PHP at Scale: Knowing enough to be dangerous!"
Алексей Петров "PHP at Scale: Knowing enough to be dangerous!"
Fwdays
 
Александр Махомет "Beyond the code или как мониторить ваш PHP сайт"
Александр Махомет "Beyond the code или как мониторить ваш PHP сайт"
Fwdays
 
Server side scripting smack down - Node.js vs PHP
Server side scripting smack down - Node.js vs PHP
Marc Gear
 
Webdevcon Keynote hh-2012-09-18
Webdevcon Keynote hh-2012-09-18
Pierre Joye
 
Basic testing with selenium
Basic testing with selenium
Søren Lund
 
HTTP::Parser::XS - writing a fast & secure XS module
HTTP::Parser::XS - writing a fast & secure XS module
Kazuho Oku
 
Chenli linux-kerne-community
Chenli linux-kerne-community
力 陈
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
Tom Croucher
 
Searching CPAN Offline
Searching CPAN Offline
acme
 
Nginx وب سروری برای تمام فصول
Nginx وب سروری برای تمام فصول
efazati
 
A slightly advanced introduction to node.js
A slightly advanced introduction to node.js
Sudar Muthu
 
Web Development with Python and Django
Web Development with Python and Django
Michael Pirnat
 
Great Tools Heavily Used In Japan, You Don't Know.
Great Tools Heavily Used In Japan, You Don't Know.
Junichi Ishida
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
Sylvain Zimmer
 
5 things MySql
5 things MySql
sarahnovotny
 
Jruby a Pi and a database
Jruby a Pi and a database
Philipp Fehre
 
A rough guide to JavaScript Performance
A rough guide to JavaScript Performance
allmarkedup
 
OSDC 2016 - DNS for Developers by Jan-Piet Mens
OSDC 2016 - DNS for Developers by Jan-Piet Mens
NETWAYS
 
Applied Shell Scripting - stills to time-lapse
Applied Shell Scripting - stills to time-lapse
Shoaib Sufi
 
Алексей Петров "PHP at Scale: Knowing enough to be dangerous!"
Алексей Петров "PHP at Scale: Knowing enough to be dangerous!"
Fwdays
 
Александр Махомет "Beyond the code или как мониторить ваш PHP сайт"
Александр Махомет "Beyond the code или как мониторить ваш PHP сайт"
Fwdays
 
Server side scripting smack down - Node.js vs PHP
Server side scripting smack down - Node.js vs PHP
Marc Gear
 
Webdevcon Keynote hh-2012-09-18
Webdevcon Keynote hh-2012-09-18
Pierre Joye
 
Basic testing with selenium
Basic testing with selenium
Søren Lund
 
HTTP::Parser::XS - writing a fast & secure XS module
HTTP::Parser::XS - writing a fast & secure XS module
Kazuho Oku
 
Chenli linux-kerne-community
Chenli linux-kerne-community
力 陈
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
Tom Croucher
 
Searching CPAN Offline
Searching CPAN Offline
acme
 
Nginx وب سروری برای تمام فصول
Nginx وب سروری برای تمام فصول
efazati
 
A slightly advanced introduction to node.js
A slightly advanced introduction to node.js
Sudar Muthu
 
Web Development with Python and Django
Web Development with Python and Django
Michael Pirnat
 
Great Tools Heavily Used In Japan, You Don't Know.
Great Tools Heavily Used In Japan, You Don't Know.
Junichi Ishida
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
Sylvain Zimmer
 
Jruby a Pi and a database
Jruby a Pi and a database
Philipp Fehre
 
A rough guide to JavaScript Performance
A rough guide to JavaScript Performance
allmarkedup
 
OSDC 2016 - DNS for Developers by Jan-Piet Mens
OSDC 2016 - DNS for Developers by Jan-Piet Mens
NETWAYS
 

Viewers also liked (20)

Writing native bindings to node.js in C++
Writing native bindings to node.js in C++
nsm.nikhil
 
Use of 3D Immersive Technology for the Support of Gifted Learners
Use of 3D Immersive Technology for the Support of Gifted Learners
Giftedkids.ie
 
♥♥♥
♥♥♥
jennybabe
 
Vision2music
Vision2music
DirkVerhoeven
 
优丽奇中国 公司手册
优丽奇中国 公司手册
Goodrich Global
 
Goodrich Global Corporate Brochure
Goodrich Global Corporate Brochure
Goodrich Global
 
Digiaika - Mikä Muuttuu Markkinoinnissa
Digiaika - Mikä Muuttuu Markkinoinnissa
Darwin Oy
 
Accelerating innovation and diffusion of renewable energy technologies: techn...
Accelerating innovation and diffusion of renewable energy technologies: techn...
CambridgeIP Ltd
 
Voita peli verkossa: Parempia asiakkaita verkkopalvelun optimoinnilla
Voita peli verkossa: Parempia asiakkaita verkkopalvelun optimoinnilla
Darwin Oy
 
CambridgeIP Webinar: Developing a fact Based IP Strategy
CambridgeIP Webinar: Developing a fact Based IP Strategy
CambridgeIP Ltd
 
CambridgeIP: Case Studies Of Recent Client Engagements
CambridgeIP: Case Studies Of Recent Client Engagements
CambridgeIP Ltd
 
Goodrich Global 产品常见问题 (地板)
Goodrich Global 产品常见问题 (地板)
Goodrich Global
 
Blowin In The Wind
Blowin In The Wind
goznevi
 
Renewable energy and water treatment: emerging opportunities
Renewable energy and water treatment: emerging opportunities
CambridgeIP Ltd
 
Smart camera monitoring system
Smart camera monitoring system
Arvind Krishnaa
 
E11 Physics Evaluation Sheet
E11 Physics Evaluation Sheet
guest411ccf79
 
The HFA pMDI Patent Landscape: Minefield or Goldmine
The HFA pMDI Patent Landscape: Minefield or Goldmine
CambridgeIP Ltd
 
Burns supper (3ºb 2012 13)
Burns supper (3ºb 2012 13)
Rocio Torres
 
TodiCastle: villa rentals & historic hotel in Umbria
TodiCastle: villa rentals & historic hotel in Umbria
Mario Santoro
 
Ota sosiaalinen media tehokäyttöön
Ota sosiaalinen media tehokäyttöön
Darwin Oy
 
Writing native bindings to node.js in C++
Writing native bindings to node.js in C++
nsm.nikhil
 
Use of 3D Immersive Technology for the Support of Gifted Learners
Use of 3D Immersive Technology for the Support of Gifted Learners
Giftedkids.ie
 
优丽奇中国 公司手册
优丽奇中国 公司手册
Goodrich Global
 
Goodrich Global Corporate Brochure
Goodrich Global Corporate Brochure
Goodrich Global
 
Digiaika - Mikä Muuttuu Markkinoinnissa
Digiaika - Mikä Muuttuu Markkinoinnissa
Darwin Oy
 
Accelerating innovation and diffusion of renewable energy technologies: techn...
Accelerating innovation and diffusion of renewable energy technologies: techn...
CambridgeIP Ltd
 
Voita peli verkossa: Parempia asiakkaita verkkopalvelun optimoinnilla
Voita peli verkossa: Parempia asiakkaita verkkopalvelun optimoinnilla
Darwin Oy
 
CambridgeIP Webinar: Developing a fact Based IP Strategy
CambridgeIP Webinar: Developing a fact Based IP Strategy
CambridgeIP Ltd
 
CambridgeIP: Case Studies Of Recent Client Engagements
CambridgeIP: Case Studies Of Recent Client Engagements
CambridgeIP Ltd
 
Goodrich Global 产品常见问题 (地板)
Goodrich Global 产品常见问题 (地板)
Goodrich Global
 
Blowin In The Wind
Blowin In The Wind
goznevi
 
Renewable energy and water treatment: emerging opportunities
Renewable energy and water treatment: emerging opportunities
CambridgeIP Ltd
 
Smart camera monitoring system
Smart camera monitoring system
Arvind Krishnaa
 
E11 Physics Evaluation Sheet
E11 Physics Evaluation Sheet
guest411ccf79
 
The HFA pMDI Patent Landscape: Minefield or Goldmine
The HFA pMDI Patent Landscape: Minefield or Goldmine
CambridgeIP Ltd
 
Burns supper (3ºb 2012 13)
Burns supper (3ºb 2012 13)
Rocio Torres
 
TodiCastle: villa rentals & historic hotel in Umbria
TodiCastle: villa rentals & historic hotel in Umbria
Mario Santoro
 
Ota sosiaalinen media tehokäyttöön
Ota sosiaalinen media tehokäyttöön
Darwin Oy
 
Ad

Similar to Functional Programming with Streams in node.js (20)

Streams
Streams
Marielle Lange
 
Streams
Streams
Eugene Lazutkin
 
File System.pptx
File System.pptx
Bareen Shaikh
 
Node js lecture
Node js lecture
Darryl Sherman
 
Node.js streams talk
Node.js streams talk
zladuric
 
FunctionalJS - May 2014 - Streams
FunctionalJS - May 2014 - Streams
darach
 
module node jsbhgnbgtyuikmnbvcfyum2.pptx
module node jsbhgnbgtyuikmnbvcfyum2.pptx
hemalathas752360
 
Flow Base Programming with Node-RED and Functional Reactive Programming with ...
Flow Base Programming with Node-RED and Functional Reactive Programming with ...
Sven Beauprez
 
Learning a node stream
Learning a node stream
kumatch kumatch
 
Streams in node js
Streams in node js
Kushal Likhi
 
Introduction to Node.js
Introduction to Node.js
Richard Lee
 
Web streams
Web streams
Vladimir Shevchuk
 
Node js beginner
Node js beginner
Sureshreddy Nalimela
 
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Tom Croucher
 
Presentation.pptx
Presentation.pptx
Naor Tedgi
 
Nodejs a-practical-introduction-oredev
Nodejs a-practical-introduction-oredev
Felix Geisendörfer
 
Streams in Node .pdf
Streams in Node .pdf
SudhanshiBakre1
 
Node.js Stream API
Node.js Stream API
The Software House
 
Node.js File system & Streams
Node.js File system & Streams
Eyal Vardi
 
About Node.js
About Node.js
Artemisa Yescas Engler
 
Node.js streams talk
Node.js streams talk
zladuric
 
FunctionalJS - May 2014 - Streams
FunctionalJS - May 2014 - Streams
darach
 
module node jsbhgnbgtyuikmnbvcfyum2.pptx
module node jsbhgnbgtyuikmnbvcfyum2.pptx
hemalathas752360
 
Flow Base Programming with Node-RED and Functional Reactive Programming with ...
Flow Base Programming with Node-RED and Functional Reactive Programming with ...
Sven Beauprez
 
Streams in node js
Streams in node js
Kushal Likhi
 
Introduction to Node.js
Introduction to Node.js
Richard Lee
 
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Tom Croucher
 
Presentation.pptx
Presentation.pptx
Naor Tedgi
 
Nodejs a-practical-introduction-oredev
Nodejs a-practical-introduction-oredev
Felix Geisendörfer
 
Node.js File system & Streams
Node.js File system & Streams
Eyal Vardi
 
Ad

Recently uploaded (20)

Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
 
Hyderabad MuleSoft In-Person Meetup (June 21, 2025) Slides
Hyderabad MuleSoft In-Person Meetup (June 21, 2025) Slides
Ravi Tamada
 
You are not excused! How to avoid security blind spots on the way to production
You are not excused! How to avoid security blind spots on the way to production
Michele Leroux Bustamante
 
Security Tips for Enterprise Azure Solutions
Security Tips for Enterprise Azure Solutions
Michele Leroux Bustamante
 
The Future of Product Management in AI ERA.pdf
The Future of Product Management in AI ERA.pdf
Alyona Owens
 
From Manual to Auto Searching- FME in the Driver's Seat
From Manual to Auto Searching- FME in the Driver's Seat
Safe Software
 
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
 
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
Priyanka Aash
 
Curietech AI in action - Accelerate MuleSoft development
Curietech AI in action - Accelerate MuleSoft development
shyamraj55
 
Salesforce Summer '25 Release Frenchgathering.pptx.pdf
Salesforce Summer '25 Release Frenchgathering.pptx.pdf
yosra Saidani
 
“MPU+: A Transformative Solution for Next-Gen AI at the Edge,” a Presentation...
“MPU+: A Transformative Solution for Next-Gen AI at the Edge,” a Presentation...
Edge AI and Vision Alliance
 
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
caoyixuan2019
 
Agentic AI for Developers and Data Scientists Build an AI Agent in 10 Lines o...
Agentic AI for Developers and Data Scientists Build an AI Agent in 10 Lines o...
All Things Open
 
"Scaling in space and time with Temporal", Andriy Lupa.pdf
"Scaling in space and time with Temporal", Andriy Lupa.pdf
Fwdays
 
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Safe Software
 
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
digitaljignect
 
Quantum AI Discoveries: Fractal Patterns Consciousness and Cyclical Universes
Quantum AI Discoveries: Fractal Patterns Consciousness and Cyclical Universes
Saikat Basu
 
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
Safe Software
 
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
Fwdays
 
Quantum AI: Where Impossible Becomes Probable
Quantum AI: Where Impossible Becomes Probable
Saikat Basu
 
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
 
Hyderabad MuleSoft In-Person Meetup (June 21, 2025) Slides
Hyderabad MuleSoft In-Person Meetup (June 21, 2025) Slides
Ravi Tamada
 
You are not excused! How to avoid security blind spots on the way to production
You are not excused! How to avoid security blind spots on the way to production
Michele Leroux Bustamante
 
Security Tips for Enterprise Azure Solutions
Security Tips for Enterprise Azure Solutions
Michele Leroux Bustamante
 
The Future of Product Management in AI ERA.pdf
The Future of Product Management in AI ERA.pdf
Alyona Owens
 
From Manual to Auto Searching- FME in the Driver's Seat
From Manual to Auto Searching- FME in the Driver's Seat
Safe Software
 
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
 
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
Priyanka Aash
 
Curietech AI in action - Accelerate MuleSoft development
Curietech AI in action - Accelerate MuleSoft development
shyamraj55
 
Salesforce Summer '25 Release Frenchgathering.pptx.pdf
Salesforce Summer '25 Release Frenchgathering.pptx.pdf
yosra Saidani
 
“MPU+: A Transformative Solution for Next-Gen AI at the Edge,” a Presentation...
“MPU+: A Transformative Solution for Next-Gen AI at the Edge,” a Presentation...
Edge AI and Vision Alliance
 
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
caoyixuan2019
 
Agentic AI for Developers and Data Scientists Build an AI Agent in 10 Lines o...
Agentic AI for Developers and Data Scientists Build an AI Agent in 10 Lines o...
All Things Open
 
"Scaling in space and time with Temporal", Andriy Lupa.pdf
"Scaling in space and time with Temporal", Andriy Lupa.pdf
Fwdays
 
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Safe Software
 
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
digitaljignect
 
Quantum AI Discoveries: Fractal Patterns Consciousness and Cyclical Universes
Quantum AI Discoveries: Fractal Patterns Consciousness and Cyclical Universes
Saikat Basu
 
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
Safe Software
 
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
Fwdays
 
Quantum AI: Where Impossible Becomes Probable
Quantum AI: Where Impossible Becomes Probable
Saikat Basu
 

Functional Programming with Streams in node.js

  • 1. Functional Programming with Streams in node.js
  • 2. Who Am I?   HP webOS Framework Team   @CrabDude (Twitter, Github, etc…)   Organizer BayNode & Dallas.node   Author:   trycatch – async try/catch in node.js   stepup – step node.js control flow + trycatch   tiki – in-browser package manager
  • 3. The Problem VFS VFS VFS hermesClient hermesClient hermesClient hermesBuild 3rd Party API hermes hermesDeploy Ares (browser)
  • 4. The Goal   Don’t buffer any data   Don’t write to disk   Minimize latency   Allow reuse of functionality (e.g., readdir)   Functionality may be local or remote (e.g., getFolder)
  • 5. Why node.js? “Node's goal is to provide an easy way to build scalable network programs.” – nodejs.org
  • 6. node.js === JavaScript JavaScript is the future of the web, and code reuse is its unfair advantage.
  • 8. Basics   EventEmitter   Stream   Traditional streams (aka non-node.js lazy streams)
  • 11. Streams “Streams are to time as arrays are to space.” – Jed Schmidt @ JSConf.eu/2010
  • 12. Readable Streams   Events: data, end, error, close   Methods: pause, resume, end, destroy
  • 13. Writable Streams   Events: drain, error, close, pause, resume   Methods: write, end, destroy
  • 14. readable.pipe(writeable) Readable streams can be piped to writable streams (and vice-versa!)   on(‘data’) => write()   on(‘end’) => end()   on(‘drain’) => resume()   on(‘close’) => destroy()   on(‘error’) => on(‘error’)   on(‘pause’) => pause()
  • 16. Lazy Streams Streams in functional programming…
  • 17. Lazy Streams: Higher Order Functions A higher-order function is a function that does at least one of the following:   Take one or more functions as input   output a function
  • 20. Hermes: Revisited FTP Dropbox Box.net Filesystem hermesFtp hermesDropbox hermesBox hermesFilesystem hermesBuild 3rd Party API hermesDeploy Ares (browser)
  • 21. Still Got All That?
  • 22. We need… We need to recursively zip a directory…   Let’s start with abstract implementation non-specific functional versions of…   Filter paths from readdir   Filter directories from paths   Filter files from paths   Recursively return all file paths in a directory
  • 24. We need…   More “abstract implementation non-specific functional versions” (errr…) of…   ls (aka readdir)   getFile (aka readFile)   getFolder (stream of gets)
  • 26. We need…   Yet More AINSFV of...   Zip a stream of files   Allow node.js streams to be piped to lazy streams   pipe(): lazy stream => node.js stream   pump(): node.js stream => lazy stream
  • 29. Now we can… Zip a folder up using an arbitrary…   getFile / readFile   ls / readdir … And stream the response! Yeeeeehaw!
  • 32. Now What? Since all the implementation specific functions are passed in, we can…
  • 33. Port this to the browser!
  • 34. Say whaaat? node-browserify by @Substack does a lot of this already   Stream, Buffer, Path, etc… We need…   Complete buffer implementation (browser has TypedArrays & ArrayBuffer)   fs (browser has FileSystem API)   zlib…   inflate(), deflate(), gzip(), gunzip(), unzip()
  • 35. Stay Tuned. I’m currently porting the fs and buffer modules to the browser, and eventually zlib. http://github.com/crabdude/fs-browserify http://github.com/crabdude/buffer-browserify http://github.com/crabdude/zlib-browserify
  • 36. We’re HIRING! webOS Framework Team If you think this is pretty cool… Or you think I’M pretty cool… Stop by the Enyo booth or talk to me afterwards.
  • 37. Thank you. [email protected] https://github.com/Gozala/streamer/blob/master/readme.js http://en.wikipedia.org/wiki/Higher-order_function https://github.com/mikeal/request http://felixge.s3.amazonaws.com/11/nodejs-streams.pdf https://github.com/substack/node-browserify Functional Programming in JavaScript: http://igstan.ro/posts/2011-05-02-understanding-monads-with- javascript.html http://ndc2011.macsimum.no/mp4/Day2%20Thursday/ Track6%201500-1600.mp4 http://drboolean.tumblr.com/ http://osteele.com/sources/javascript/functional/