Defensive programming in Javascript and Node.js

Defensive Programming
in Javascript & node.js
Wednesday, May 29, 13

INTRODUCTION
Ruben Tan Long Zheng 陈龙正
VP of Engineering, OnApp CDN KL
Lead Engineer, 40 Square Sdn Bhd
Javascript > 5 years
@roguejs
Organizer of Nodehack KL

OVERVIEW
Dependency Awareness
Javascript Mastery
Methodology Improvements

SEASON 1
DEPENDENCY
AWARENESS

Internal dependencies
Libraries
require(), include()
External dependencies
Services, ﬁles, databases, etc
socket.connect(), db.open()
DEPENDENCY TYPES

NEVER ASSUME!
Never assume a dependency is reliable!
var db = require(‘database’);
db.open();
db.write(‘foo bar’, function (err, data) {
// ... do something ...
});

NEVER ASSUME!
db.open();
db.write(‘foo bar’, function (err, data) {
// ... do something ...
});
What if this failed?
will write() throw an error? will open() throw
an exception?

NEVER ASSUME!
db.open(function (err) {
db.write(‘mr-big’, bigData, function (err, data) {
// ... unrelated logic
db.close();
});
db.read(‘foo2’, function (err, data) {
// ... some work done
});
});
Accidents happen...

NEVER ASSUME!
db.open(function (err) {
db.write(‘mr-big’, bigData, function (err, data) {
// ... unrelated logic
db.close();
});
db.read(‘foo2’, function (err, data) {
// ... some work done
});
});
close() might affect read()

A MORE COMPLEX EXAMPLE...

VIDEO STREAMING SERVICE
Video
Streamer
Origin
Stats
Logger
VOD
Client
User
Accounting
UploadLive
Client
User
Stream
Stream
LogReport
Render
Render

VIDEO STREAMING SERVICE
Video
Streamer
Origin
Stats
Logger
VOD
Client
User
Accounting
UploadLive
Client
User
Stream
Stream
LogReport
Render
Render
1
2
3
4
5
6
7
8
9
10
11

DEPENDENCY AWARENESS
What can fail, WILL FAIL!
Never assume a dependency is reliable!
Contingency plans - failover, redundancy, fail-fast, etc
Pro-active monitoring
Load test, stress test, chaos monkey, etc
Remember, what can fail, WILL FAIL!

SEASON 2
JAVASCRIPT MASTERY

JAVASCRIPT MASTERY
Code Execution Order
Sanitization & Validation
Scope
Control Flow

I KNOW CODE-FU!

EXECUTION ORDER
var mq = require(‘mq’);
mq.conn(...);
mq.on(‘ready’, function () {
mq.send(‘batman’);
mq.on(‘message’, function (msg) {
console.log(msg);
mq.close();
});
});
mq is never closed!
send() executes before on()

DOIN’ IT RIGHT!
var mq = require(‘mq’);
mq.conn(...);
mq.on(‘ready’, function () {
mq.on(‘message’, function (msg) {
console.log(msg);
mq.close();
});
mq.send(‘batman’);
});
Swap places

SANITIZATION & VALIDATION
function foodForKittens(num) {
return num * 10;
}
foodForKittens();
num is not validated, is undefined
this will fail!

TOO SIMPLE?

SANITIZATION & VALIDATION
var conn = db.open(...);
function writeToDb(conn, cb) {
conn.write(bigData, function (err, res) {
if (err) {
cb(err);
return;
}
cb(null, res);
});
});
writeToDb(conn, ghostCallback);

if (err) {
cb(err);
return;
}
cb(null, res);
});
});
what if open() returned undefined?
this will throw an exception!

if (err) {
cb(err);
return;
}
cb(null, res);
});
});
What if ghostCallback is undefined?
These will fail too!

DOIN’ IT RIGHT!
if (typeof conn !== ‘object’) {
// ... handle error ...
}
if (typeof cb !== ‘function’) {
// ... handle error ...
}
if (err) {
cb(err);
return;
}
cb(null, res);
});
});
Validate your input,
especially when they
involve functions or
methods that you need to
invoke in your code.
These are not the time to
fail-fast!

DON’T GO OVERBOARD...
Validate only necessary parameters
Method invocations (anObject.method())
Function invocations (aFunction())
Have a proper error/exception handling policy
Validate for correctness, not existence
Correctness: typeof a === ‘object’
Existence: a !== undefined

SCOPE AWARENESS
Plagues most callback-based code
Bad practice leads to costly debugging waste
New JS programmers not aware of scoping
JS scoping is a simple but weird thing (to non-JS
programmers)

SCOPE!!!
var a = ‘outside’;
if (true) {
var a = ‘inside’;
console.log(a);
}
console.log(a);
What is the output?
> node test.js
inside
inside

SCOPE!!!
Non-JS programmers:
a inside the if block is “inside”
a outside the if block is “outside”
JS programmers:
they are both “inside”
JS scope by function

SCOPE CHAINS!!!
var avar = 1;
(function outer1() {
var avar = 2;
(function inner1() {
var avar = 3;
console.log(avar); // outputs 3
})();
})();
})();
(function outer2() {
})();
})();
inner1()
local - found!
inner2()
local - nope
outer1() - found!
inner3()
local - nope
outer2() - nope
global - found!

HOISTING VARIABLES
function () {
for (var i = 0; i < 10; i++) {
for (var j = 0; j < 10; j++) {
// ... do something
}
}
}
function () {
var i, j; // now the scope is clear for i & j
for (i = 0; i < 10; i++) {
for (j = 0; j < 10; j++) {
// ... do something
}
}
}
Below is far clearer what individual variable scopes are:

CONTROL FLOW
Node.js’ async nature makes it unintuitive to predict
control ﬂow
I <3 async (github.com/caolan/async)
Control ﬂow is ugly. Welcome to Javascript.
Async will save your life. Use it.

CONTROL FLOW
var fs;
fs = require(‘fs’);
fs.readFile(‘./myfile.txt’, function (err, data) {
if (err) {
console.log(err);
return;
}
fs.writeFile(‘./myfile2.txt’, data, function (err) {
if (err) {
console.log(err);
return;
}
// ... do stuff ...
});
})

CONTROL FLOW
Callback hell!
Step 1
Step 2
Step 3
Step 4
Step 5

mod.step1(function () {
// ... too many levels ...
});
});
}
});
});

CONTROL FLOW
var async, fs;
async = require(‘async’);
fs = require(‘fs’);
async.waterfall([
function step1(callback) {
fs.readFile(‘./myfile.txt’, callback);
},
function step2(data, callback) {
fs.writeFile(‘./myfile2.txt’, data, callback);
}
], function (err) {
// ... execute something in the end ...
});

SEASON 3
METHODOLOGY
IMPROVEMENTS

GOLDEN RULES
Golden Rules of Defensive Programming
Proper error handling policy
Intelligent logging
Design for failure

ERROR HANDLING
Never, ever HIDE errors
> node app.js 2>&1 /dev/null
ob.callback(function (err, data) {
if (err) {}
console.log(data);
});
socket.on(‘error’, function () {});

ERROR HANDLING
I WILL FIND YOU
AND I WILL CRASH YOU

ERROR HANDLING
Standardize error handling in the app
Log to error DB
Output to error ﬁle
Output error to a stream
Use a logging library
Ask a leprechaun to manage it
etc

LOGGING
How do you feel if your “log” looks like this?
> tail -f error.log
[12:01:55] ERROR - General error detected

LOGGING
Logs are the ﬁrst place you go to ﬁnd out what
happened
Standardize a log location for each app
Make logs easy to access for developers

DESIGN FOR FAILURE
Common steps to designing software:
1 - what should it do?
2 - how do I do it?
3 - how do I deploy?
4 - done

DESIGN FOR FAILURE
Proper steps in defensive programming:
1 - what should it do?
2 - how many ways can it fail?
3 - how do I know when it fails?
4 - how do I prevent it from failing?
5 - write code accordingly

DESIGN FOR FAILURE
Nothing is reliable
TCP can fail
Network can go down
Servers can run out of memory
Cows might ﬂy through the sky crashing into your
datacenter and ﬂooding the server rooms with milk
and destroying everything

DESIGN FOR FAILURE
Designing for failure mindset & methodologies:
Identify SPOF (single point of failures)
Redundancy, failover, monitoring
Fail-fast, start-fast
Persist important data
Reliability & Consistency > Speed
Code is liability

~ The End ~

Defensive programming in Javascript and Node.js

More Related Content

What's hot (11)

Viewers also liked (16)

Similar to Defensive programming in Javascript and Node.js (20)

More from Ruben Tan (11)

Recently uploaded (20)

Defensive programming in Javascript and Node.js