defensive programming in javascript and node.js

Defensive Programming

in Javascript & node.js

Wednesday, May 29, 13

INTRODUCTION

Ruben Tan Long Zheng 陈龙正

VP of Engineering, OnApp CDN KL

Lead Engineer, 40 Square Sdn Bhd

Javascript > 5 years

@roguejs

Organizer of Nodehack KL

OVERVIEW

Dependency Awareness

Javascript Mastery

Methodology Improvements

SEASON 1

DEPENDENCY AWARENESS

Internal dependencies

Libraries

require(), include()

External dependencies

Services, files, databases, etc

socket.connect(), db.open()

DEPENDENCY TYPES

NEVER ASSUME!

Never assume a dependency is reliable!

var db = require(‘database’);

db.open();

db.write(‘foo bar’, function (err, data) { // ... do something ... });

NEVER ASSUME!

db.open();

db.write(‘foo bar’, function (err, data) { // ... do something ... });

What if this failed?will write() throw an error? will open() throwan exception?

NEVER ASSUME!

db.open(function (err) { db.write(‘mr-big’, bigData, function (err, data) { // ... unrelated logic db.close(); });

db.read(‘foo2’, function (err, data) { // ... some work done });});

Accidents happen...

NEVER ASSUME!

db.open(function (err) { db.write(‘mr-big’, bigData, function (err, data) { // ... unrelated logic db.close(); });

db.read(‘foo2’, function (err, data) { // ... some work done });});

close() might affect read()

A MORE COMPLEX EXAMPLE...

VIDEO STREAMING SERVICE

VideoStreamer

Origin

StatsLogger

VODClient

Accounting

UploadLiveClient

UserStream

Stream

LogReport

Render

VIDEO STREAMING SERVICE

VideoStreamer

Origin

StatsLogger

VODClient

Accounting

UploadLiveClient

UserStream

Stream

LogReport

Render

Render1

DEPENDENCY AWARENESS

What can fail, WILL FAIL!

Never assume a dependency is reliable!

Contingency plans - failover, redundancy, fail-fast, etc

Pro-active monitoring

Load test, stress test, chaos monkey, etc

Remember, what can fail, WILL FAIL!

SEASON 2

JAVASCRIPT MASTERY

Code Execution Order

Sanitization & Validation

Control Flow

I KNOW CODE-FU!

EXECUTION ORDER

var mq = require(‘mq’);

mq.conn(...);

mq.on(‘ready’, function () { mq.send(‘batman’);

mq.on(‘message’, function (msg) { console.log(msg); mq.close(); });});

mq is never closed!

send() executes before on()

DOIN’ IT RIGHT!

var mq = require(‘mq’);

mq.conn(...);

mq.on(‘ready’, function () { mq.on(‘message’, function (msg) { console.log(msg); mq.close(); });

mq.send(‘batman’);});

Swap places

SANITIZATION & VALIDATION

function foodForKittens(num) { return num * 10;}

foodForKittens();

num is not validated, is undefined

this will fail!

TOO SIMPLE?

SANITIZATION & VALIDATION

var db = require(‘database’);var conn = db.open(...);

function writeToDb(conn, cb) { conn.write(bigData, function (err, res) { if (err) { cb(err); return; }

cb(null, res); });});

writeToDb(conn, ghostCallback);

what if open() returned undefined?

this will throw an exception!

What if ghostCallback is undefined?

These will fail too!

DOIN’ IT RIGHT!var db = require(‘database’);var conn = db.open(...);

function writeToDb(conn, cb) { if (typeof conn !== ‘object’) { // ... handle error ... }

if (typeof cb !== ‘function’) { // ... handle error ... }

conn.write(bigData, function (err, res) { if (err) { cb(err); return; }

Validate your input,especially when theyinvolve functions ormethods that you need toinvoke in your code.

These are not the time tofail-fast!

DON’T GO OVERBOARD...

Validate only necessary parameters

Method invocations (anObject.method())

Function invocations (aFunction())

Have a proper error/exception handling policy

Validate for correctness, not existence

Correctness: typeof a === ‘object’

Existence: a !== undefined

SCOPE AWARENESS

Plagues most callback-based code

Bad practice leads to costly debugging waste

New JS programmers not aware of scoping

JS scoping is a simple but weird thing (to non-JS programmers)

SCOPE!!!

var a = ‘outside’;

if (true) { var a = ‘inside’; console.log(a);}

console.log(a);

What is the output?

> node test.jsinsideinside

SCOPE!!!

Non-JS programmers:

a inside the if block is “inside”

a outside the if block is “outside”

JS programmers:

they are both “inside”

JS scope by function

SCOPE CHAINS!!!var avar = 1;

(function outer1() { var avar = 2; (function inner1() { var avar = 3; console.log(avar); // outputs 3 })();

(function inner2() { console.log(avar); // outputs 2 })();})();

(function outer2() { (function inner3() { console.log(avar); // outputs 1 })();})();

inner1()

local - found!

inner2()

local - nope

outer1() - found!

inner3()

local - nope

outer2() - nope

global - found!

HOISTING VARIABLESfunction () { for (var i = 0; i < 10; i++) { for (var j = 0; j < 10; j++) { // ... do something } }}

function () { var i, j; // now the scope is clear for i & j

for (i = 0; i < 10; i++) { for (j = 0; j < 10; j++) { // ... do something } }}

Below is far clearer what individual variable scopes are:

CONTROL FLOW

Node.js’ async nature makes it unintuitive to predict control flow

I <3 async (github.com/caolan/async)

Control flow is ugly. Welcome to Javascript.

Async will save your life. Use it.

CONTROL FLOWvar fs;

fs = require(‘fs’);

fs.readFile(‘./myfile.txt’, function (err, data) { if (err) { console.log(err); return; }

fs.writeFile(‘./myfile2.txt’, data, function (err) { if (err) { console.log(err); return; }

// ... do stuff ... });})

CONTROL FLOW

Callback hell!

Step 1

Step 2

Step 3

Step 4

Step 5

mod.step1(function () { mod.step2(function () { mod.step3(function () { mod.step4(function () { mod.step5(function () { // ... too many levels ... }); }); } });});

CONTROL FLOW

var async, fs;

async = require(‘async’);fs = require(‘fs’);

async.waterfall([ function step1(callback) { fs.readFile(‘./myfile.txt’, callback); },

function step2(data, callback) { fs.writeFile(‘./myfile2.txt’, data, callback); }], function (err) { // ... execute something in the end ...});

SEASON 3

METHODOLOGY IMPROVEMENTS

GOLDEN RULES

Golden Rules of Defensive Programming

Proper error handling policy

Intelligent logging

Design for failure

ERROR HANDLING

Never, ever HIDE errors

> node app.js 2>&1 /dev/null

ob.callback(function (err, data) { if (err) {} console.log(data);});

socket.on(‘error’, function () {});

ERROR HANDLING

I WILL FIND YOU

AND I WILL CRASH YOU

ERROR HANDLING

Standardize error handling in the app

Log to error DB

Output to error file

Output error to a stream

Use a logging library

Ask a leprechaun to manage it

LOGGING

How do you feel if your “log” looks like this?

> tail -f error.log[12:01:55] ERROR - General error detected[12:01:56] ERROR - General error detected[12:01:57] ERROR - General error detected[12:01:58] ERROR - General error detected[12:01:59] ERROR - General error detected[12:02:00] ERROR - General error detected[12:02:01] ERROR - General error detected

LOGGING

Logs are the first place you go to find out what happened

Standardize a log location for each app

Make logs easy to access for developers

DESIGN FOR FAILURE

Common steps to designing software:

1 - what should it do?

2 - how do I do it?

3 - how do I deploy?

4 - done

DESIGN FOR FAILURE

Proper steps in defensive programming:

1 - what should it do?

2 - how many ways can it fail?

3 - how do I know when it fails?

4 - how do I prevent it from failing?

5 - write code accordingly

DESIGN FOR FAILURE

Nothing is reliable

TCP can fail

Network can go down

Servers can run out of memory

Cows might fly through the sky crashing into your datacenter and flooding the server rooms with milk and destroying everything

DESIGN FOR FAILURE

Designing for failure mindset & methodologies:

Identify SPOF (single point of failures)

Redundancy, failover, monitoring

Fail-fast, start-fast

Persist important data

Reliability & Consistency > Speed

Code is liability

~ The End ~

defensive programming in javascript and node.js

Technology

full-stack javascript with node.js

node.js - javascript thread programming

malpraktek & defensive medicine

meteor.js only javascript, only javascript, only...

القيادة الأمنة defensive driving

[awsマイスターシリーズ] aws sdk for php / ruby /...

gvcesmediadrawer.gvces.com.br/columbia/original/session...busca...

vs code win -...

defensive driving

una elecciÓn confiable y de calidad para su negocio. ·...

programme / syllabus développeur fullstack - it-akademy,...

sinon.js best practices seamless degradation javascript ......

1v1 defensive

node.js - exploitez la puissance de javascript côté …...

· node.js, un javascript lato-server per ... node.js...

construção de sites 2 - christianosantos.com · conteúdo...

javascript everywhere - node.js | devon 2012

mutode: generic javascript and node.js mutation testing tool

defensive strategy - mooc.aptikom.or.id

4 4 defensive playbook