Why I don’t recommend the Step module [node.js, npm]

I prefer asyc to step. The async module has a cleaner interface, more options, and utilities. It's a little beefier in size, but those things are beside the point.

Here's why I don't really like step. Tell me what you'd expect from this function:

var Step = require('step');

Step(
	function first(){
		console.log('first');
		
		this();
	},
	
	function second(){
		var x;
		console.log('second');
		if(!x){
			// do something that should set x
		}
		return x;
	},
	
	function third(){
		console.log('third');
	}
)

Did you guess:

$ node step-example.js 
first
second

If you're not familiar with Step, you'll probably look at that first function and wonder what this(); actually does. Well, it is actually shorthand for next(); (I'm simplifying a little here).

Assuming you're at least somewhat familiar with asynchronous control flow, you could assume this(); calls the next function. But, what about the second function? It's returning a value. Does that return from second, or from Step, or from some other function? It returns from second(), but... it passes the value to third. Or, it should. Unless x is undefined, in which case it will be considered an unchained function. Now, I don't think your variables should regularly be 'undefined' at the point of return, but what if you use or someone on your team uses the popular convention of calling a callback as return callback(x);. If the Step module's convention of tweaking this semantics is ignored, another developer may look at it as a typo, "You can't call the context like that..." Right? Also, what if someone doesn't understand the return value can't be undefined and comments out the middle of that function? You may have cleanup logic that isn't getting checked in third().
We've all seen that happen before.

In the above example, if x was not undefined, it would be passed as a parameter to third().

It's this inconsistency which makes me feel like Step is an accident waiting to happen in a regular development team. The source code is pretty well written and concise. I think the author has done a great job, but the usage is unintuitive and I wouldn't recommend using the module.

On the other hand, async is beautifully intuitive.

Consider this example:

var async = require('async');

async.series([
	function first(done){
		console.log('first');
		done(null, 'first');
	},
	function second(done){
		var x;
		console.log('second');
		if(!x){
			// do something that should set x
		}
		done(null, x);
	},
	function third(done){
		console.log('third');
		
		done(null, 'third');
	}
], function(err, results){
	console.log(results);
})

async.series allows us to run a series of tasks (functions defined within the array), which pass a value to the callback (that last function you see).

If you forget to pass a value, the results will contain undefined at the index of the array. Your third function won't be skipped unless you forget to invoke the callback. To me, this is a pretty obvious error and one that is easy to spot and correct. Here's is the output of the above call:

first
second
first
third
[ 'first', undefined, 'third' ]

To be fair, the example of Step acts more like a waterfall function. Here's what that would look like in async.

async.waterfall([
	function first(done){
		console.log('first');
		done(null, 'first');
	},
	function second(val, done){
		var x;
		console.log('second has val: %j', val);
		if(!x){
			// do something that should set x
		}
		done(null, x);
	},
	function(val, done){
		console.log('third has val: %j', val);
		
		done(null, 'third');
	}
], function(err, val){
	console.log('callback has val: %j', val)
});

The above code is available on github: https://github.com/jimschubert/blogs

[node.js] Simple Logging in tweeter.js

tweeter.js is a simple OAuth implementation I started working on for a node.js application I had envisioned. The library works, as in it runs through the whole OAuth dance and allows the client to make API calls. There are some more in-depth things I never got around to (like caching the tokens and clearing sessions and whatnot) before I realized I don't like Twitter. Oh well.

I was just looking through my github repositories and I thought it would be cool to post about the simple logging mechanism I created for tweeter.js. Although not the coolest pattern I used in the project (which probably goes to mixins), I like how simple the solution is. Check out the code for other cool features..

Logging

The cool thing about this logging framework is that it allows you to write out logs to different streams. For instance, suppose you want to write all logging information other than errors to log.txt, but you want to keep errors in errors.txt.

The way I implemented this was to create an object of streams on the Log object. I used getters/setters to provide validation that an object being assigned to one of the properties is actually a stream. Although I didn't bother with it, this object could possibly be sealed or frozen.

// An object with safe getters/setters for each stream.  
// Streams are:  
// debug, info, error, warn
Log.prototype.streams = {
    get none() { return { write: function() { }}; },
    get debug() { return this._debug; },
    set debug(val) {
        requiresStream(val);
        this._debug = val;
    },
    get info() { return this._info; },
    set info(val) {
        requiresStream(val);
        this._info = val;
    },
    get warn() { return this._warn; },
    set warn(val) { 
        requiresStream(val);
        this._warn = val;
    },
    get error() { return this._error; },
    set error(val) {
        requiresStream(val);
        this._error = val;
    }
}

Because this is written for node.js, requiresStream() verifies the value is an instance of EventEmitter with a write function.

Defaults for these streams are set in the constructor with a default logLevel of -1 (disabled).

var Log = module.exports = exports = function Log(level) {
    // default error to stderr and the rest to stdout
    this.streams.debug = process.stdout;
    this.streams.info = process.stdout;
    this.streams.warn = process.stdout;
    this.streams.error = process.stderr;

    // default the logLevel to disabled
    this.logLevel = level || -1;
}

An interesting thing about this logger is how I make the calling convention flexible. I use .NET regularly and I would call a logging function in .NET with an enum value to specify the logLevel. I wanted a similar functionality in this logger, so I created a lookup object to map a key (i.e. DEBUG) to a logLevel (0) and a stream (debug).

// [DISABLED] else [DEBUG > INFO > WARN > ERROR]
Log.prototype.logLevels = {
    DISABLED:   { value: -1, stream: 'none' },
    DEBUG:      { value: 0, stream: 'debug' },
    INFO:       { value: 1, stream: 'info' },
    WARN:       { value: 2, stream: 'warn' },
    ERROR:      { value: 3, stream: 'error' }
}

I considered accessing these on the streams object directly, but did it this way for readability and flexibility. A major deciding factor in choosing this route was the ability to reorder logging levels in a single place in the code.

// ## log(level msg);
// logs the output with a nice colorful indicator of the stream.  
// `msg` may be a string or a formattable string and options.
Log.prototype.log = function(level, msg) {
    var original = level;
    try {
        if(typeof level === 'string'){
            level = this.logLevels[level.toUpperCase()];
        }
        // get logLevel value
        if( (this.logLevel > -1) && (level.value >= this.logLevel) ) {
            var stream = this.streams[level.stream];
            stream.write( util.format.call(this, '[\033[%sm%s\033[39m]\t', colors[level.value], level.stream) );
            stream.write( util.format.apply(this, slice.call(arguments,1) ) + '\n');
        }
    } catch (err) { 
        console.error("logging is improperly configured!\nIs %j a supported logLevel?\nError:\n%j\n",original, err);
    }
}

An example

I've uploaded an example to github.

In this example, I write out logs for each of the 4 supported levels. The default is to write these out to stdout. After writing to the defaults, I set

var Log = require('./log.js'),
    fs = require('fs');

var level = process.argv[2] || 1;
var logger = new Log(level),
    level = logger.logLevels;

logger.log(level.DEBUG, "This is a debug message");
logger.log(level.INFO, "This is an info message");
logger.log(level.WARN, "This is a warning message");
logger.log(level.ERROR, "This is an error message");

var debugStream = fs.createWriteStream('./debug.txt');
var infoStream = fs.createWriteStream('./info.txt');
var warnStream = fs.createWriteStream('./warn.txt');
var errorStream = fs.createWriteStream('./error.txt');

logger.streams.debug = debugStream;
logger.streams.info = infoStream;
logger.streams.warn = warnStream;
logger.streams.error = errorStream;

logger.log(level.DEBUG, "This is a debug message");
logger.log(level.INFO, "This is an info message");
logger.log(level.WARN, "This is a warning message");
logger.log(level.ERROR, "This is an error message");

debugStream.end();
infoStream.end();
warnStream.end();
errorStream.end();

This provides the following input when passing a log level in via the command line:

Makefile setup for minifying .js files in an expressjs application

There's not really any reason to uglify or minify your JavaScript files on a per-request basis or on a pre-cache basis. This should be done at deployment time only. This can be achieved by adding uglify-js to your package.json file.

// default package.js for an expressjs app using ejs and uglify-js
{
    "name": "application-name"
  , "version": "0.0.1"
  , "private": true
  , "dependencies": {
      "express": "2.4.6"
    , "ejs": ">=0.4.3"
    , "uglify-js": ">=1.1.0"
  }
}

Make sure the dependencies are installed with

npm install -d

Then, add the following make file and minify your JavaScripts as part of your make process!

JS = $(shell find public/javascripts/*.js)
MINIFY = $(JS:.js=.min.js)

all: clean $(MINIFY)

clean:
	rm -f ./public/javascripts/*.min.js

%.min.js: %.js
	node ./node_modules/uglify-js/bin/uglifyjs -o $@ $<

.PHONY: clean js minify

(expandable source is the same as the gist. It is doubled here to show in RSS feed readers.)