Tag Archives: nodejs

io.js 1.0.1 (unstable) is out. ES6!

A team of developers, including some core node.js developers, forked node.js a while ago into a project called io.js. They’ve just released version 1.0.1 (Unstable) and it brings ES6 natively to server-side JavaScript applications.

To test drive, you can download io.js. Be warned, it will completely replace your node and npm system links. Some people have complained about this or reported unexpected behavior after compiling io.js from source.

“Why io.js?” you ask? Joyent was a little stingy with node.js progress (it’s the basis for a huge money-maker for them). The io.js team’s goal is to make io.js more of a community-driven project. It seems to have worked, judging by the progress they’ve made.

Version 1.0.0 of io.js ships with V8 3.31.71.4, which includes ES6 features well beyond version 3.26.33 that will be shipped with joyent/[email protected]

No more –harmony flag

With io.js, you can now start using some ES6 features natively (e.g. no transpiling using Traceur or 6to5). Here’s the fibonacci example using an ES6 iterator taken from 6to5’s tour:

// fib.js
var fibonacci = { };
fibonacci[Symbol.iterator] = function*() {
  var pre = 0, cur = 1;
  for (;;) {
    var temp = pre;
    pre = cur;
    cur += temp;
    yield cur;
  }
}

for (var n of fibonacci) {
  // truncate the sequence at 1000
  if (n > 1000) {
    break;
  }
  process.stdout.write(n + '\n');
}

You’ll notice that I’ve had to change how an iterator is constructed and how the output is written. Run this with node fib.js (remember, io.js will overwrite the system node).

While this is excellent news, you’ll need to be aware of what ES6 features are currently supported. For example, io.js does not yet support classes. This won’t work in io.js:

class Animal{
  constructor(legs) {
    switch (legs) {
      case 2:
        this.locomotion = 'bipedal';
        break;
      case 4:
        this.locomotion = 'quadrupedal';
      default:
    }
  }
  move() {
    process.stdout.writeln("I'm a %j animal!", this.locomotion);
  }
  static getTypeName() {
    return "Animal";
  }
}

class Bird extends Animal {
  constructor(){
    super(2);
  }
}

class Lion extends Animal {
  constructor(){
    super(4);
  }
}

var bird = new Bird();
bird.move();

var lion = new Lion();
lion.move();

This will, however, work via transpilers:

These examples are on github.

Your First App: Node.js is complete!

I’m stoked to announce that I’ve finished writing my first self-published book, Your First App: node.js.

The book is a full-stack application development tutorial using what is commonly known as the ‘MEAN’ stack, but with a heavier focus on the backend technologies: node.js, express.js, and MongoDB. The book ends with a starter AngularJS project of an HTML5 instant messaging chat application.

While following along in the book, it may be useful to run test queries in a REST client. Postman is available for Google Chrome as a packaged application. Import `yfa-nodejs.postman_dump.json` (located in the root of the supplemental code’s repository) into Postman to evaluate the application’s API.

Check out the code on GitHub

Your first app: Node.js (update)

I am currently writing a book called Your first app: Node.js. I am self-publishing this book through leanpub.com. You can check it out here.

One cool thing about self-publishing on leanpub is that people can purchase the book early and receive all future updates as I write. Feedback from this process may also help me to revisit areas I’ve unintentionally left a little light or not explained very well.

The book is currently about 66% completed. I recently completed the chapters through the node.js API. To complement that API, I will be writing a chapter on creating an AngularJS front-end. Then I’ll explain how to deploy your application to Digital Ocean (although the steps will be similar for many other hosts). Finally, I will provide some suggestions for further learning.

Below is the book description as shown on leanpub.

ABOUT THE BOOK

This book is intended for professionals or students who have a comfortable working knowledge of JavaScript, HTML, CSS, and the stateless nature of HTTP. If you’re not comfortable with JavaScript, I highly recommend reading both JavaScript: The Definitive Guide and JavaScript Garden. Although I’ll quickly go over some necessary fundamentals for beginners, it will be difficult to become a ‘node.js developer’ without a solid foundation in JavaScript. HTML, CSS, and client-side JavaScript will be necessary for the presentation of your application, while understanding the stateless nature of HTTP will make some of the techniques discussed in this book easier to learn.

Why this book?

Many books on a given technology focus on really digging into a certain framework or a few simple concepts. Node.js is an interesting technology for many reasons, including:

  • it popularized server-side JavaScript
  • encourages modularization
  • has a fairly large community
  • provides amazing throughput for data

I have read many technology books and yet I’ve always felt like there was a gap between books covering a given technology and how you’d actually undertake creating a full application on your own. When I realized that node.js is attracting a lot of front-end developers and hobbyists to the community, I understood what that missing “gap” was: the application life cycle. For developers without a degree in Computer Science or Information Systems, it may be difficult to figure out how to actually start creating a full application. I hope to address this in the text by providing some simple guidelines that help to lay out a clear path between the starting point and the finished product. The intended audience for this book is anyone familiar with HTML, CSS and JavaScript. You don’t need to be a professional engineer or even have years of experience. In early chapters, I discuss the aspects of JavaScript that I feel are crucial to understanding how to write an application in node.js.  Where I feel newer developers may benefit from further reading, I suggest online or print resources that I’ve found helpful.

What makes this book different?

When working with Junior developers, I try to avoid the sink or swim mentality that many Seniors seem to have.  Instead, I like to explain why I’ve chosen one thing over another. In this book, I include an entire chapter to discuss development considerations which are usually omitted from mainstream texts.  For instance, when was the last time you read a book that suggested a build automation tool? I can’t remember any book that clearly explained why a sample application was written to use MongoDB instead of Redis or MySQL. These are things I attempt to cover in this book before diving in to the meat of the text you’d find in similar books. I even have a short prototyping chapter for readers who are unfamiliar with the technologies used in the book to try some hands-on code before applying this knowledge to the sample application.

These are all things people with professional full-stack (e.g. client-server-database) experience do on a regular basis. With such a fresh, young community being attracted to node.js application development (many without server-side or database experience) I think this book’s early focus on process and maintainability are important aspects for any beginner to pick up.

Technologies

Writing an application requires a developer to pick up skills in multiple technologies for frameworks. I plan to at least cover the basics of all the tools necessary to create a full node.js application. Some of the technologies covered in this book are:

  • express.js (node.js)
  • grunt.js (node.js)
  • mocha (node.js)
  • Twitter’s Bootstrap (CSS)
  • angular.js (client JavaScript)
  • MongoDB (database)
  • git (source control)

The intention is not to fully teach a developer everything about every one of these technologies, but I will go over the basics for everything that is not considered node.js and go over the node.js technologies in greater detail.

Planned Chapters

Future chapters will focus on:

  • ✓ git workflow
  • ✓ project structure and setup
  • ✓ setting up a database
  • ✓ modeling a database “schema”
  • ✓ creating a minimal API
  • creating a front-end application
  • asynchronous communication between client and server
  • ✓ authentication strategies
  • … more

As this book is written, I welcome your feedback and comments at [email protected]

Status

This book is currently about 66% complete.

The final product is estimated to be between 200 and 250 pages in length (not including appendices).

Sample Code

Sample code for this book is located on github at jimschubert/yfa-nodejs-code.

What is node.js?

I’ve recently decided to write a book about node.js. I don’t feel like a lot of books out there (on any technology) cover real-world scenarios. Most of them are introductions to languages or deep-dives into a single aspect of a language or technology. There are very few books that cover real-world application development scenarios to include things like process, build, sample application, and deployment. A good example of the type of book I’d like to write is Agile Web Development with Rails.

Below is an excerpt from one of the first chapters of the book. This is *very* rough draft, but I thought I would share because it may be useful for anyone beginning in node.js development.


What is node.js?

Node.js is a server-side JavaScript environment which provides evented asynchronous I/O and a minimalistic set of core features. In the beginning, node.js combined v8 with libev. Unfortunately, libev is an evented I/O library for Unix-based systems only. Eventually, libev was replaced with a higher-level abstraction library called libuv which allows node.js to run on Unix-based systems and Windows. In 2011, Microsoft began contributing to node.js to bring it to the Windows platform. More recently, libuv removed libev as a dependency.

The key concept for node.js is evented I/O (mostly network-based) operations. This is handled via an event loop. Although understanding the implementation details of event loops can be pretty complicated, the idea is fairly simple. Let’s analogize.

Try to think about an application as a fast food restaurant. A cashier is patiently waiting at the register. A customer walks up to the register and places an order. The cashier hands the order off to the kitchen and maybe fulfills the customer’s drink order before returning to the register. Another customer arrives and places an order, initiating the same scenario as before. A third customer is walking to the register, but just before the customer reaches the register a cook dings a bell and the cashier steps away to grab the first order and hand it to the customer. The cashier then begins taking the third customer’s order. Midway through the order, the second customer’s order is ready. Because the cashier is not rude (maybe this is in Canada?), the third order must be completed before the second customer’s order can be handed back to the customer. But, the cashier can easily hand over the second customer’s order before filling the third customer’s drink order. Eventually, the third customer’s order is fulfilled and there are no more customers. The cashier waits patiently.

It may sound strange, but the above scenario is a simplification of an event loop. We see things like this every day. New node.js developers seem to have a hard time either understanding the event loop or understanding where it comes from. Take the following example, for instance:

function main(){
  console.log("Hello, world!");
}

main();

There’s a little more going on in this example than the standard obligatory printing of “Hello, world!” I’ve wrapped the output line in a function called main and subsequently called that function. If you run this file, you’ll see that it prints out “Hello, world!” and exits. There is no event loop because we haven’t told the runtime to listen to or emit any events.

There are a couple ways we can initiate an event loop. Possibly the easiest way to visualize this is to use the standard JavaScript setInterval function to cause our application to write out “Hello, world!” every 250ms. To kill this application, you’d need to press Control-c.

function main(){
  console.log("Hello, world!");
}

setInterval(main, 250);

The above example is technically an event loop, although there aren’t really any events going on other than the setInterval function. Another way to initiate an event loop is to bind to or set up any event which interacts in some way with a file handle. In Unix, nearly everything is considered a file, so this would mean binding to a socket, file descriptor, or even screen input:

// respond to the SIGINT event
process.on('SIGINT', function(){
    process.stdout.write('\nHandling SIGINT event!\n');
    // cleanly exit with non-error status
    process.exit(0);
});

// listen to events on standard input
process.stdin.setEncoding('utf8');
process.stdin.on('data', function(chunk) {
  process.stdout.write('Hello, world!\n');
});

// Start reading from stdin so we don't exit.
process.stdin.resume();
console.log('Type anything and press ENTER...');

In the above example, process.stdin.resume() causes node.js to continually monitor the readable standard input stream for data. Binding a listener to watch for ‘SIGINT’ does not cause node to create an event loop on its own. Play around with this last example to get an idea of how the event loop works. If you’re interested in how libuv creates and maintains an event loop, there is a free uvbook available which dives into the internals of libuv.

Karma error: RangeError: Maximum call stack size exceeded

I’ve been working on a bootstrap application for express.js, angular.js, and bootstrap. The idea being that I can provide a boilerplate for quick prototyping of applications.

I was having some issues setting up karma, the test running used by angular.js. I was receiving an error:

...
	
INFO [Chrome 28.0 (Linux)]: Connected on socket id 2uhtx5qf6HDM8_A1qssQ
INFO [karma]: Delaying execution, these browsers are not ready: Chrome 28.0 (Linux)
Chrome 28.0 (Linux) controllers should .... FAILED
	RangeError: Maximum call stack size exceeded
	    at String.match (native)
	    at Object.b.event.remove (/home/jim/projects/express-angular-bootstrap-boiler/src/public/javascripts/vendor/jquery-1.9.1.min.js:3:25827)

...

I messed around with the file includes for a while, then Googled a bit for any possible issues.

It turns out that I was using bootstrap v 2.3.0, which appears to have issues with jQuery 1.9.1 as reported in this issue.

As noted in the issue, upgrading to a newer version of Twitter’s Bootstrap (2.3.2) resolved the issue.

git prepare-commit-msg with node.js

I think people often overlook using node.js as a scripting tool like ruby or python. If you search the web for examples of customized git hooks, you’ll probably have visited git-scm.com, where most of the examples are in ruby.

I wanted to show a pretty simple prepare-commit-msg hook written in node.js. This hook will append the name and description of whatever working branch you commit from. I use this technique to create a working branch for a JIRA ticket number, so all my commits to this ticket automatically include that ticket number. Then, JIRA can be configured to display github commits directly in the ticket details when that commit references the ticket in the commit message.

The script:

#!/usr/bin/env node
var exec = require('child_process').exec,
    util = require('util'),
    fs = require('fs'),
    contents = null,
    branch, desc;

console.log(process.argv);

// expect .git/COMMIT_EDITMSG
if(/COMMIT_EDITMSG/g.test(process.argv[2])){
    // look for current branch name
    branch = exec("git branch | grep '*'",
      function (err, stdout, stderr) {
        if(err){
			// git branch will fail if initial commit has not been done,
			// so we can safely skip this hook
			process.exit(0); 
		}

		// opens .git/COMMIT_EDITMSG
        contents = fs.readFileSync(process.argv[2]);
		
		// trims extra characters from start/end of line
        var name = stdout.replace('* ','').replace('\n','');
		
		// If the branch has a description, pull that
        desc = exec('git config branch.'+ name +'.description',
            function(err, stdout, stderr){
				// don't handle errors (because we write out branch anyway)
                
				// we found a description, add to 'name'
				if(stdout){ name = util.format('%s (%s)', name, stdout.replace(/\n/g,'')); }

				// '(no branch)' indicates we are in a rebase or other non-HEAD scenario
                if(name !== '(no branch)'){
				
					// Append branch name to original contents.
                    contents = util.format('%s\n\n:%s\n', contents, name);
					
					// write contents back out to .git/COMMIT_EDITMSG
                    fs.writeFileSync(process.argv[2], contents);
                    process.exit(0);
                } else {
                    process.exit(0);
                }
        });
    });
}

The above script is pretty well-commented. If you decide to use it, you may want to comment out the first console.log which dumps all arguments sent to the script.

Here’s an example of its usage.

Create and initialize a new local git repository.

$ mkdir example
$ cd example
$ git init

Then, copy the above example script into the git hooks directory under .git/hooks/prepare-commit-msg

Now, in the example directory, create and commit a file into the repository:

$ touch first
$ git add .
$ git commit -m 'Initial commit'

You’ll see the array of arguments passed through the custom git hook, but if you issue a git log, you’ll only see the commit message ‘Initial commit’. That’s because this custom hook expects a git branch, but there is no git branch until your initial commit is complete. To verify the script is working:

$ touch second
$ git add .
$ git commit -m 'Second commit'

A git log will now show you a tweaked message:

    Second commit
    
    :master

Now, you can create a branch named ‘ABC-123’ and do another commit:

$ git checkout -b ABC-123
$ touch third
$ git add .
$ git commit -m 'Third commit'

The commit message:

    Third commit
    
    :ABC-123

This hook also allows you to print out the branch description. To test, invoke the branch description editor:

$ git branch --edit-description ABC-123

I’ve added the text ‘Supporting branch for ABC-234 and ABC-567’. Save this buffer according to your editor (:wq if you’re using vim).

Add a fourth file as we have before, then verify with git log:

    Fourth commit
    
    :ABC-123 (Supporting branch for ABC-234 and ABC-567)

Because this script strips newlines from the branch description, you’d need to keep it short. Usually, that’s not a problem.

This shows how to append text to your commit message, but it can easily be modified to replace commonly misspelled words (teh -> the).

This prepare-commit-msg hook is available in my blogs repo on github.

Why I don’t recommend the Step module [node.js, npm]

I prefer asyc to step. The async module has a cleaner interface, more options, and utilities. It’s a little beefier in size, but those things are beside the point.

Here’s why I don’t really like step. Tell me what you’d expect from this function:

var Step = require('step');

Step(
	function first(){
		console.log('first');
		
		this();
	},
	
	function second(){
		var x;
		console.log('second');
		if(!x){
			// do something that should set x
		}
		return x;
	},
	
	function third(){
		console.log('third');
	}
)

Did you guess:

$ node step-example.js 
first
second

If you’re not familiar with Step, you’ll probably look at that first function and wonder what this(); actually does. Well, it is actually shorthand for next(); (I’m simplifying a little here).

Assuming you’re at least somewhat familiar with asynchronous control flow, you could assume this(); calls the next function. But, what about the second function? It’s returning a value. Does that return from second, or from Step, or from some other function? It returns from second(), but… it passes the value to third. Or, it should. Unless x is undefined, in which case it will be considered an unchained function. Now, I don’t think your variables should regularly be ‘undefined’ at the point of return, but what if you use or someone on your team uses the popular convention of calling a callback as return callback(x);. If the Step module’s convention of tweaking this semantics is ignored, another developer may look at it as a typo, “You can’t call the context like that…” Right? Also, what if someone doesn’t understand the return value can’t be undefined and comments out the middle of that function? You may have cleanup logic that isn’t getting checked in third().
We’ve all seen that happen before.

In the above example, if x was not undefined, it would be passed as a parameter to third().

It’s this inconsistency which makes me feel like Step is an accident waiting to happen in a regular development team. The source code is pretty well written and concise. I think the author has done a great job, but the usage is unintuitive and I wouldn’t recommend using the module.

On the other hand, async is beautifully intuitive.

Consider this example:

var async = require('async');

async.series([
	function first(done){
		console.log('first');
		done(null, 'first');
	},
	function second(done){
		var x;
		console.log('second');
		if(!x){
			// do something that should set x
		}
		done(null, x);
	},
	function third(done){
		console.log('third');
		
		done(null, 'third');
	}
], function(err, results){
	console.log(results);
})

async.series allows us to run a series of tasks (functions defined within the array), which pass a value to the callback (that last function you see).

If you forget to pass a value, the results will contain undefined at the index of the array. Your third function won’t be skipped unless you forget to invoke the callback. To me, this is a pretty obvious error and one that is easy to spot and correct. Here’s is the output of the above call:

first
second
first
third
[ 'first', undefined, 'third' ]

To be fair, the example of Step acts more like a waterfall function. Here’s what that would look like in async.

async.waterfall([
	function first(done){
		console.log('first');
		done(null, 'first');
	},
	function second(val, done){
		var x;
		console.log('second has val: %j', val);
		if(!x){
			// do something that should set x
		}
		done(null, x);
	},
	function(val, done){
		console.log('third has val: %j', val);
		
		done(null, 'third');
	}
], function(err, val){
	console.log('callback has val: %j', val)
});

The above code is available on github: https://github.com/jimschubert/blogs