Getting Started

Data scraping and processing code is organised into modular and extendable jobs written in JavaScript or CoffeeScript. A typical node.io job consists of of taking some input, processing / reducing it in some way, and then outputting the emitted results, although no step is compulsory. Some jobs may not require any input, etc.

Running jobs

Jobs can be run from the command line or through a web interface. To run a job from the command line, run

$ node.io myjob

To run jobs through the web interface, copy your jobs to ~/.node_modules and run

$ node.io-web

Basic examples

Let's run through some simple examples highlighting the basic anatomy of a job. Each example includes a JavaScript and CoffeeScript version and omits the required var nodeio = require('node.io');

Example 1: Hello World!

hello.js

var methods = {
    input: false,
    run: function() {
        this.emit('Hello World!');
    }
};

exports.job = new Job(methods);

hello.coffee

class Hello extends nodeio.JobClass
    input: false
    run: (num) -> @emit 'Hello World!'
    
@class = Hello
@job = new Hello()

Example 2: Double each element of input

double.js

var methods = {
    input: [0,1,2],
    run: function(num) {
        this.emit(num * 2);
    }
};

exports.job = new Job(methods);

double.coffee

class Double extends nodeio.JobClass
    input: [0,1,2]
    run: (num) -> @emit num * 2
    
@class = Double
@job = new Double()

Example 3: Extend the previous example to quadruple elements

quad.js

var double = require('./double').job;

var methods = {
    run: function(num) {
        this.__super__.run(num * 2);
        //Same as: this.emit(num * 4)
    }
};

exports.job = new Job(methods);

quad.coffee

Double = require('./double').Class

class Quad extends Double
    run: (num) -> super num * 2
    
@class = Quad
@job = new Quad()

Advanced examples

By default, node.io reads input from STDIN (elements are separated by \n or \r\n), and writes output to STDOUT.