Writing Workers in Node.js

Notice -- For the NPM Certificate Error. Please use the following in your ".worker" file, instead of build "npm install":

build "npm config set strict-ssl false; npm install --production"

Node.js is an evented language that brings the well-known Javascript language to server-side development, using Google's V8 runtime. The evented model of programming lends itself nicely to the asynchronous nature of workers, making it a natural fit for IronWorker. This article will walk you through getting your Node.js workers running on IronWorker, but you should still be familiar with the basics of IronWorker.

Table of Contents

Quick Start

Get the CLI

We've created a command line interface to the IronWorker service that makes working with the service a lot easier and more convenient. It does, however, require you to have Ruby 1.9+ installed and to install the iron_worker_ng gem. Once Ruby 1.9+ is installed, you can just the following command to get the gem:

Command Line

$ gem install iron_worker_ng

Create Your Configuration File

The CLI needs a configuration file or environment variables set that tell it what your credentials are. We have some pretty good documentation about how this works, but for simplicity's sake, just save the following as iron.json in the same folder as your .worker file:

iron.json

{
  "project_id": "INSERT YOUR PROJECT ID HERE",
  "token": "INSERT YOUR TOKEN HERE"
}

You should insert your project ID and token into that iron.json file. Then, assuming you're running the commands from within the folder, the CLI will pick up your credentials and use them automatically.

Write Your Node.js Worker

hello_worker.js

console.log("Hello World from Node.js.");

Accessing the Params and Config Variables.

To access the contents of the configuration and payload variables from within your worker use the following helpers we've included in your environment. see source for these helpers here.

hello_worker.js

var worker = require('node_helper');
console.log("params:", worker.params);
console.log("config:", worker.config);
console.log("task_id:", worker.task_id);

Create a .worker File

Worker files are a simple way to define your worker and its dependencies. Save the following in a file called hello.worker:

hello.worker

# set the runtime language; this should be "node" for Node.js workers
runtime "node"
# exec is the file that will be executed when you queue a task
exec "hello_worker.js" # replace with your file

To change your worker's version, you may place stack "node-0.10" (e.x.) in your .worker file, for more see .worker syntax.

Upload Your Worker

Command Line

$ iron_worker upload hello

That command will read your .worker file, create your worker code package and upload it to IronWorker. Head over to hud.iron.io, click the Worker link on your projects list, then click the Tasks tab. You should see your new worker listed there with zero runs. Click on it to show the task list which will be empty, but not for long.

Let’s quickly test it by running:

iron_worker queue hello

Now look at the task list in HUD and you should see your task show up and go from "queued" to "running" to "completed".

Now that we know it works, let’s queue up a bunch of tasks from code. Note: Once you upload a code package, you can queue as many tasks as you'd like against it. You only need to re-upload the code package when your code changes.

Queue Up Tasks for Your Worker

Once your code has been uploaded, it's easy to queue a task to it. It's a single, authenticated POST request with a JSON object. The example below queues up a task for your NodeWorker. Just insert your project ID and token at the bottom (that third argument is the name of your worker).

enqueue.js

var https = require("https");

function queue_task(project, token, code_name) {
  // Build the payload
  var payload = {
    "arg1": "Test",
    "another_arg": ["apples", "oranges"]
  };

  var req_json = {
    "tasks": [{
      "code_name": code_name,
      "payload": JSON.stringify(payload)
    }]
  }

  // Convert the JSON data
  var req_data = JSON.stringify(req_json);

  // Create the request headers
  var headers = {
    'Authorization': 'OAuth ' + token,
    'Content-Type': "application/json"
  };

  // Build config object for https.request
  var endpoint = {
    "host": "worker-aws-us-east-1.iron.io",
    "port": 443,
    "path": "/2/projects/" + project + "/tasks",
    "method": "POST",
    "headers": headers
  };

  var post_req = https.request(endpoint, function(res) {
    console.log("statusCode: ", res.statusCode);

    res.on('data', function(d) {
      process.stdout.write(d);
    });
  });

  post_req.write(req_data)
  post_req.end();

  post_req.on('error', function(e) {
    console.error(e);
  });
}

queue_task("INSERT PROJECT ID", "INSERT TOKEN", "NodeWorker");

Save this as "enqueue.js" and use node enqueue.js to queue up the task to your worker. You should get a response similar to this:

statusCode:  200
{"msg":"Queued up","status_code":200,"tasks":[{"id":"4f9ecdd01bab47589b02a097"}]}

Note: For most people, calling the API by hand is overkill. Please make sure to check out our node client libraries, including a community supported and potentially more active project called node-ironio by Andrew Hallock. Thanks Andrew!

Deep Dive

Payload Example

Retrieving the payload in Node.js is the same as it is on any other language. Retrieve the -payload argument passed to the script, load that file, and parse it as JSON.

We've included a useful helper module in node to assist in retrieving the payload and configuration variables in node. Simply require the helper module and call config, params, task_id.

payload.js

var worker = require('node_helper');
console.log("params:", worker.params);

// you can also access the following
console.log("config:", worker.config);
console.log("task_id:", worker.task_id);

Packaging Worker Dependencies using Node

dependencies with Node require that you create a package.json file To generate a package.json the following more info:npm init

npm-init

when adding and installing modules run then following to automatically update your package.json manifest.

npm install <module name> --save

Ensuring your script exits with the right exit code

It is important in some cases to declare a explicit exit code to give our systems a indication if your worker has completed sucessfully or failed. this also prevents instances where your worker may just hang or wait. In your worker:

process.exit(1);
process.exit(0);

Local build

requirements - package.json with included dependencies -/node_modules directory

If you're using NPM modules within your worker, you're going to need to package those dependencies when you upload the worker. To do this, add

dir "node_modules"

and

file "package.json"

to your .worker file:

hello.worker

# set the runtime language; this should be "node" for Node.js workers
runtime "node"
# exec is the file that will be executed when you queue a task
exec "hello_worker.js" # replace with your file
dir "node_modules" # include dependency files when uploading
file "package.json" # include dependency manifest when uploading

Remote build

requirements - package.json with included dependencies

If you're using NPM modules within your worker, you're going to need to package those dependencies when you upload the worker. To do this, add a dir "node_modules" line and a file "package.json" line to your .worker file:

hello.worker

runtime "node"
exec "hello_worker.js" # replace with your file
file "package.json" # include dependency manifest when uploading
build "npm install" # run npm install
# build your dependencies remotely from package.json
remote # you can use "full_remote_build true" or shorthand "remote"