zkplus: The API ZooKeeper should have had Documentation

• Installation
• API
  • zkplus.ZKError
  • zkplus.createClient(options)
  • Class: zkplus.ZKClient
  • zkplus.createElection(options)
  • Class: zkplus.Election
• License (MIT)
• Development Guidelines
  • Directory Layout
  • Makefile
  • Coding Style
  • Testing

About zkplus

curl localhost:8080

Installation

npm install zkplus

API

The zkplus API resembles Node's fs module quit a bit, with the caveat that data is always assumed to be JSON. That seems sensible and universal for most uses of ZK, and indeed makes this API quite a bit nicer. If you're doing something crazy like storing images and videos in ZooKeeper, you're doing it wrong, so move along.

At a high-level, this API provides facilities for creating "directories", "files", and setting "watches" (ZK only provides the latter as an actual primitive; everything else is approximated here). Additionally, there is bonus functionality to perform leader election, a la the ZK cookbook .

Hello world example:

var assert = require('assert');
var zkplus = require('zkplus');

var client = zkplus.createClient({
        servers: [{
            host: 'localhost'
            port: 2181
        }]
});

client.on('connect', function () {
        client.mkdirp('/foo/bar', function (err) {
                assert.ifError(err);
                client.rmr('/foo', function (err) {
                        assert.ifError(err);
                        client.close();
                });
        });
});

All that did was make some set of znodes in ZooKeeper recursively, and clean up.

zkplus.ZKError

Most all of the APIs from Client and Election will always return an instanceof ZKError. This Error object will have code and message properties, that are directly returned from ZooKeeper. You can switch on them with off the properties hung off the main exports of zkplus (which are just re-exported from node-zookeeper).

zkplus.createClient(options)

Creating a client is straightforward, as you simply invoke the createClient API, which takes an options object with the options below. Note that the servers parameter can be omitted if you only want to talk to a single ZooKeeper node; in that case, you can just use host as a top-level argument (useful for development).

var zkplus = require('zkplus');

var client = zkplus.createClient({
        host: 'localhost' // instead of servers: [{host: 'localhost'}]
});

Class: zkplus.ZKClient

This is an EventEmitter with the following events and methods.

Event: 'close'

function onClose() { }

Emitted when the client has successfully been disconnected from the ZooKeeper server. Typically emitted after a call to the close() API.

Event: 'connect'

function onConnect() { }

Emitted when the client has successfully connected to the ZooKeeper server.

Event: 'error'

function onError(err) { }

If the client driver has an unexpected error, it is sent here.

client.connect([callback])

Explicitly connects to the ZooKeeper server(s) passed in at instantiation time. You only need to call this if you explicitly passed connect: false into createClient. Optionally takes a callback of the form function (err); also emits the connect event when complete.

client.close()

Shuts down the connection to the ZooKeeper server(s). Emits the close event when done.

client.creat(path, [options], callback)

Creates a "file" (really a znode) in ZooKeeper. the options parameter may be specified, in which case, that drives the semantics. If you pass in flags to sequence, then ZK will automatically generate a new "key" for you. Either way, you can pass in ephemeral, such that ZK will remove the znode when this client disconnects. Lastly, you can pass in an object parameters that gets serialized as the data blob. callback is of the form function (err, path) where the path is provided back to you (essential if sequence was set).

var opts = {
        flags: ['sequence'],
        object: {
                hostname: require('os').hostname()
        }
};
client.creat('/foo/bar', opts, function (err, path) {
        assert.ifError(err);
        console.log(path); // => /foo/bar/00000000
});

client.get(path, callback)

Returns the data associated with a znode, as a JS Object (remember, zkplus assumes all data is JSON). Callback is of the form function (err, object)

client.get('/foo/bar/00000000', function (err, obj) {
        assert.ifError(err);
        console.log('%j', obj); // => { "hostname": "your_host_here" }
});

client.getState()

Returns the state of the underlying ZooKeeper driver. Possible states are:

• expiredSession
• authFailed
• connecting
• associating
• connected
• unknown (shouldn't happen)

Realistically, you probably only care about connected.

console.log(client.getState()); // => 'connected'

client.mkdirp(path, callback)

Does what you think it does. Recursively creates all znodes specified if they don't exist. Note this API is idempotent, as it will not error if the path already exists. Callback is of the form function (err).

client.mkdirp('/foo/bar/baz', function (err) {
        assert.ifError(err);
});

client.put(path, object, [options], callback)

Creates or overwrites path with object. You can pass in an options with flags of ephemeral if you want that behavior. This API is idempotent. Callback is of the form function (err).

client.put('/foo/bar/hello', {value: 'world'}, function (err) {
        assert.ifError(err);
});

client.readdir(path, callback)

Lists all nodes under a given path, and returns you the keys as relative paths only. The keys returned will be sorted in ascending order. callback is of the form function (err, nodes).

client.readdir('/foo/bar', function (err, nodes) {
        assert.ifError(err);
        console.log(nodes.join()); // => ['00000000', 'baz']
});

client.rmr(path, callback)

Recursively deletes everything under a given path. I.e., what you'd think rm -r would be. callback is of the form function (err).

client.rmr('/foo/bar', function (err) {
        assert.ifError(err);
});

client.stat(path, callback)

Returns a ZK stat object for a given path. ZK stats look like:

{
        czxid,           // created zxid (long)
        mzxid,           // last modified zxid (long)
        ctime,           // created (long)
        mtime,           // last modified (long)
        version,         // version (int)
        cversion,        // child version (int)
        aversion,        // acl version (int)
        ephemeralOwner,  // owner session id if ephemeral, 0 otw (string)
        dataLength,      //length of the data in the node (int)
        numChildren,     //number of children of this node (int)
        pzxid            // last modified children (long)
}

Reference the ZooKeeper documentation for more info.

client.stat('/foo', function (err, stats) {
        assert.ifError(err);
        console.log('%j', stats);  // => stuff like above :)
});

client.unlink(path, [options], callback)

Removes a "file" from ZooKeeper. options allows you to pass in a version field of the znode you're trying to purge (i.e., test/set semantics). Otherwise the object is stat'd first and then removed (whatever the version is). callback is of the form function (err).

client.unlink('/foo', function (err) {
        assert.ifError(err);
});

client.unlink('/foo', {version: 0}, function (err) { ... });

client.update(path, object, [options], callback)

Allows you to overwrite the data associated with a given "file". Again, object is JS Object. options is just link unlink, it's not required, but it allows you to pass in a version field for test/set. callback is of the form function (err).

client.update('/foo', { hello: 'world' }, function (err) {
        assert.ifError(err);
});

client.watch(path, [options], callback)

The watch API makes usable the atrociousness that are ZooKeeper notifications (although, as unusable as they are, they're one of its most useful features). Using this API, you are able to set watches any time the content of a single node changes, or any time children are changed underneath that node. Unlink the raw ZooKeeper API, this will also automatically "rewatch" for you, such that future changes are still fired through the same listener.

The defaults for this API are to listen only for data changes, and not to return you the initial data (i.e., assume you already know what you've got, and just want to get notifications about it). The options parameter drives the other behavior, and specifically allows you to set two flags currently: method and initialData. method defaults to data, and the semantics are such that only content changes to the znode you've passed in via path will be listened for. If you set method to list, then the semantics of the watch are to notify you when any children change (add/del) _under_ path. You cannot listen for both simultaneously; if you want both, you'll need to set two watches. On data watches, the returned listener will fire data events. On list watches, the returned listener will fire children events. In either case, the listener will also have a stop() method on it which takes no arguments and closes down the watch.

Additionally, the semantics are not to perform a get, but to only notify you on updates. Setting initialData to true will make the watch fire once "up front".

callback is of the form function (err, listener).

client.watch('/foo', function (err, listener) {
        assert.ifError(err);
        listener.on('error', function (err) {
                console.error(err.stack);
                process.exit(1);
        });

        listener.on('data', function (obj) {
                console.log('%j', obj); // => updated record
                listener.stop(); // => shutdown the listener
        });
});

client.watch('/foo', { method: 'list' }, function (err, listener) {
        assert.ifError(err);
        listener.on('error', function (err) {
                console.error(err.stack);
                process.exit(1);
        });

        listener.on('children', function (children) {
                console.log('%j', children); // => ['00000000', 'bar', ...]
                listener.stop();
        });
});

zkplus.createElection(options)

zkplus additionally has an "experimental" (albeit functional) ZooKeeper leader election class (this is basically a rewrite of node-leader on top of zkplus).

Leader election is one of the classic use cases for ZooKeeper, and is described in their cookbook. This implementation is subtly different from that, as it explicitly leaves in what their literature describes as "thundering herd" because in every use case I've seen for it, frankly the herd isn't that big, and it's highly common that non-leaders need to know when there's a new leader (i.e., to change slave properties). To that end, the election class returned here allows one to monitor when $self is the new leader, and when someone else has become the leader. Lastly, this can be done without actually participating in the election itself. Note that this API is build on top of ZKClient.

var os = require('os);
var zkplus = require('zkplus');

var election = zkplus.createElection({
        client: client,   // returned from zkplus.createClient()
        path: '/foo/bar', // where to hold the election
        object: {         // Data to put in our ephemeral node
                hostname: os.hostname()
        }
});

// Need to explicitly vote if we want to participate in the election,
// as opposed to just observing it.
election.vote(function (err, isLeader) {
        assert.ifError(err);
        console.log(isLeader); // => 'true' || 'false'
});

Class: zkplus.Election

This is an EventEmitter with the following events and methods.

Event: 'close'

function onClose() { }

Emitted when the election has successfully been terminated from the ZooKeeper server. Typically emitted after a call to the stop() API.

Event: 'error'

function onError(err) { }

If the client driver has an unexpected error, it is sent here.

Event: 'leader'

function onLeader() { }

Emitted when $self is the new leader.

Event: 'newLeader'

function onNewLeader(leader) { }

Emitted when another node has assumed the role of leader. leader is the relative path; you can get the contents of it by doing zkClient.get(path + '/' + leader, function (err, obj) { });

election.getLeader(callback)

Returns the name and data of whichever node in the election is currently the leader. callback is of the form function (err, leader, object) where leader is the znode name, and object is the data for that node.

election.getLeader(function (err, leader, object) {
        assert.ifError(err);
        console.log(leader);       // => '00000000'
        console.log('%j', object); // => { hostname: 'foo' }
});

election.isLeader(callback)

Returns whether or not $self is the leader. callback is of the form function (err, isLeader), where isLeader is a boolean.

election.isLeader(function (err, isLeader) {
        assert.ifError(err);
        console.log(isLeader); // => 'true' || 'false'
});

election.stop()

Shuts down this election, and takes $self out of the voting process (if you ever called vote()). Emits the close event when complete.

election.on('close', function () {
    // all done
});
election.stop();

election.vote(callback)

Registers $self as a participant in the election (if you don't explicitly call this, you're only going to be observing someone else's election). The artifact of creating this is that a new ephemeral sequence znode will have been created under path, that was passed in at instantiation time. Calling stop (or crashing, exiting the process, etc.) will clean up and remove $self from the election. callback is of the the form function (err, isLeader) (just like election.isLeader().

election.vote(function (err, isLeader) {
        assert.ifError(err);
        console.log(isLeader); // => 'true' || 'false'
});

License (MIT)

Copyright (c) 2012 Mark Cavage, All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE

Development Guidelines

I definitely would like to see others contribute here, and in particular add some of the ZK recipes (locks, queues, etc.) as first-class citizens to this API. In order to hack on this project, read this.

tl;dr run ZK_HOST=<your ZK server IP here> make prepush before sending a pull request.

Directory Layout

build/          Built bits.
deps/           Git submodules and/or commited 3rd-party deps should go
                here. See "node_modules/" for node.js deps.
docs/           Project docs. Uses restdown from `deps`.
lib/            JavaScript source files.
node_modules/   Node.js deps, either populated at build time or commited.
                See Managing Node Dependencies.
test/           Test suite.
tools/          Miscellaneous dev/upgrade/deployment tools and data.
Makefile        See below.
package.json    npm module info, if applicable (holds the project version)
README.md       obvious

Makefile

This repos has a Makefile that defines the following targets:

• all: builds all intermediate objects (e.g., binaries, executables, docs, etc.). This is the default target.
• check: checks all files for adherence to lint, style, and other repo-specific rules not described here.
• clean: removes all built files
• prepush: runs all checks/tests required before pushing changes to the repo
• docs: builds documentation (restdown, man pages)
• test: runs the automated test suite
• cover: generates a coverage report.

Coding Style

Basically tab-free "C style" coding. 80 columns, 8 space indentation, etc. Run make check to validate lint and style.

Testing

In order to test this, you must have a ZooKeeper server running somewhere. By default, it's assumed to be on localhost but you can override with the environment variable ZK_HOST (and optionally ZK_PORT). Also, to get (lots of) debug logging, set LOG_LEVEL=trace. For example:

LOG_LEVEL=trace ZK_HOST=192.168.1.2 make test

The same goes for the make cover task.