C++ Add-ons for Node.js v4 - NodeSource

The NodeSource Blog

You have reached the beginning of time!

All Posts

Node.js

C++ Add-ons for Node.js v4

by:

Rod Vagg Rod Vagg

in Node.js on Sep 23 2015

NAN, or Native Abstractions for Node.js (a tongue-in-cheek backronym), exists to help ease the complexity of writing C++ add-ons for Node.js that support the changing landscape of the V8 API as it has evolved, and will continue to evolve.

NAN: What and why

Until recently, the 1.x series of NAN releases has been providing support for the full range of maintained Node.js and io.js releases, from Node.js 0.8, 0.10, 0.12 to io.js 1.x and 2.x. This also includes the versions of io.js used in both Electron / Atom and NW.js.

But now, as of io.js v3 and Node.js v4, we have a new series of V8 releases that have introduced new API shifts that are large enough to warrant changing significant parts of NAN. The authors of NAN have taken this chance to do a major rework of the API to move it away from being primarily a set of macros to being a light wrapper layer over newer versions of V8 and an emulation layer over older versions—providing mirrors of new APIs that don't exist in these old versions. So, as of NAN v2, programming against the NAN API is much closer to programming directly against V8 in terms of the patterns and datatypes you have to use.

V8 versions v4.3 (skipped by io.js) and v4.4 (included in io.js v3) and continued in v4.5 (in Node.js v4) introduce some new API features and remove existing ones, and it has become impractical to keep on covering over the changes. NAN's promise is to provide a single API to develop against, not a forever-stable API, it was always anticipated that it would change and adapt to V8 but do so in a way that you get to remain compatible with older versions of V8.

A quick recap of V8 API drift

The dramatic departure from the V8 API that we had become used to for Node.js v0.8 and v0.10 necessitated a compatibility layer for writing add-ons if you wanted to maintain support across actively supported versions of Node.js. What's more, it became clear that V8 was a moving target, with the team willing to break backward-compatibility at relatively short notice and in ways that make it difficult keep up.

Some of the most significant shifts since the somewhat stable days of Node.js v0.8 (V8 v3.11) and Node.js v0.10 (V8 v3.14) as we moved into Node.js v0.12 (V8 3.28) and the early io.js series (V8 3.31 and 4.1+):

  • The removal of the Arguments object, replacing it with a completely new API for JavaScript-accessible methods around the FunctionCallbackInfo and PropertyCallbackInfo classes. Return values are now set on the *CallbackInfo object rather than returned from the method.
  • The removal of Persistent from the Handle class hierarchy so that they are no longer easily interchangeable with Local handles.
  • The introduction of Isolate as a required argument for the creation of new JavaScript objects and many other APIs.

Fast-forward to io.js v3.x and Node.js v4, we come to V8 v4.4 and v4.5 with some new major changes:

  • The deprecation of Handle, with its complete removal slated for a coming release.
  • The introduction of a new Maybe type and MaybeLocal handle to represent values that may or may not exist. These are now used by a number of basic types, giving you additional hoops to jump through to get at their underlying values.
  • The removal of SetIndexedPropertiesToExternalArrayData and related APIs which Node has previously relied upon for its Buffer implementation. This has forced a complete rewrite of Buffer, now implemented on top of Uint8Array. Thankfully we had a bit of advance warning from the V8 team although this API change was the primary reason for the io.js v3 delay and the fact that we skipped shipping V8 4.3.

If you're interested in all of the minor changes and a full catalogue of changes in the C++ API, refer to this document which also lists future API changes the V8 team has planned in the short-term.

All of the major changes combined tell a story of the changing ideas of the V8 team regarding what an ideal and safe API is, but it also makes clear that there isn't much appetite for maintaining the kind of API stability that we are used to in JavaScript-land, making it difficult for developers trying to extend Node.js at the native layer.

A change of approach

As mentioned above, NAN started off as a collection of macros and a few helper classes and evolved this approach through v0.x and v1.x. But with new maintainers, particularly Benjamin Byholm, and the complexity of dealing with such broad API changes, NAN v2 is less a macro soup and more a compatibility layer. The new approach taken for NAN v2 is to present an API that is similar to recent versions of V8, providing light wrappers when targeting recent releases of V8 and more complete implementations for older versions.

A prime example of this is the Nan::FunctionCallbackInfo and Nan::PropertyCallbackInfo classes and the associated JavaScript-accessible method signatures. In the past you would have to use NAN_METHOD(name) to implement a JavaScript-accessible method and this in turn would translate to a function taking a v8::Arguments object for older Node and v8::FunctionCallbackInfo for newer Node. But now, even though this macro is still available, it's not necessary to use it, as NAN provides its own method signatures that can be applied via Nan::SetMethod() and related functions. For newer versions of V8, the Nan::*CallbackInfo classes are lightweight wrappers over the V8 equivalents, while for older versions of V8 they are more complete implements, presenting the new API while interacting with the very different V8 API.

A minimal example

#include <nan.h>

// This method signature magically works from Node.js v0.8 up to
// through io.js to Node.js v4, even though it looks nothing like
// the signature normally required when writing against
// Node.js 0.8 and 0.10.
// It can still be written as NAN_METHOD(Method) if desired.
void Method(const Nan::FunctionCallbackInfo<v8::Value>& info) {
  info.GetReturnValue().Set(Nan::New("world").ToLocalChecked());
}

NAN_MODULE_INIT(Init) {
  // Note the replacement of NanNew() with the namespaced Nan::New().
  // The new MaybeLocal API requires ToLocalChecked() for many V8
  // types.
  v8::Local<v8::Function> helloFn = Nan::GetFunction(
      Nan::New<v8::FunctionTemplate>(Method)).ToLocalChecked();
  Nan::Set(target, Nan::New("hello").ToLocalChecked(), helloFn);
}

NODE_MODULE(hello, Init)

More practical examples can be found in npm, look for add-ons that are using at least version 2 of NAN. This example from bignum implements the add() method that can be used to add one BigNum object to another:

NAN_METHOD(BigNum::Badd)
{
  BigNum *bignum = Nan::ObjectWrap::Unwrap<BigNum>(info.This());

  BigNum *bn = Nan::ObjectWrap::Unwrap<BigNum>(info[0]->ToObject());
  BigNum *res = new BigNum();

  BN_add(&res->bignum_, &bignum->bignum_, &bn->bignum_);

  WRAP_RESULT(res, result);

  info.GetReturnValue().Set(result);
}

Important changes here are the use of Nan::ObjectWrap instead of node::ObjectWrap, the use of info instead of args which is actually a Nan::FunctionCallbackInfo mirroring the v8::FunctionCallbackInfo implementation, hence the new style return value setting at the end of the method. More on all of this below.

Major changes

Project ownership and management

On the non-technical side, the NAN project has been moved into the nodejs organisation on GitHub and is now maintained by the Addon API working group. The repository's new home is https://github.com/nodejs/nan.

The Nan namespace

Most exports from NAN are exposed in the new Nan namespace. This was chosen over the more idiomatic nan due to a conflict with the function of the same name in the commonly used <math.h>. Even many of the old NAN macros are now types, templates, functions or other elements that can be namespaced. Only a few macros remain global, including:

Support for Maybe types

There are two new classes in V8, MaybeLocal and Maybe.

In the words of the V8 header documentation on Maybe:

A simple Maybe type, representing an object which may or may not have a value, see https://hackage.haskell.org/package/base/docs/Data-Maybe.html. If an API method returns a Maybe<>, the API method can potentially fail either because an exception is thrown, or because an exception is pending, e.g. because a previous API call threw an exception that hasn't been caught yet, or because a TerminateExecution exception was thrown. In that case, a Nothing value is returned.

So, for many V8 APIs that return primitive types, including bool, double, int32_t, int64_t, uint32_t and double, they will do so now by wrapping it in a Maybe object. You can check whether there values inside these objects with the obj.IsNothing() or the opposite, obj.IsJust(). You can fetch the raw value from Maybe with obj.FromJust() but your program will crash if it's actually a Nothing. Alternatively, use the obj.FromMaybe(default_value) method to fetch a raw value or a default value in the case of a Nothing.

As indicated by the V8 documentation, this concept is inspired by Haskell style monads ... yay ... although you may be best to think of it more akin to a Promise in that it encapsulates a state and a possible value or error.

Stepping beyond primitives, enter MaybeLocal:

A MaybeLocal<> is a wrapper around Local<> that enforces a check whether the Local<> is empty before it can be used. If an API method returns a MaybeLocal<>, the API method can potentially fail either because an exception is thrown, or because an exception is pending, e.g. because a previous API call threw an exception that hasn't been caught yet, or because a TerminateExecution exception was thrown. In that case, an empty MaybeLocal is returned.

It's important to note that you only get MaybeLocals returned from a limited set of V8 types that were previously returned as simple Locals, including Array, Boolean, Number, Integer, Int32, Uint32, String, RegExp, Function, Promise, Script and UnboundScript.

A MaybeLocal has a simple obj.IsEmpty() method to check whether there is a value inside the Local. You can retrieve the underlying Local using the obj.ToLocalChecked() method but like Maybe#FromJust(), if the Local is empty, your program will crash. Also like Maybe, there is an obj.FromMaybe(default_value) which you can provide with a new Local to be used as a default if the MaybeLocal has an empty Local.

The reason for introducing this additional abstraction layer (recall that Local already has an IsEmpty() method!) according to the V8 team:

... it's important to always assume that API methods can return empty handles. To make this explicit, we will make those API methods return MaybeLocal<> instead of Local<>

Like a lot of recent changes to the V8 API, this is an attempt to increase safety for V8 embedders.

NAN deals with this new API by providing its own version of these two classes, Nan::Maybe and Nan::MaybeLocal. There is also Nan::Nothing and Nan::Just. When compiling against newer V8, these are simple wrappers but when compiling against older versions of V8 you get a re-implementation of what you are missing.

An additional step that NAN v2 has taken to accommodate the introduction of the Maybe types is to expose some utility functions to help deal with V8 APIs that now deal with Maybe or MaybeLocal but don't in previous versions. The following functions, so far, have variants in the current V8 API that either return a Maybe type or accept one as an argument. For maximum portability, opt for using the NAN version.

So, it's time to grok Maybe and accept it as part of your C++. Be sure to stick to the Nan namespaced versions if you want portable code.

NanNew() → Nan::New()

Nan::New() is the namespaced version of the old NanNew() but it's been completely rewritten to be more flexible and be cleverer at matching the types you want to use with it. It's important that you use Nan::New() to create new JavaScript objects because of the differences in the New() APIs for various objects across V8 versions since Node.js v0.10, NAN hides all of these discrepancies and will continue to do so.

Additionally it also now supports the new Maybe types, so where V8 wants to give these to you, you'll get a Nan::MaybeLocal.

The old functions that return the basic singletons, such as NanUndefined() have also been namespaced:

Type conversion

Normally you would use obj->ToX() where X is a new type that you want to convert to. Perhaps a String to a Number. Because this isn't guaranteed to succeed, V8 now uses the Maybe types to give you a bit of added safety. So, for maximum portability, you should now avoid obj->ToX() and instead use the Nan::To() function. Specify the type to get what you want, for example, perhaps you are sure that the first argument of your method is a Number and you want it as a Local:

v8::Local<Number> numberArg = Nan::To<v8::Number>(info[0]).ToLocalChecked();

In this case, info[0] fetches a Local<Value> and in the past we'd have used info[0].To<Number>() to convert it, but thanks to the MaybeLocal in the middle now we have to use Nan::To() to ensure maximum compatibility. It's important to note here that we are jumping straight to ToLocalChecked() whereas the intention of the creators of MaybeLocal was that we'd first check if it was empty because doing so without first checking will crash your program. So beware.

Errors

Yep, v8::TryCatch now interacts with Maybe types, it was also modified to take an Isolate argument, so there is now a Nan::TryCatch to address this where you need it.

NAN has also moved its old error creation and throwing utilities to new namespaced functions. These can be used for maximum Node version compatibility and also simpler use of V8's exceptions. e.g. Nan::ThrowTypeError("Pretty simple to throw an error");.

Buffers

Interacting with Buffers is important for most compiled addons as it's a primary type for passing around binary data with Node.js. The new namespaced versions of buffer creation functions are:

  • Nan::NewBuffer(): Use this if you are handing off an existing char* to be owned and managed by the new Buffer. This is the most efficient way of creating a buffer but it means you have to have full confidence that the ownership of that area of memory can be safely handed-off. This is often not the case, e.g. third-party libraries that manage their own memory.
  • Nan::CopyBuffer(): Use this where you need Node to make a copy of the data you're providing. This is obviously slower than re-using the existing memory but also the safest if you don't have full control over the char* you are passing.

An example of this can be found in LevelDOWN, where the LevelDB instance in use is responsible for managing the underlying data extracted from the data store so LevelDOWN has to resort to making a copy of it:

v8::Local<v8::Value> returnValue;
if (asBuffer) {
  // TODO: could use NewBuffer if we carefully manage the lifecycle of
  // `value` and avoid an an extra allocation. We'd have to clean up
  // properly when not OK and let the new Buffer manage the data when OK
  returnValue = Nan::CopyBuffer(
                  (char*)value.data(), value.size()).ToLocalChecked();
} else {
  returnValue = Nan::New<v8::String>(
                  (char*)value.data(), value.size()).ToLocalChecked();
}

As per the TODO, it would be ideal if LevelDOWN could take responsibility for the original char* and avoid the memcpy() that happens when you call CopyBuffer().

There is also Nan::FreeCallback that can be used to define a callback function that is passed to Nan::NewBuffer() if you need a particular action to be taken when the Buffer hits the garbage collector. By default, the memory is freed with free, if this is not going to work with what you have provided NewBuffer() then implement a custom FreeCallback function. If you are passing a pointer to static memory, then provide an empty function, if you are passing something created with new then implement a function that uses delete. Nan::FreeCallback is necessary because node::smalloc::FreeCallback had to be moved to node::Buffer::FreeCallback for io.js v3 and Node.js v4 when the smalloc module was removed from core.

Asynchronous work helpers

Nan::AsyncWorker and Nan::AsyncProgressWorker are helper classes that make working with asynchronous code easier. AsyncWorker was in NAN from the begining (as NanAsyncWorker) but AsyncProgressWorker came in v1.4. It works like AsyncWorker except it doesn't just have a single return point back to JavaScript, it can post on-going updates to JavaScript as the work progresses.

Additionally don't forget about Nan::Callback for managing callbacks over the life of an asynchronous execution. This helper, formally NanCallback primarily exists to ensure that the callback Function remains free from garbage collection while you are waiting for asynchronous execution to return.

Some examples of how Nan::AsyncWorker and Nan::Callback can be used to simplify working with V8 in an asynchronous environment can be found scattered through LevelDOWN, simply look through the *<i>async.cc files in the src directory.

Encodings and V8 internals

NAN v2 namespaces its encoding/decoding functions for dealing with strings and bytes, see the documentation for more details.

Nan::Utf8String should now be used in place of v8::String::Utf8Value to get its more recent functionality enhancements that are not present in past versions of Node.

Dealing with V8 internals have also been namespaced and expanded to deal with changing argument and return types as well as wholesale renaming of APIs. See the documentation for more details.

The _current Context should now be accessed via Nan::GetCurrentContext(). Interact with Isolate using Nan::SetIsolateData() and Nan::GetIsolateData().

Node.js helpers

Instead of opting for the pure Node.js versions of the following APIs, you should use the NAN implementations for maximum version compatibility:

Also, use NAN_MODULE_INIT() to define an "Init" function for Node.js add-ons due to the change from Handle to Local. Generally you'll be writing NAN_MODULE_INIT(Init) { /* ... export things here on 'target' ... */ }. Note that in older versions of Node.js you'll receive a Handle rather than a Local, this may require some massaging to get full compatibility. It's also common to have multiple init functions in a non-trivial add-on as various components need to register their exports. Such as this code from node-canvas:

NAN_MODULE_INIT(init) {
  Canvas::Initialize(target);
  Image::Initialize(target);
  ImageData::Initialize(target);
  Context2d::Initialize(target);
  Gradient::Initialize(target);
  Pattern::Initialize(target);
#ifdef HAVE_FREETYPE
  FontFace::Initialize(target);
#endif

  ...

Within an init function you should use Nan::Export() to attach properties / functions to target (a.k.a exports), this deals with both the Handle vs Local difference as well as managing some of the MaybeLocal mess for you.

What next?

NodeSource is now providing an upgrade-utils package via npm. Install it with npm install upgrade-utils -g then move to the root directory of your Node.js package and run upgrade-utils. See the documentation for more information on usage options.

The upgrade-utils utility is designed to help with the process of upgrading your packages to Node.js v4 and one of it's features is that it can do 90% of the work of converting your C++ code from NAN v1 to NAN v2. It's not all automatic, however, you'll need to pay attention to:
- brace and bracket confusion because these are just regexes. - missing returns where NAN v1 might have had an implicit returns, e.g. NanReturnNull() and NanReturnValue() which should now simply use info.GetReturnValue().Set() however are not accompanied by a return in the script.

Note that upgrade-utils doesn't just do C++, try it against your existing Node.js modules to see what it comes up with!

Get in touch: file an issue on the NAN repository if you are struggling with NAN v2 and have questions. Or, if you have in-house C++ add-ons and need some professional help, feel free to contact NodeSource and find out how we can help.

Featured Articles

Categories

The NodeSource platform offers a high-definition view of the performance, security and behavior of Node.js applications and functions.

Start for Free