workers: initial implementation #1159

petkaantonov · 2015-03-15T22:38:28Z

Workers are light-weight processes that are backed by OS threads.

This is an initial implementation that is behind the flag --experimental-workers. It should have very little effect on other io.js components, even if the flag is enabled. The module can be required by doing var Worker = require('worker'); - see how the tests do it in test/workers folder.

What is currently implemented:

Worker constructor which takes an entry module and an optional options object. Currently (I have some ideas for more) the only option is keepAlive:

var worker = new Worker('worker.js', {keepAlive: false})

The option defaults to true, which means the worker thread will stay alive even when its event loop is completely empty. This is because you might not have
anything to send to the worker right away when a server is started for instance.

A worker object is an event emitter, with the events 'message', 'exit', and 'error' and public methods postMessage, terminate, ref and unref:

'message' event will receive currently a single argument, that is directly the message posted from the worker thread.
'exit' event is emitted when the worker thread exits and receives the exit code as argument.
'error' event is emitted when there is an uncaught exception in the worker which would abort a normal node process. The argument is the error thrown,
if it's an error object, its builtin type, stack, name and message are retained in the copy. It will additionally contain extra properties that were in the
original error, such as .code.
postMessage([message]) where message is structured clonable* can be used to send a message to the worker thread. Additionally internal messaging support is implemented
that doesn't need to copy data or go through JS. (* currently implemented as JSON to keep PR small, more details later)
terminate([callback]) terminates the worker by using V8 TerminateExecution and uv_async_send, so the only thing that
can defer a worker's termination is when it is executing C code which shouldn't contain any infinite loop or such :-).
unref and ref work like timer's respective methods, however keep in mind that a worker will terminate if its owner terminates.

Inside a worker thread, several process-wide features are disabled for example setting the process' current working directory, title, environment variables or umask. Workers can still read these, but only the main thread can change them. Otherwise all core modules and javascript works normally inside a worker, including console.log.

Inside worker thread, the Worker constructor object is an event emitter, with the event 'message' and the method postMessage() to communicate with its owner (see tests for usage). The worker constructor can be used to construct nested workers with the worker being their owner. Passing messages from grand child to grand parent requires the child's parent to pass its message through, however transferable message ports can be implemented if use cases emerge.

Nested workers were implemented because Web Workers support them too (except Chrome) and I envision that a setup with ${CPU cores} amount of web servers with N amounts of sub-workers for each will be very useful.

2 new public process read-only properties are introduced process.isMainInstance and process.isWorkerInstance.

Advantages of workers over processes:

Internal ITC (not passing JS objects) is ridiculously fast with the wait-free message channel
User ITC is probably faster as well, not having the overhead of JSON parsing and serialiazing
- And can also transfer more kinds of objects, such as Dates, Maps, Sets, objects with circular references...
- Additionally ownership transfer of external resources like ArrayBuffers and external strings can be done without copying
- Ownership transfer of uv handles will probably require support from uv so that handles can be detached from an event loop
  and adopted by another.
Threads have less context-switch overhead, making it feasible to run many more workers than there are CPU cores
which is more graceful in high load situations. And if we implement worker-cluster, there shouldn't be the need
to use home-made round-robing scheduler.
Free of problems like process.send not always being synchronous
There is just one process to kill and no runaway processes

Disadvantages:

Cannot rely on OS to free resources upon worker exit. However the architecture that
enables freeing all resources upon Environment destruction is there and
is already used to free all resources that a new worker that doesn't do
anything but exit immediately allocates.
C code maintainers need to think about thread-safety. However, there is generally
little amount of shared resources as most stuff is tied to a v8 isolate or a uv event loop
which are exclusive to a worker.

Compared to traditional threading:

Workers are expensive to create, one does not simply spin up a worker to do 1 task and throw it away.
Workers cannot share any JS memory (this is impossible in V8), to JS users they appear pretty much as isolated as real processes

To keep the PR size small (believe it or not, deep copying will take much more code than workers alone), objects are still copied by JSON serialization. I want to implement something like the structured clone with some differences as the spec has some seriously nasty parts that are not really needed in practice but force the implementation to be so slow as to defeat the whole point of using it in the first place.

So what cannot be implemented from the algorithm:

The html5 objects like File (since we don't have those)
Named properties of Arrays (inconsistent and extremely nasty to implement using v8's api)
(implicit) If you have sparse array and define indexed properties in protototypes, then those values in prototypes are used in the resulting array. V8 has internal check that array and object prototypes are clean of indexed properties, but we don't have access to that.

What could be implemented but don't want to implement from the algorithm:

Normal properties of Map and Set. This would be inconsistent and add more stuff to implement, but certainly doable.

What I mean by inconsistent is that copying map, set and array properties is inconsistent because the properties of Date, RegExp, Boolean, Number, and String objects are dropped.

To make reviewing this easier, a description of changes in each file follows:

*test/workers/**_, _tools/test.py, Makefile
- For testing. Run make test-workers. Worker tests are not ran as a part of make test
worker.cc, worker.h, worker.js
- Majority of the worker implementation is here
util.h, util-inl.h, util.cc
- Added STATIC_ASSERT
- Added some threading helpers
- Added a hack to get a char* from js string (TODO)
- Added Remove method to ListHead
smalloc.cc, smalloc.h
- Extracted CallbackInfo class declaration to a header
- Moved ALLOC_ID to env.h
producer-consumer-queue.h
- Modified folly's wait-free single-producer single-consumer queue.
persistent-handle-cleanup.cc, persistent-handle-cleanup.h.
- Because finalizers are not guaranteed to be called even when a v8 isolate is disposed, these implement PersistentHandleVisitor that is used to walk all unfinalized weak persistent handles to release their C resources upon Environment destruction
notification-channel.cc, notification-channel.h
- These work around uv_async_send unreliability when async handles need to be unreffed
node-internals.h
- Removed NodeInstanceData - the event loop code for worker became too different from the main thread event loop code for this to be useful
node-contextify.cc
- Extracted out ContextifyScript class declaration to a header. The diff looks horrible but there is no real changes
- Added check to prevent cancellation of termination in EvalMachine method
node.js
- Added worker mode startup and cleanup registration for stdio and signal handles. (environment->CleanupHandles() is actually unused by main instance, but workers use it)
node.cc
- Added locking around process-wide methods. (If uv's rwlock still causes deadlocks in Windows 2003, it should be changed to mutex).
- Added worker-only locking for module initialization, the ApiLock() is null for main instance but workers need it because module initialization isn't termination safe.
- Made MakeCallback safe to call inside workers that are being abruptly terminated
- Changed all exit calls to env->Exit() (which only exits the process if called from main thread)
- Added an export for process._registerHandleCleanup()
- Moved atexit from LoadEnvironment to Init (probably not the right place, but it cannot be in LoadEnvironment)
- Added CreateEnvironment overload to be called from worker threads
- Extracted Environment initialization to InitializeEnvironment
- Removed all worker checks from StartNodeInstance and renamed it to RunMainInstance. Added a call to TerminateSubWorkers() before exit event is emitted.
- Added ShutdownPlatform and delete platform at the end of Start method.
handle_wrap.cc, handle_wrap.h
- Added a way to register handle cleanups upon Environment destruction.
env.h, env-inl.h
- Introduced a ClassId enum which weak persistent wrappers can use so that they can be walked upon Environment destruction and cleaned up for sure.
- Added some worker related methods
- Added set_using_cares(), which means that in Environment destructor destroy_cares is called
- Added CanCallIntoJs() method: main instance can always call into js but workers can be in the middle of termination in which case one cannot call into js anymore
- Added Exit method which does process exit if called from main thread and worker termination if called from worker thread
cares_wrap.cc
- Added InitAresOnce to do one-time process-wide initialization
- Added a call to env->set_using_cares() (see env.h above)
async-wrap.cc
- Made same changes to MakeCallback as were made to node::MakeCallback
timers.js
- Added handle cleanup registrations for timer handles so that when a worker Environment is destructed, the handles won't leak
child_process.js
- Added handle cleanup registration for the pipe handle so that when a worker Environment is destructed, the handle won't leak. The other handles probably leak too but this was the only one that is caught by current tests. There is currently an assertion in worker termination code for leaked handles.
LICENSE
- Added ProducerConsumerQueue's license

mscdex · 2015-03-15T23:02:00Z

What exactly is meant by "Named properties of Arrays?"

mscdex · 2015-03-15T23:06:23Z

lib/worker.js

+    }, null, WorkerContextBinding.ERROR);
+  };
+
+  require('util')._extend(Worker, EventEmitter.prototype);


Can this reuse the existing cached require('util')?

petkaantonov · 2015-03-15T23:08:58Z

Property that is a string or symbol as opposed to an integer (which are called indexed properties)

domenic · 2015-03-16T11:51:20Z

Epic feature :D.

Regarding structured clone, @dslomov-chromium had plans to bring that into the ES spec and presumably also into V8 directly. I am not sure what the status of that is though. It would be good to collaborate to avoid future back-compat breakage if we do ever get the chance to switch to V8's structured clone.

bnoordhuis · 2015-03-16T11:57:13Z

That's the ES7 typed objects strawman, isn't it?

vkurchatkin · 2015-03-16T11:58:00Z

https://github.com/dslomov-chromium/ecmascript-structured-clone

petkaantonov · 2015-03-16T13:58:34Z

@domenic Yea, the thing should have been implemented in V8 to begin with. Firefox for example implements it in SpiderMonkey, with callbacks for host objects.

Another solution can be patching V8 with some API additions that are already possible in v8 but just not done. E.g.:

Array::HaveIndexedPropertiesInPrototypeChain
Array::IsSparse()
Object::GetOwnRealNamedProperties()
Object::GetRealIndexedOwnProperty

Also what would be nice to have:

Object::GetRealNamedOwnProperty

But of course most optimal solution is implementing structured clone in v8, the above additions would just enable avoiding completely ridiculous things like having to call GetPropertyNames() on an array.

bnoordhuis · 2015-03-16T15:17:23Z

Object::GetRealNamedOwnProperty

You should be able to emulate that like this:

Local<Object> object = /* ... */;
Local<String> key = /* ... */;
Local<Value> property = object->GetRealNamedProperty(key);
const bool is_own_property =
    !property.IsEmpty() && object->GetRealNamedPropertyInPrototypeChain(key).IsEmpty();

Array::HaveIndexedPropertiesInPrototypeChain

Local<Array> array = /* ... */;
const uint32_t index = /* ... */;
const bool is_from_prototype =
    !array->HasRealIndexedProperty(index) &&
    !array->HasIndexedLookupInterceptor() &&
    array->Has(index);

(Or just check HasRealIndexedProperty for each element in the prototype chain.)

bnoordhuis · 2015-03-16T15:25:07Z

lib/worker.js

+  var context = this._workerContext;
+  this._workerContext = null;
+  if (typeof callback === 'function') {
+    this.on('exit', function(exitCode) {


Maybe this.once() so the closure doesn't stay around?

petkaantonov · 2015-03-16T15:34:07Z

@bnoordhuis The check should be much cheaper than emulation, I mean at the cost of calling HasRealIndexedProperty alone you could have already retrieved a real own indexed property. And only if it's undefined would you need to check if it's a hole, which is a rare case.

bnoordhuis · 2015-03-16T15:38:51Z

src/env-inl.h

+      to_remove = hc;
+      break;
+    }
+  }


An O(n) operation per closed handle is kind of unfortunate...

Yes I thought about that and we should consider changing this to a structure that allows faster removal

Or if the HandleCleanup object is available to the registerer, removal is constant time

Notable changes * deps: Fixed an out-of-band write in utf8 decoder. This is an important security update as it can be used to cause a denial of service attack.

See next commit for the actual fix. PR-URL: nodejs#1373 Reviewed-By: Fedor Indutny <fedor@indutny.com>

Update AUTHORS list using tools/update-authors.sh PR-URL: nodejs#2100 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James Hartig <fastest963@gmail.com>

Improve performance by: + Not leaking the `arguments` object! + Getting the last character of a string by index, instead of with `.substr()` or `.slice()` Improve code consistency by: + Using `[]` instead of `.charAt()` where possible + Using a function declaration instead of a var declaration + Using `.slice()` with clearer arguments + Checking if `dir` is truthy in `win32.format` (added tests for this) Improve both by: + Making the reusable `trimArray()` function + Standardizing getting certain path statistics with the new `win32StatPath()` function PR-URL: nodejs#1778 Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com> Reviewed-By: Roman Reiss <me@silverwind.io>

Path functions being benchmarked are: * format * isAbsolute * join * normalize * relative * resolve PR-URL: nodejs#1778 Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com> Reviewed-By: Roman Reiss <me@silverwind.io>

This reverts commit 9436a86.

PR-URL: nodejs#2112 Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>

Every npm version bump requires a few patches to be floated on node-gyp for io.js compatibility. These patches are found in 03d1992, 5de334c, and da730c7. This commit squashes them into a single commit. PR-URL: nodejs#990 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

The delay-load hook allows node.exe/iojs.exe to be renamed. See efadffe for more background. PR-URL: nodejs#1433 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

PR-URL: nodejs#2099 Author: Jeremiah Senkpiel <fishrock123@rocketmail.com>

The original test uses a variable to explicitly count how many times the callback is invoked. This patch uses common.mustCall() to track if the callback is called or not. This makes the test more robust, as we don't explicitly hardcode the number of times to be called. PR-URL: nodejs#2122 Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

gdbinit provided by V8 can be very useful for low-level debugging of crashes in node and in binary addons. Most useful commands at 'jst' for JS stack traces and 'job' for printing a heap object. This patch installs the file at $PREFIX/share/doc/node/gdbinit. Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> PR-URL: nodejs#2123

Add a check for crypto before using it, similar to how other tests work. PR-URL: nodejs#2129 Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp> Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>

alubbe · 2015-07-08T10:18:36Z

Can we do anything to help out @petkaantonov? It would be awesome to play around with this!

bnoordhuis · 2015-07-08T10:31:27Z

@alubbe There are still some outstanding issues / comments. I don't think anyone will object if you adopt this PR and make the necessary changes. Just fork Petka's branch and add your changes on top.

petkaantonov · 2015-07-08T12:32:14Z

#2133 replaces this PR

GnorTech · 2016-12-13T02:44:29Z

For those who want to write Node.js code in multithread program: NW.js implemented this by enabling Node.js in Web Workers: https://nwjs.io/blog/v0.18.4/

mscdex reviewed Mar 15, 2015
View reviewed changes

bnoordhuis reviewed Mar 16, 2015
View reviewed changes

Fishrock123 and others added 14 commits July 3, 2015 17:10

2015-07-04 io.js v2.3.3 Release

2faae58

Notable changes * deps: Fixed an out-of-band write in utf8 decoder. This is an important security update as it can be used to cause a denial of service attack.

Working on v2.3.4

3754920

test: add test for missing close/finish event

9436a86

See next commit for the actual fix. PR-URL: nodejs#1373 Reviewed-By: Fedor Indutny <fedor@indutny.com>

doc: update AUTHORS list

4614033

Update AUTHORS list using tools/update-authors.sh PR-URL: nodejs#2100 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James Hartig <fastest963@gmail.com>

Revert "test: add test for missing close/finish event"

fb05c8e

This reverts commit 9436a86.

deps: upgrade to npm 2.12.1

863cdbd

PR-URL: nodejs#2112 Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>

win,node-gyp: enable delay-load hook by default

dd523c7

The delay-load hook allows node.exe/iojs.exe to be renamed. See efadffe for more background. PR-URL: nodejs#1433 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

doc: document current release procedure

84b3915

PR-URL: nodejs#2099 Author: Jeremiah Senkpiel <fishrock123@rocketmail.com>

test: add missing crypto checks

2ba1740

Add a check for crypto before using it, similar to how other tests work. PR-URL: nodejs#2129 Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp> Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>

petkaantonov added 6 commits July 8, 2015 13:40

workers: initial implementation

ccc3277

fix most jslint warnings

cff61ba

address inline comments

327ae0a

Commonize duplicated code

0a0bb58

Comment out tmp dir using fs tests for now

06689de

Fix remaining lint errors

5c4dbba

petkaantonov force-pushed the workers-implementation branch from fab6db9 to 5c4dbba Compare July 8, 2015 12:27

petkaantonov mentioned this pull request Jul 8, 2015

workers: initial implementation #2133

Closed

4 tasks

petkaantonov closed this Jul 8, 2015

formula1 mentioned this pull request Jul 11, 2015

Threads runtimejs/runtime#75

Open

benjamingr mentioned this pull request Aug 31, 2017

please develop native threads that can load native modules with require and share/lock objects ayojs/ayo#31

Open

benjamingr mentioned this pull request May 23, 2018

npm module name / Node.js built-in worker module blake-regalia/worker.js#1

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

workers: initial implementation #1159

workers: initial implementation #1159

petkaantonov commented Mar 15, 2015

mscdex commented Mar 15, 2015

mscdex Mar 15, 2015

petkaantonov commented Mar 15, 2015

domenic commented Mar 16, 2015

bnoordhuis commented Mar 16, 2015

vkurchatkin commented Mar 16, 2015

petkaantonov commented Mar 16, 2015

bnoordhuis commented Mar 16, 2015

bnoordhuis Mar 16, 2015

petkaantonov commented Mar 16, 2015

bnoordhuis Mar 16, 2015

petkaantonov Mar 16, 2015

petkaantonov Mar 16, 2015

alubbe commented Jul 8, 2015

bnoordhuis commented Jul 8, 2015

petkaantonov commented Jul 8, 2015

GnorTech commented Dec 13, 2016

workers: initial implementation #1159

workers: initial implementation #1159

Conversation

petkaantonov commented Mar 15, 2015

mscdex commented Mar 15, 2015

mscdex Mar 15, 2015

Choose a reason for hiding this comment

petkaantonov commented Mar 15, 2015

domenic commented Mar 16, 2015

bnoordhuis commented Mar 16, 2015

vkurchatkin commented Mar 16, 2015

petkaantonov commented Mar 16, 2015

bnoordhuis commented Mar 16, 2015

bnoordhuis Mar 16, 2015

Choose a reason for hiding this comment

petkaantonov commented Mar 16, 2015

bnoordhuis Mar 16, 2015

Choose a reason for hiding this comment

petkaantonov Mar 16, 2015

Choose a reason for hiding this comment

petkaantonov Mar 16, 2015

Choose a reason for hiding this comment

alubbe commented Jul 8, 2015

bnoordhuis commented Jul 8, 2015

petkaantonov commented Jul 8, 2015

GnorTech commented Dec 13, 2016