Related
I have a wasm process (compiled from c++) that processes data inside a web application. Let's say the necessary code looks like this:
std::vector<JSONObject> data
for (size_t i = 0; i < data.size(); i++)
{
process_data(data[i]);
if (i % 1000 == 0) {
bool is_cancelled = check_if_cancelled();
if (is_cancelled) {
break;
}
}
}
This code basically "runs/processes a query" similar to a SQL query interface:
However, queries may take several minutes to run/process and at any given time the user may cancel their query. The cancellation process would occur in the normal javascript/web application, outside of the service Worker running the wasm.
My question then is what would be an example of how we could know that the user has clicked the 'cancel' button and communicate it to the wasm process so that knows the process has been cancelled so it can exit? Using the worker.terminate() is not an option, as we need to keep all the loaded data for that worker and cannot just kill that worker (it needs to stay alive with its stored data, so another query can be run...).
What would be an example way to communicate here between the javascript and worker/wasm/c++ application so that we can know when to exit, and how to do it properly?
Additionally, let us suppose a typical query takes 60s to run and processes 500MB of data in-browser using cpp/wasm.
Update: I think there are the following possible solutions here based on some research (and the initial answers/comments below) with some feedback on them:
Use two workers, with one worker storing the data and another worker processing the data. In this way the processing-worker can be terminated, and the data will always remain. Feasible? Not really, as it would take way too much time to copy over ~ 500MB of data to the webworker whenever it starts. This could have been done (previously) using SharedArrayBuffer, but its support is now quite limited/nonexistent due to some security concerns. Too bad, as this seems like by far the best solution if it were supported...
Use a single worker using Emterpreter and using emscripten_sleep_with_yield. Feasible? No, destroys performance when using Emterpreter (mentioned in the docs above), and slows down all queries by about 4-6x.
Always run a second worker and in the UI just display the most recent. Feasible? No, would probably run into quite a few OOM errors if it's not a shared data structure and the data size is 500MB x 2 = 1GB (500MB seems to be a large though acceptable size when running in a modern desktop browser/computer).
Use an API call to a server to store the status and check whether the query is cancelled or not. Feasible? Yes, though it seems quite heavy-handed to long-poll with network requests every second from every running query.
Use an incremental-parsing approach where only a row at a time is parsed. Feasible? Yes, but also would require a tremendous amount of re-writing the parsing functions so that every function supports this (the actual data parsing is handled in several functions -- filter, search, calculate, group by, sort, etc. etc.
Use IndexedDB and store the state in javascript. Allocate a chunk of memory in WASM, then return its pointer to JavaScript. Then read database there and fill the pointer. Then process your data in C++. Feasible? Not sure, though this seems like the best solution if it can be implemented.
[Anything else?]
In the bounty then I was wondering three things:
If the above six analyses seem generally valid?
Are there other (perhaps better) approaches I'm missing?
Would anyone be able to show a very basic example of doing #6 -- seems like that would be the best solution if it's possible and works cross-browser.
For Chrome (only) you may use shared memory (shared buffer as memory). And raise a flag in memory when you want to halt. Not a big fan of this solution (is complex and is supported only in chrome). It also depends on how your query works, and if there are places where the lengthy query can check the flag.
Instead you should probably call the c++ function multiple times (e.g. for each query) and check if you should halt after each call (just send a message to the worker to halt).
What I mean by multiple time is make the query in stages (multiple function cals for a single query). It may not be applicable in your case.
Regardless, AFAIK there is no way to send a signal to a Webassembly execution (e.g. Linux kill). Therefore, you'll have to wait for the operation to finish in order to complete the cancellation.
I'm attaching a code snippet that may explain this idea.
worker.js:
... init webassembly
onmessage = function(q) {
// query received from main thread.
const result = ... call webassembly(q);
postMessage(result);
}
main.js:
const worker = new Worker("worker.js");
const cancel = false;
const processing = false;
worker.onmessage(function(r) {
// when worker has finished processing the query.
// r is the results of the processing.
processing = false;
if (cancel === true) {
// processing is done, but result is not required.
// instead of showing the results, update that the query was canceled.
cancel = false;
... update UI "cancled".
return;
}
... update UI "results r".
}
function onCancel() {
// Occurs when user clicks on the cancel button.
if (cancel) {
// sanity test - prevent this in UI.
throw "already cancelling";
}
cancel = true;
... update UI "canceling".
}
function onQuery(q) {
if (processing === true) {
// sanity test - prevent this in UI.
throw "already processing";
}
processing = true;
// Send the query to the worker.
// When the worker receives the message it will process the query via webassembly.
worker.postMessage(q);
}
An idea from user experience perspective:
You may create ~two workers. This will take twice the memory, but will allow you to "cancel" "immediately" once. (it will just mean that in the backend the 2nd worker will run the next query, and when the 1st finishes the cancellation, cancellation will again become immediate).
Shared Thread
Since the worker and the C++ function that it called share the same thread, the worker will also be blocked until the C++ loop is finished, and won't be able to handle any incoming messages. I think the a solid option would minimize the amount of time that the thread is blocked by instead initializing one iteration at a time from the main application.
It would look something like this.
main.js -> worker.js -> C++ function -> worker.js -> main.js
Breaking up the Loop
Below, C++ has a variable initialized at 0, which will be incremented at each loop iteration and stored in memory.
C++ function then performs one iteration of the loop, increments the variable to keep track of loop position, and immediately breaks.
int x;
x = 0; // initialized counter at 0
std::vector<JSONObject> data
for (size_t i = x; i < data.size(); i++)
{
process_data(data[i]);
x++ // increment counter
break; // stop function until told to iterate again starting at x
}
Then you should be able to post a message to the web worker, which then sends a message to main.js that the thread is no longer blocked.
Canceling the Operation
From this point, main.js knows that the web worker thread is no longer blocked, and can decide whether or not to tell the web worker to execute the C++ function again (with the C++ variable keeping track of the loop increment in memory.)
let continueOperation = true
// here you can set to false at any time since the thread is not blocked here
worker.expensiveThreadBlockingFunction()
// results in one iteration of the loop being iterated until message is received below
worker.onmessage = function(e) {
if (continueOperation) {
worker.expensiveThreadBlockingFunction()
// execute worker function again, ultimately continuing the increment in C++
} {
return false
// or send message to worker to reset C++ counter to prepare for next execution
}
}
Continuing the Operation
Assuming all is well, and the user has not cancelled the operation, the loop should continue until finished. Keep in mind you should also send a distinct message for whether the loop has completed, or needs to continue, so you don't keep blocking the worker thread.
I want to run a simple function asynchronously in Electron, so it doesn't block my render thread. So, something (very roughly) like this (within render.js):
var max = 42; // Somehow needs to be passed to the function
function foo() {
for (var i = 0; i < max; i++) {
// Do something...
// ... and tell the render thread about it.
}
}
foo(); // Should run asynchronously
There are two requirements:
I must pass parameters to the function (here: max). These could not only be integers, but also objects. The function must not run before it recieves these parameters.
While running, there must be a communication channel to the render thread. For example, to periodically report progress from within the for loop all the way to the UI, or to abort the function when an event in the render thread fires.
Here is a more specific minimal working (or rather, not working) example. The purpose is to send serial commands to a physical apparatus, on which a probe should be moved to all locations in a specified grid. So I need two loops (one for x, one for y). The loop body will contain functions that block until the motors have moved, than a mesurement from that location must be communicated back to the UI. Also, before the loops start running, the specifications about the grid must be known (hence my requirement about passing parameters.)
var Parameters = {x_length: 50, y_length: 50};
//These functions interact with a serial device and block until it is done with its task
function moveTo(x, y) {/*...*/};
function measure() {/*...*/};
//This function should eventually be executed asynchronously
function scan() {
for (var x = 0; x < Parameters.x_length; x++) {
for (var y = 0; y < Parameters.y_length; y++) {
moveTo(x, y);
var result = measure();
// Here, we need to tell the render thread about results. I used
// postMessage (WebWorker syntax) as a placeholder.
postMessage({
position: {x: x, y: y},
data: result
});
}
}
}
// This is what will be actually called, e. g. when the user clicks a
// button.
function scan_async() {
//Again, I used WebWorker (or rather, TinyWorker) syntax as a placeholder.
var scan_worker = new Worker(scan);
scan_worker.onmessage = function (msg) {
console.log(msg.data);
};
}
After hours of very frustrating googeling, I found a lot of approaches, but non of them seems like the "definitive" way to do it, or doesn't meet the above points, or seems way to complicated for what I want to achieve.
What I found so far:
Actually using WebWorkers (or rather something TinyWorkers, https://github.com/avoidwork/tiny-worker), like in the code above. Here, there seems to be no elegant way to pass the starting parameters before the worker starts running.
Create a new, hidden BrowserWindow, as mentioned here. However, I can't find anything about this method elsewhere.
So, is there a "correct", more straight-forward way to achieve my goal?
Actually using WebWorkers (or rather something TinyWorkers)
Yes. This is how you move code to a different thread.
Here, there seems to be no elegant way to pass the starting parameters before the worker starts running.
Create the worker. Don't have it do anything except listen for postMessage events.
Send a message to it, with postMessage with the starting parameters
An event listener in the worker gets the message and starts running the job
When it is done, postMessage from the work to the main application
An event listener in the main application gets the message with the results
I have a method that runs every 2 seconds to capture a video stream to canvas and write it to file:
function capture(streamName, callback) {
var buffer,
dataURL,
dataSplit,
_ctx;
_ctx = _canvas[streamName].getContext('2d');
_ctx.drawImage(_video[streamName], 0, 0);
dataURL = _canvas[streamName].toDataURL('image/png');
dataSplit = dataURL.split(",")[1];
buffer = new Buffer(dataSplit, 'base64');
fs.writeFileSync(directory + streamName + '.png', buffer);
}
setInterval(function() {
// Called from here
captureState.capture(activeScreens[currentScreenIndex]);
gameState.pollForState(processId, activeScreens[currentScreenIndex], function() {
// do things...
});
}, 2000);
Assuming _video[streamName] exists as a running <video> and _canvas[streamName] exists as a <canvas>. The method works, it just causes a memory leak.
The issue:
Garbage collection can't keep up with the amount of memory the method uses, memory leak ensues.
I have narrowed it down to this line:
buffer = new Buffer(dataSplit, 'base64');
If I comment that out, there is some accumulation of memory (~100MB) but it drops back down every 30s or so.
What I've tried:
Some posts suggested buffer = null; to remove the reference and mark for garbage collection, but that hasn't changed anything.
Any suggestions?
Timeline:
https://i.imgur.com/wH7yFjI.png
https://i.imgur.com/ozFwuxY.png
Allocation Profile:
https://www.dropbox.com/s/zfezp46um6kin7g/Heap-20160929T140250.heaptimeline?dl=0
Just to quantify. After about 30 minutes of run time it sits at 2 GB memory used. This is an Electron (chromium / desktop) app.
SOLVED
Pre-allocating the buffer is what fixed it. This means that in addition to scoping buffer outside of the function, you need to reuse the created buffer with buffer.write. In order to keep proper headers, make sure that you use the encoded parameter of buffer.write.
Matt, I am not sure what was not working with the pre-allocated buffers, so I've posted an algorithm of how such pre-allocated buffers could be used. The key thing here is that buffers are allocated only once for that reason there should not be any memory leak.
var buffers = [];
var bsize = 10000;
// allocate buffer pool
for(var i = 0; i < 10; i++ ){
buffers.push({free:true, buf: new Buffer(bsize)});
}
// sample method that picks one of the buffers into use
function useOneBuffer(data){
// find a free buffer
var theBuf;
var i = 10;
while((typeof theBuf==='undefined')&& i < 10){
if(buffers[i].free){
theBuf = buffers[i];
}
i++;
}
theBuf.free = false;
// start doing whatever you need with the buffer, write data in needed format to it first
// BUT do not allocate
// also, you may want to clear-write the existing data int he buffer, just in case before reuse or after the use.
if(typeof theBuf==='undefined'){
// return or throw... no free buffers left for now
return;
}
theBuf.buf.write(data);
// .... continue using
// dont forget to pass the reference to the buffers member along because
// when you are done, toy have to mark it as free, so that it could be used again
// theBuf.free = true;
}
Did you try something like this? Where did it fail?
There is no leak of buffer object in your code.
Any Buffer objects that you no longer retain a reference to in your code will be immediately available for garbage collection.
the problem caused by callback and how you use it out of capture function.
notice that GC can not cleans the buffer or any other variable as long as callback is running.
I have narrowed it down to this line:
buffer = new Buffer(dataSplit, 'base64');
Short solution is not to use Buffer, as it is not necessary to write file to filesystem, where a file reference exists at base64 portion of data URI. setInterval does not appear to be cleared. You can define a reference for setInterval, then call clearInterval() at <video> ended event.
You can perform function without declaring any variables. Remove data, MIME type, and base64 portions of data URI returned by HTMLCanvasElement.prototype.toDataURL() as described at NodeJS: Saving a base64-encoded image to disk , this Answer at NodeJS write base64 image-file
function capture(streamName, callback) {
_canvas[streamName].getContext("2d")
.drawImage(_video[streamName], 0, 0);
fs.writeFileSync(directory + streamName + ".png"
, _canvas[streamName].toDataURL("image/png").split(",")[1], "base64");
}
var interval = setInterval(function() {
// Called from here
captureState.capture(activeScreens[currentScreenIndex]);
gameState.pollForState(processId, activeScreens[currentScreenIndex]
, function() {
// do things...
});
}, 2000);
video[/* streamName */].addEventListener("ended", function(e) {
clearInterval(interval);
});
I was having a similar issue recently with a software app that uses ~500MB of data in arrayBuffer form. I thought I had a memory leak, but it turns out Chrome was trying to do optimizations on a set of large-ish ArrayBuffer's and corresponding operations (each buffer ~60mb in size and some slightly larger objects). The CPU usage appeared to never allow for GC to run, or at least that's how it appeared. I had to do two things to resolve my issues. I Have not read any specific spec for when the GC gets scheduled to prove or disprove that. What I had to do:
I had to break the reference to the data in my arrayBuffers and some other large objects.
I had to force Chrome to have downtime, which appeared to give it time to schedule and then run the GC.
After applying those two steps, things ran for me and were garbage collected. Unfortunately, when applying those two things independently from each other, my app kept on crashing (exploding into GB of memory used before doing so). The following would be my thoughts on what I'd try on your code.
The problem with the garbage collector is that you cannot force it to run. So you can have objects that are ready to be malloced, but for whatever reason the browser doesn't give the garbage collector opportunity. Another approach to the buffer = null would be instead to break the reference explicitly with the delete operator -- this is what I did, but in theory ... = null is equivalent. It's important to note that delete cannot be run on any variable created by the var operator. So something like the following would be my suggestion:
function capture(streamName, callback) {
this._ctx = _canvas[streamName].getContext('2d');
this._ctx.drawImage(_video[streamName], 0, 0);
this.dataURL = _canvas[streamName].toDataURL('image/png');
this.dataSplit = dataURL.split(",")[1];
this.buffer = new Buffer(dataSplit, 'base64');
fs.writeFileSync(directory + streamName + '.png', this.buffer);
delete this._ctx;//because the context with the image used still exists
delete this.dataURL;//because the data used in dataSplit exists here
delete this.dataSplit;//because the data used in buffer exists here
delete this.buffer;
//again ... = null likely would work as well, I used delete
}
Second, the small break. So it appears you've got some intensive processes going on and the system cannot keep up. It's not actually hitting the 2s save mark, because it needs more than 2 seconds per save. There is always a function on the queue for executing the captureState.capture(...) method and it never has time to garbage collect. Some helpful posts on the scheduler and differences between setInterval and setTimeout:
http://javascript.info/tutorial/settimeout-setinterval
http://ejohn.org/blog/how-javascript-timers-work/
If that is for sure the case, why not use setTimeout and simple check that roughly 2 seconds (or more) time has passed and execute. In doing that check always force your code to wait a set period of time between saves. Give the browser time to schedule/run GC -- something like what follows (100 ms setTimeout in the pollForState):
var MINIMUM_DELAY_BETWEEN_SAVES = 100;
var POLLING_DELAY = 100;
//get the time in ms
var ts = Date.now();
function interValCheck(){
//check if 2000 ms have passed
if(Date.now()-ts > 2000){
//reset the timestamp of the last time save was run
ts = Date.now();
// Called from here
captureState.capture(activeScreens[currentScreenIndex]);
//upon callback, force the system to take a break.
setTimeout(function(){
gameState.pollForState(processId, activeScreens[currentScreenIndex], function() {
// do things...
//and then schedule the interValCheck again, but give it some time
//to potentially garbage collect.
setTimeout(intervalCheck,MINIMUM_DELAY_BETWEEN_SAVES);
});
}
}else{
//reschedule check back in 1/10th of a second.
//or after whatever may be executing next.
setTimeout(intervalCheck,POLLING_DELAY);
}
}
This means that a capture will happen no more than once every 2 seconds, but will also in some sense trick the browser into having the time to GC and remove any data that was left.
Last thoughts, entertaining a more traditional definition of memory leak, The candidates for a memory leak based on what I see in your code would be activeScreens, _canvas or _video which appear to be objects of some sort? Might be worthwhile to explore those if the above doesn't resolve your issue (wouldn't be able to make any assessments based on what is currently shared).
Hope that helps!
In general, I would recommend using a local map of UUID / something that will allow you to control your memory when dealing with getImageData and other buffers.
The UUID can be a pre-defined identifier e.g: "current-image" and "prev-image" if comparing between slides
E.g
existingBuffers: Record<string, UInt8ClampedArray> = {}
existingBuffers[ptrUid] = ImageData.data (OR something equivalent)
then if you want to override ("current-image") you can (overkill here):
existingBuffers[ptrUid] = new UInt8ClampedArray();
delete existingBuffers[ptrUid]
In addition, you will always be able to check your buffers and make sure they are not going out of control.
Maybe it is a bit old-school, but I found it comfortable.
I've done an HTML form which has a lot of questions (coming from a database) in many different tabs. User then gives answers in those questions. Each time a user changes a tab my Javascript creates a save. The problem is that I have to loop through all questions each time the tab is changed and it freezes the form for about 5 seconds every time.
I've been searching for an answer how I can run my save function in the background. Apparently there is no real way to run something in the background and many recommend using setTimeout(); For example this one How to get a group of js function running in background
But none of these examples does explain or take into consideration that even if I use something like setTimeout(saveFunction, 2000); it doesn't solve my problem. It only postpones it by 2 seconds in this case.
Is there a way to solve this problem?
You can use web workers. Some of the older answers here say that they're not widely supported (which I guess they weren't when those answers were written), but today they're supported by all major browsers.
To run a web worker, you need to create an instance of the built-in Worker class. The constructor takes one argument which is the URI of the javascript file containing the code you want to run in the background. For example:
let worker = new Worker("/path/to/script.js");
Web workers are subject to the same origin policy so if you pass a path like this the target script must be on the same domain as the page calling it.
If you don't want to create an new Javascript file just for this, you can also use a data URI:
let worker = new Worker(
`data:text/javascript,
//Enter Javascript code here
`
);
Because of the same origin policy, you can't send an AJAX request from a data URI, so if you need to send an AJAX request in the web worker, you must use a separate Javascript file.
The code that you specify (either in a separate file or in a data URI) will be run as soon as you call the Worker constructor.
Unfortunately, web workers don't have access to neither outside Javascript variables, functions or classes, nor the DOM, but you can get around this by using the postMessage method and the onmessage event. In the outside code, these are members of the worker object (worker in the example above), and inside the worker, these are members of the global context (so they can be called either by using this or just like that with nothing in front).
postMessage and onmessage work both ways, so when worker.postMessage is called in the outside code, onmessage is fired in the worker, and when postMessage is called in the worker, worker.onmessage is fired in the outside code.
postMessage takes one argument, which is the variable you want to pass (but you can pass several variables by passing an array). Unfortunately, functions and DOM elements can't be passed, and when you try to pass an object, only its attributes will be passed, not its methods.
onmessage takes one argument, which is a MessageEvent object. The MessageEvent object has a data attribute, which contains the data sent using the first argument of postMessage.
Here is an example using web workers. In this example, we have a function, functionThatTakesLongTime, which takes one argument and returns a value depending on that argument, and we want to use web workers in order to find functionThatTakesLongTime(foo) without freezing the UI, where foo is some variable in the outside code.
let worker = new Worker(
`data:text/javascript,
function functionThatTakesLongTime(someArgument){
//There are obviously faster ways to do this, I made this function slow on purpose just for the example.
for(let i = 0; i < 1000000000; i++){
someArgument++;
}
return someArgument;
}
onmessage = function(event){ //This will be called when worker.postMessage is called in the outside code.
let foo = event.data; //Get the argument that was passed from the outside code, in this case foo.
let result = functionThatTakesLongTime(foo); //Find the result. This will take long time but it doesn't matter since it's called in the worker.
postMessage(result); //Send the result to the outside code.
};
`
);
worker.onmessage = function(event){ //Get the result from the worker. This code will be called when postMessage is called in the worker.
alert("The result is " + event.data);
}
worker.postMessage(foo); //Send foo to the worker (here foo is just some variable that was defined somewhere previously).
Apparently there is no real way to run something on background...
There is on most modern browsers (but not IE9 and earlier): Web Workers.
But I think you're trying to solve the problem at the wrong level: 1. It should be possible to loop through all of your controls in a lot less than five seconds, and 2. It shouldn't be necessary to loop through all controls when only one of them has changed.
I suggest looking to those problems before trying to offload that processing to the background.
For instance, you could have an object that contains the current value of each item, and then have the UI for each item update that object when the value changes. Then you'd have all the values in that object, without having to loop through all the controls again.
You could take a look at HTML5 web workers, they're not all that widely supported though.
This works in background:
setInterval(function(){ d=new Date();console.log(d.getTime()); }, 500);
If you can't use web workers because you need to access the DOM, you can also use async functions. The idea is to create an async refreshUI function that refreshes the UI, and then call that function regularly in your function that takes long time.
The refreshUI function would look like this:
async function refreshUI(){
await new Promise(r => setTimeout(r, 0));
}
In general, if you put await new Promise(r => setTimeout(r, ms)); in an async function, it will run all the code before that line, then wait for ms milliseconds without freezing the UI, then continues running the code after that line. See this answer for more information.
The refreshUI function above does the same thing except that it waits zero milliseconds without freezing the UI before continuing, which in practice means that it refreshes the UI and then continues.
If you use this function to refresh the UI often enough, the user won't notice the UI freezing.
Refreshing the UI takes time though (not enough time for you to notice if you just do it once, but enough time for you to notice if you do it at every iteration of a long for loop). So if you want the function to run as fast as possible while still not freezing the UI, you need to make sure not to refresh the UI too often. So you need to find a balance between refreshing the UI often enough for the UI not to freeze, but not so often that it makes your code significantly slower. In my use case I found that refreshing the UI every 20 milliseconds is a good balance.
You can rewrite the refreshUI function from above using performance.now() so that it only refreshes the UI once every 20 milliseconds (you can adjust that number in your own code if you want) no matter how often you call it:
let startTime = performance.now();
async function refreshUI(){
if(performance.now() > startTime + 20){ //You can change the 20 to how often you want to refresh the UI in milliseconds
startTime = performance.now();
await new Promise(r => setTimeout(r, 0));
}
}
If you do this, you don't need to worry about calling refreshUI to often (but you still need to make sure to call it often enough).
Since refreshUI is an async function, you need to call it using await refreshUI() and the function calling it must also be an async function.
Here is an example that does the same thing as the example at the end of my other answer, but using this method instead:
let startTime = performance.now();
async function refreshUI(){
if(performance.now() > startTime + 20){ //You can change the 20 to how often you want to refresh the UI in milliseconds
startTime = performance.now();
await new Promise(r => setTimeout(r, 0));
}
}
async function functionThatTakesLongTime(someArgument){
//There are obviously faster ways to do this, I made this function slow on purpose just for the example.
for(let i = 0; i < 1000000000; i++){
someArgument++;
await refreshUI(); //Refresh the UI if needed
}
return someArgument;
}
alert("The result is " + await functionThatTakesLongTime(3));
This library helped me out a lot for a very similar problem that you describe: https://github.com/kmalakoff/background
It basically a sequential background queue based on the WorkerQueue library.
Just create a hidden button. pass the function to its onclick event.
Whenever you want to call that function (in background), call the button's click event.
<html>
<body>
<button id="bgfoo" style="display:none;"></button>
<script>
function bgfoo()
{
var params = JSON.parse(event.target.innerHTML);
}
var params = {"params":"in JSON format"};
$("#bgfoo").html(JSON.stringify(params));
$("#bgfoo").click(bgfoo);
$("#bgfoo").click(bgfoo);
$("#bgfoo").click(bgfoo);
</script>
</body>
</html>
If I have a var on my main page, and have a worker thread trying to set this var, is there a way the page can access it? Assuming everything is synchronized?
var routeWorker = new Worker('getroute.js');
var checkPatrolRouteFoundTimer;
var rw_resultRoute;
var routeFound = false;
routeWorker.onmessage = function(e) {
rw_resultRoute = e.data.route;
routeFound = true;
}
function checkPatrolReady() {
if(!routeFound)
checkPatrolRouteFoundTimer = setTimeout("checkPatrolReady()", 1000);
}
function ForcePatrol(index) {
routeWorker.postMessage(index);
checkPatrolReady();
...
//do work on route
...
}
in this case, the var I'm talking about is rw_resultRoute, and I can see it get set correctly when debugging. But the only thing is that it's set in the worker thread, not in the page thread.
I flow through the ForcePatrol() method the way i'm expecting to, and it looks like the rw_resultRoute is being set, since routeFound evaluates to true after the worker finishes.
Technically, it doesn't make sense, since routeFound can be set by the worker and read by the page thread, but rw_resultRoute can only be accessed by the worker.
I truly hope this is possible, otherwise I don't see a purpose for worker threads other than showing alert() messages and updating page HTML.
I truly hope this is possible, otherwise I don't see a purpose for worker threads other than showing alert() messages and updating page HTML.
It is meant to handle processing that would normally lock up the browser. Great for crunching numbers for canvas and running hashing.
in this case, the var I'm talking about is rw_resultRoute, and I can see it get set correctly when debugging. But the only thing is that it's set in the worker thread, not in the page thread.
The worker is separate from the page that spawns it. Only way to pass data is through messaging. You need to send the data with postMessage and have the onMessage handle the result. If you are handling different things, set up a switch statement to handle the different message types.
I solved the problem. There was some synchronization I wasn't doing correctly. I was using the setTimeout in the wrong way.
var routeWorker = new Worker('getroute.js');
var checkPatrolRouteFoundTimer;
var rw_resultRoute;
var routeFound = false;
routeWorker.onmessage = function(e) {
rw_resultRoute = e.data.route;
routeFound = true;
}
function checkPatrolReady() {
if(routeFound) {
...
//do work on route
...
clearInterval(checkPatrolRouteFoundTimer);
} else {
// do any maint here?
}
}
function ForcePatrol(index) {
routeWorker.postMessage(index);
checkPatrolRouteFoundTimer = setInterval("checkPatrolReady()", 1000);
}
Any call to setTimeout/setInterval will flow through, and in the first example i was using setTimeout instead of setInterval.
In the new way, calling ForcePatrol will setup the timer, and checkPatrolReady() will evaluate the flag, doing the work and clearing the timer if it is true.
So there is indeed nothing fancy in getting the results from web workers, but I was essentially creating a race condition with the worker results.