We Keep Coding

How to run expensive code parallel in the same file

I'm trying to run a piece of JavaScript code asynchronously to the main thread. I don't necessarily need the code to actually run on a different thread (so performance does not need to be better that sequential execution), but I want the code to be executed in parallel to the main thread, meaning no freezing.
Additionally, all the code needed needs to be contained within a single function.
My example workload is as follows:
function work() {
for(let i=0; i<100000; i++)
console.log("Async");
}
Additionally, I may have some work on the main thread (which is allowed to freeze the side, just for testing):
function seqWork() {
for(let i=0; i<100000; i++)
console.log("Sequential");
}
The expected output should be something like this:
Sequential
Async
Sequential
Sequential
Async
Sequential
Async
Async
...
You get the point.
Disclaimer: I am absolutely unexperienced in JavaScript and in working with async and await.
What I've tried
I did some research, and found these 3 options:
1. async/await
Seems like the obvious choice. So I tried this:
let f= async function f() {
await work();
}
f();
seqWork();
Output:
Async (100000)
Sequential (100000)
I also tried:
let f = async function f() {
let g = () => new Promise((res,rej) => {
work();
res();
});
await g();
}
f();
seqWork();
Output:
Async (100000)
Sequential (100000)
So both methods did not work. They also both freeze the browser during the async output, so it seems that that has absolutely no effect(?) I may be doing something very wrong here, but I don't know what.
2. Promise.all
This seems to be praised as the solution for any expensive task, but only seems like a reasonably choice if you have many blocking tasks and you want to "combine" them into just one blocking task that is faster than executing them sequentially. There are certainly use cases for this, but for my task it is useless, because I only have one task to execute asynchronously, and the main "thread" should keep running during that task.
3. Worker
This seemed to me like the most promising option, but I have not got it working yet. The main problem is that you seem to need a second script. I cannot do that, but even in local testing with a second file Firefox is blocking the loading of that script.
This is what I've tried, and I have not found any other options in my research. I'm starting to think that something like this is straight up not possible in JS, but it seems like a quite simple task. Again, I don't need this to be actually executed in parallel, it would be enough if the event loop would alternate between calling a statement from the main thread and the async "thread". Coming from Java, they are also able to simulate multi threading on a single hardware thread.
Edit: Context
I have some java code that gets converted to JavaScript (I have no control over the conversion) using TeaVM. Java natively supports multithreading, and a lot of my code relies on that being possible. Now since JavaScript apparently does not really support real multithreading, TeaVM converts Thread in the most simplistic way to JS: Calling Thread.start() directly calls Thread.run() which makes it completely unusable. I want to create a better multithreading emulation here which can - pretty much - execute the thread code basically without modification. Now it is not ideal but inserting "yielding" statements into the java code would be possible.
TeaVM has a handy feature which allows you to write native Java methods annotated with matching JS code that will be converted directly into that code. Problem is, you cannot set the method body so you can't make it an async method.
One thing I'm now trying to do is implement a JS native "yield" / "pause" (to not use the keyword in JS) function which I can call to allow for other code to run right from the java method. The method basically has to briefly block the execution of the calling code and instead invoke execution of other queued tasks. I'm not sure whether that is possible with the main code not being in an async function. (I cannot alter the generated JS code)
The only other way I can think of to work around this would be to let the JS method call all the blocking code, refering back to the Java code. The main problem though is, that this means splitting up the method body of the java method into many small chunks as Java does not support something like yield return from C#. This basically means a complete rework of every single parallel executed piece of code, which I would desperately try to avoid. Also, you could not "yield" from within a called method, making it way less modular. At that point I may as well just call the method chunks from within Java directly from an internal event loop.

Since JavaScript is single threaded the choice is between
running some code asynchronously in the main thread, or
running the same code in a worker thread (i.e. one that is not the main thread.
Coopererative Multitasking
If you want to run heavy code in the main thread without undue blocking it would need to be written to multitask cooperatively. This requires long running synchronous tasks to periodically yield control to the task manager to allow other tasks to run. In terms of JavaScript you could achieve this by running a task in an asynchronous function that periodically waits for a system timer of short duration. This has potential because await saves the current execution context and returns control to the task manager while an asynchronous task is performed. A timer call ensures that the task manager can actually loop and do other things before returning control to the asynchronous task that started the timer.
Awaiting a promise that is already fulfilled would only interleave execution of jobs in the microtask queue without returning to the event loop proper and is not a suitable for this purpose.
Calling code pattern:
doWork()
.then( data => console.log("work done"));
Work code:
async function doWork() {
for( i = 1; i < 10000; ++i) {
// do stuff
if( i%1000 == 0) {
// let other things happen:
await new Promise( resolve=>setTimeout(resolve, 4))
}
}
}
Note this draws on historical practice and might suit the purpose of getting prototype code working quickly. I wouldn't think it particularly suitability for a commercial production environment.
Worker Threads
A localhost server can be used to serve worker code from a URL so development can proceed. A common method is to use a node/express server listening on a port of the loopback address known as localhost.
You will need to install node and install express using NPM (which is installed with node). It is not my intention to go into the node/express eco-system - there is abundant material about it on the web.
If you are still looking for a minimalist static file server to serve files from the current working directory, here's one I wrote earlier. Again there are any number of similar examples available on the net.
"use strict";
/*
* express-cwd.js
* node/express server for files in current working directory
* Terminal or shortcut/desktop launcher command: node express-cwd
*/
const express = require('express');
const path = require('path');
const process = require("process");
const app = express();
app.get( '/*', express.static( process.cwd())); // serve files from working directory
const ip='::1'; // local host
const port=8088; // port 8088
const server = app.listen(port, ip, function () {
console.log( path.parse(module.filename).base + ' listening at http://localhost:%s', port);
})
Promise Delays
The inlined promise delay shown in "work code" above can be written as a function, not called yield which is a reserved word. For example
const timeOut = msec => new Promise( r=>setTimeout(r, msec));
An example of executing blocking code in sections:
"use strict";
// update page every 500 msec
const counter = document.getElementById("count");
setInterval( ()=> counter.textContent = +counter.textContent + 1, 500);
function block_500ms() {
let start = Date.now();
let end = start + 500;
for( ;Date.now() < end; );
}
// synchronously block for 4 seconds
document.getElementById("block")
.addEventListener("click", ()=> {
for( var i = 8; i--; ) {
block_500ms();
}
console.log( "block() done");
});
// block for 500 msec 8 times, with timeout every 500 ms
document.getElementById("block8")
.addEventListener("click", async ()=> {
for( var i = 8; i--; ) {
block_500ms();
await new Promise( resolve=>setTimeout(resolve, 5))
}
console.log("block8() done");
});
const timeOut = msec => new Promise( r=>setTimeout(r, msec));
document.getElementById("blockDelay")
.addEventListener("click", async ()=> {
for( var i = 8; i--; ) {
block_500ms();
await timeOut();
}
console.log("blockDelay(1) done");
});
Up Counter: <span id="count">0</span>
<p>
<button id="block" type="button" >Block counter for 4 seconds</button> - <strong> with no breaks</strong>
<p>
<button id="block8" type="button" >Block for 4 seconds </button> - <strong> with short breaks every 500 ms (inline)</strong>
<p>
<button id="blockDelay" type="button" >Block for 4 seconds </button> - <strong> with short breaks every 500 ms (using promise function) </strong>
Some jerkiness may be noticeable with interleaved sections of blocking code but the total freeze is avoided. Timeout values are determined by experiment - the shorter the value that works in an acceptable manner the better.
Caution
Program design must ensure that variables holding input data, intermediate results and accumulated output data are not corrupted by main thread code that may or may not be executed part way through the course of heavy code execution.

Call a function asynchronous in a synchronous code-base

I have a project consisting out of thousands of lines of Javascript code. It contains a lot of complex calculations which are divided into different functions. One function calls three or four other functions, of which one calls another one of those four functions,... Just to say, it has a complex structure.
Some functions also need to do calculations on the DOM. E.g, One functions adds an element to the DOM, and another one needs to do a calculation using the coordinates of that element in the DOM.
However, when the user visits the page, all of this is executed by one function called - in this example - bigFunction.
The problem is that bigFunction starts a long series of complex calculations and function calls resulting in stalling the page loading, and making the page irresponsive until bigFunction is finished.
I already tried simple solutions like defering my .js-file, using $(document).ready(function() { bigFunction(); }), $(window).on('load', function() { bigFunction(); }). None of that changed anything. I tried converting my code to async code but that was crazy work.
I am actually just wondering if there is a way to call bigFunction asynchronous somehow but without converting all of my code to async code and making all my functions async, is that possible?
Some test-code:
// Executing big function
bigFunction();
// "HTML"
console.log("[+] HTML Loaded!!"); // -> Must load async with bigFunction running
// This is a function I execute on a page and which stops the page from
// loading until it's finished. How do I async this?
async function bigFunction() {
console.log("[+] bigFunction started");
if (functionToReturnTrueOrFalseAfterALongCalculation()) {
console.log("WOW!");
}
console.log("[+] bigFunction finished")
}
// Ignore logic in this function... It just returns true of false randomly after 3,5 secs
function functionToReturnTrueOrFalseAfterALongCalculation() {
console.log("[+] functionToReturnTrueOrFalseAfterALongCalculation started");
let to_return = false;
// Give back random value
let random_number = Math.floor(Math.random() * 100) + 1;
if (random_number < 50) {
to_return = true;
}
sleep(3500);
// Just to show that functions are nested deep in each other
doRandomStuff();
console.log("[+] functionToReturnTrueOrFalseAfterALongCalculation ended");
return to_return;
}
// Just another slow function
function doRandomStuff() {
console.log("[+] doRandomStuff started");
sleep(1000);
console.log("[+] doRandomStuff ended");
}
// Ignore
function sleep(milliseconds) {
var start = new Date().getTime();
for (var i = 0; i < 1e7; i++) {
if ((new Date().getTime() - start) > milliseconds){
break;
}
}
}
HTML should load faster...

I am actually just wondering if there is a way to call bigFunction asynchronous somehow but without converting all of my code to async code and making all my functions async, is that possible?
Fundamentally, not really. In order for the page to remain responsive, your JavaScript code must yield control back to the browser regularly. If you have some very expensive JS that blocks for 0.5 seconds, the page will appear to the user to be unresponsive for those 0.5 seconds. (0.5 seconds of blocking is not good. 0.05 seconds of blocking every 0.1 seconds could be better, for example.)
To have the site remain responsive, all of your functions that invoke sleep (the expensive calculations) need to be refactored to stagger up their jobs. Pure JavaScript jobs can be offloaded to a service worker, thereby freeing up resources to keep the active tab responsive, but jobs that require DOM manipulation (some of which you have) can't be offloaded to a service worker.
Without seeing what exactly the expensive functions are doing, more details are difficult, but for what you want, in the end you'll probably need to end up with code that looks something like
async function bigFunction() {
console.log("[+] bigFunction started");
// the below function does not block for long periods of time,
// but only for short periods of time, staggered up
const result = await functionToReturnTrueOrFalseAfterALongCalculation();
if (result) {
console.log("WOW!");
}
console.log("[+] bigFunction finished")
}
The code that comes after the execution of bigFunction would absolutely have to be able to handle the asynchronous execution flow - if you had
doSomething();
bigFunction();
finishDoingStuff();
you would have to refactor to something like
doSomething();
await bigFunction();
finishDoingStuff();
or to
doSomething();
bigFunction(finishDoingStuff);
where bigFunction calls finishDoingStuff once its expensive tasks are complete.

NodeJs/expressjs : Run lengthy code in a callback [duplicate]

This question already has answers here:
Long-running computations in node.js
(3 answers)
Closed 8 years ago.
Callbacks are asynchronous , So does that mean that if I run a lengthy computation in a callback it wont affect my main thread ?
For example:
function compute(req,res){ // this is called in an expressjs route.
db.collection.find({'key':aString}).toArray(function(err, items) {
for(var i=0;i<items.length;i++){ // items length may be in thousands.
// Heavy/lengthy computation here, Which may take 5 seconds.
}
res.send("Done");
});
}
So, the call to database is ascnchronous. Does that mean the for loop inside the callback will NOT block the main thread ?
And if it is blocking, How may I perform such things in an async way?

For the most part, node.js runs in a single thread. However, node.js allows you to make calls that execute low-level operations (file reads, network requests, etc.) which are handled by separate threads. As such, your database call most likely happens on a separate thread. But, when the database call returns, we return back to the main thread and your code will run in the main thread (blocking it).
The way to get around this is to spin up a new thread. You can use cluster to do this. See:
http://nodejs.org/api/cluster.html
Your main program will make the database call
When the database call finishes, it will call fork() and spin up a new thread that runs your-calculations.js and sends an event to it with any input data
your-calculations.js will listen for an event and do the necessary processing when it handles the event
your-calculations.js will then send an event back to the main thread when it has finished processing (it can send any output data back)
If the main thread needs the output data, it can listen for the event that your-calculations.js emits

If you can't do, or don't want to use a thread, you can split up the long computation with setImmediates. e.g. (writing quickly on my tablet so may be sloppy)
function compute(startIndex, max, array, partialResult, callback) {
var done = false;
var err = null;
var stop = startIndex+100; // or some reasonable amount of calcs...
if (stop >= max) {
stop = max;
done = true;
}
// do calc from startIndex to stop, using partialResult as input
if (done)
callback(err, result);
else
process.setImmediate ( go look this part up or I'll edit tomorrow)...
But the idea is you call youself again with start += 100.
}
In between every 100 calculations node will have time to process other requests, handle other callbacks, etc. Of course, if they trigger another huge calculation evedntually things will grind to a halt.

How to run javascript function in "background" / without freezing UI

I've done an HTML form which has a lot of questions (coming from a database) in many different tabs. User then gives answers in those questions. Each time a user changes a tab my Javascript creates a save. The problem is that I have to loop through all questions each time the tab is changed and it freezes the form for about 5 seconds every time.
I've been searching for an answer how I can run my save function in the background. Apparently there is no real way to run something in the background and many recommend using setTimeout(); For example this one How to get a group of js function running in background
But none of these examples does explain or take into consideration that even if I use something like setTimeout(saveFunction, 2000); it doesn't solve my problem. It only postpones it by 2 seconds in this case.
Is there a way to solve this problem?

You can use web workers. Some of the older answers here say that they're not widely supported (which I guess they weren't when those answers were written), but today they're supported by all major browsers.
To run a web worker, you need to create an instance of the built-in Worker class. The constructor takes one argument which is the URI of the javascript file containing the code you want to run in the background. For example:
let worker = new Worker("/path/to/script.js");
Web workers are subject to the same origin policy so if you pass a path like this the target script must be on the same domain as the page calling it.
If you don't want to create an new Javascript file just for this, you can also use a data URI:
let worker = new Worker(
`data:text/javascript,
//Enter Javascript code here
`
);
Because of the same origin policy, you can't send an AJAX request from a data URI, so if you need to send an AJAX request in the web worker, you must use a separate Javascript file.
The code that you specify (either in a separate file or in a data URI) will be run as soon as you call the Worker constructor.
Unfortunately, web workers don't have access to neither outside Javascript variables, functions or classes, nor the DOM, but you can get around this by using the postMessage method and the onmessage event. In the outside code, these are members of the worker object (worker in the example above), and inside the worker, these are members of the global context (so they can be called either by using this or just like that with nothing in front).
postMessage and onmessage work both ways, so when worker.postMessage is called in the outside code, onmessage is fired in the worker, and when postMessage is called in the worker, worker.onmessage is fired in the outside code.
postMessage takes one argument, which is the variable you want to pass (but you can pass several variables by passing an array). Unfortunately, functions and DOM elements can't be passed, and when you try to pass an object, only its attributes will be passed, not its methods.
onmessage takes one argument, which is a MessageEvent object. The MessageEvent object has a data attribute, which contains the data sent using the first argument of postMessage.
Here is an example using web workers. In this example, we have a function, functionThatTakesLongTime, which takes one argument and returns a value depending on that argument, and we want to use web workers in order to find functionThatTakesLongTime(foo) without freezing the UI, where foo is some variable in the outside code.
let worker = new Worker(
`data:text/javascript,
function functionThatTakesLongTime(someArgument){
//There are obviously faster ways to do this, I made this function slow on purpose just for the example.
for(let i = 0; i < 1000000000; i++){
someArgument++;
}
return someArgument;
}
onmessage = function(event){ //This will be called when worker.postMessage is called in the outside code.
let foo = event.data; //Get the argument that was passed from the outside code, in this case foo.
let result = functionThatTakesLongTime(foo); //Find the result. This will take long time but it doesn't matter since it's called in the worker.
postMessage(result); //Send the result to the outside code.
};
`
);
worker.onmessage = function(event){ //Get the result from the worker. This code will be called when postMessage is called in the worker.
alert("The result is " + event.data);
}
worker.postMessage(foo); //Send foo to the worker (here foo is just some variable that was defined somewhere previously).

Apparently there is no real way to run something on background...
There is on most modern browsers (but not IE9 and earlier): Web Workers.
But I think you're trying to solve the problem at the wrong level: 1. It should be possible to loop through all of your controls in a lot less than five seconds, and 2. It shouldn't be necessary to loop through all controls when only one of them has changed.
I suggest looking to those problems before trying to offload that processing to the background.
For instance, you could have an object that contains the current value of each item, and then have the UI for each item update that object when the value changes. Then you'd have all the values in that object, without having to loop through all the controls again.

You could take a look at HTML5 web workers, they're not all that widely supported though.

This works in background:
setInterval(function(){ d=new Date();console.log(d.getTime()); }, 500);

If you can't use web workers because you need to access the DOM, you can also use async functions. The idea is to create an async refreshUI function that refreshes the UI, and then call that function regularly in your function that takes long time.
The refreshUI function would look like this:
async function refreshUI(){
await new Promise(r => setTimeout(r, 0));
}
In general, if you put await new Promise(r => setTimeout(r, ms)); in an async function, it will run all the code before that line, then wait for ms milliseconds without freezing the UI, then continues running the code after that line. See this answer for more information.
The refreshUI function above does the same thing except that it waits zero milliseconds without freezing the UI before continuing, which in practice means that it refreshes the UI and then continues.
If you use this function to refresh the UI often enough, the user won't notice the UI freezing.
Refreshing the UI takes time though (not enough time for you to notice if you just do it once, but enough time for you to notice if you do it at every iteration of a long for loop). So if you want the function to run as fast as possible while still not freezing the UI, you need to make sure not to refresh the UI too often. So you need to find a balance between refreshing the UI often enough for the UI not to freeze, but not so often that it makes your code significantly slower. In my use case I found that refreshing the UI every 20 milliseconds is a good balance.
You can rewrite the refreshUI function from above using performance.now() so that it only refreshes the UI once every 20 milliseconds (you can adjust that number in your own code if you want) no matter how often you call it:
let startTime = performance.now();
async function refreshUI(){
if(performance.now() > startTime + 20){ //You can change the 20 to how often you want to refresh the UI in milliseconds
startTime = performance.now();
await new Promise(r => setTimeout(r, 0));
}
}
If you do this, you don't need to worry about calling refreshUI to often (but you still need to make sure to call it often enough).
Since refreshUI is an async function, you need to call it using await refreshUI() and the function calling it must also be an async function.
Here is an example that does the same thing as the example at the end of my other answer, but using this method instead:
let startTime = performance.now();
async function refreshUI(){
if(performance.now() > startTime + 20){ //You can change the 20 to how often you want to refresh the UI in milliseconds
startTime = performance.now();
await new Promise(r => setTimeout(r, 0));
}
}
async function functionThatTakesLongTime(someArgument){
//There are obviously faster ways to do this, I made this function slow on purpose just for the example.
for(let i = 0; i < 1000000000; i++){
someArgument++;
await refreshUI(); //Refresh the UI if needed
}
return someArgument;
}
alert("The result is " + await functionThatTakesLongTime(3));

This library helped me out a lot for a very similar problem that you describe: https://github.com/kmalakoff/background
It basically a sequential background queue based on the WorkerQueue library.

Just create a hidden button. pass the function to its onclick event.
Whenever you want to call that function (in background), call the button's click event.
<html>
<body>
<button id="bgfoo" style="display:none;"></button>
<script>
function bgfoo()
{
var params = JSON.parse(event.target.innerHTML);
}
var params = {"params":"in JSON format"};
$("#bgfoo").html(JSON.stringify(params));
$("#bgfoo").click(bgfoo);
$("#bgfoo").click(bgfoo);
$("#bgfoo").click(bgfoo);
</script>
</body>
</html>

How would I design a client-side Queue system?

OVERVIEW
I'm working on a project and I've come across a bit of a problem in that things aren't happening in the order I want them to happen. So I have been thinking about designing some kind of Queue that I can use to organize function calls and other miscellaneous JavaScript/jQuery instructions used during start-up, i.e., while the page is loading. What I'm looking for doesn't necessarily need to be a Queue data structure but some system that will ensure that things execute in the order I specify and only when the previous task has been completed can the new task begin.
I've briefly looked at the jQuery Queue and the AjaxQueue but I really have no idea how they work yet so I'm not sure if that is the approach I want to take... but I'll keep reading more about these tools.
SPECIFICS
Currently, I have set things up so that some work happens inside $(document).ready(function() {...}); and other work happens inside $(window).load(function() {...});. For example,
<head>
<script type="text/javascript">
// I want this to happen 1st
$().LoadJavaScript();
// ... do some basic configuration for the stuff that needs to happen later...
// I want this to happen 2nd
$(document).ready(function() {
// ... do some work that depends on the previous work do have been completed
var script = document.createElement("script");
// ... do some more work...
});
// I want this to happen 3rd
$(window).load(function() {
// ... do some work that depends on the previous work do have been completed
$().InitializeSymbols();
$().InitializeBlock();
// ... other work ... etc...
});
</script>
</head>
... and this is really tedious and ugly, not to mention bad design. So instead of dealing with that mess, I want to design a pretty versatile system so that I can, for example, enqueue $().LoadJavaScript();, then var script = document.createElement("script");, then $().InitializeSymbols();, then $().InitializeBlock();, etc... and then the Queue would execute the function calls and instructions such that after one instruction is finished executing, the other can start, until the Queue is empty instead of me calling dequeue repeatedly.
The reasoning behind this is that some work needs to happen, like configuration and initialization, before other work can begin because of the dependency on the configuration and initialization steps to have completed. If this doesn't sound like a good solution, please let me know :)
SOME BASIC WORK
I've written some code for a basic Queue, which can be found here, but I'm looking to expand its functionality so that I can store various types of "Objects", such as individual JavaScript/jQuery instructions and function calls, essentially pieces of code that I want to execute.
UPDATE
With the current Queue that I've implemented, it looks like I can store functions and execute them later, for example:
// a JS file...
$.fn.LoadJavaScript = function() {
$.getScript("js/Symbols/Symbol.js");
$.getScript("js/Structures/Structure.js");
};
// another JS file...
function init() { // symbols and structures };
// index.html
var theQueue = new Queue();
theQueue.enqueue($().LoadJavaScript);
theQueue.enqueue(init);
var LJS = theQueue.dequeue();
var INIT = theQueue.dequeue();
LJS();
INIT();
I also think I've figured out how to store individual instructions, such as $('#equation').html(""); or perhaps even if-else statements or loops, by wrapping them as such:
theQueue.enqueue(function() { $('#equation').html(""); // other instructions, etc... });
But this approach would require me to wait until the Queue is done with its work before I can continue doing my work. This seems like an incorrect design. Is there a more clever approach to this? Also, how can I know that a certain function has completed executing so that the Queue can know to move on? Is there some kind of return value that I can wait for or a callback function that I can specify to each task in the Queue?
WRAP-UP
Since I'm doing everything client-side and I can't have the Queue do its own thing independently (according to an answer below), is there a more clever solution than me just waiting for the Queue to finish its work?
Since this is more of a design question than a specific code question, I'm looking for suggestions on an approach to solving my problem, advice on how I should design this system, but I definitely welcome, and would love to see, code to back up the suggestions :) I also welcome any criticism regarding the Queue.js file I've linked to above and/or my description of my problem and the approach I'm planning to take to resolve it.
Thanks, Hristo

I would suggest using http://headjs.com/ It allows you to load js files in parallel, but execute them sequentially, essentially the same thing you want to do. It's pretty small, and you could always use it for inspiration.
I would also mention that handlers that rely on execution order are not good design. I am always able to place all my bootstrap code in the ready event handler. There are cases where you'd need to use the load handler if you need access to images, but it hasn't been very often for me.

Here is something that might work, is this what you're after?
var q = (function(){
var queue = [];
var enqueue = function(fnc){
if(typeof fnc === "function"){
queue.push(fnc);
}
};
var executeAll = function(){
var someVariable = "Inside the Queue";
while(queue.length>0){
queue.shift()();
}
};
return {
enqueue:enqueue,
executeAll:executeAll
};
}());
var someVariable = "Outside!"
q.enqueue(function(){alert("hi");});
q.enqueue(function(){alert(someVariable);});
q.enqueue(function(){alert("bye");});
alert("test");
q.executeAll();
the alert("test"); runs before anything you've put in the queue.

how do I store pieces of code in the Queue and have it execute later
Your current implementation already works for that. There are no declared types in JavaScript, so your queue can hold anything, including function objects:
queue.enqueue(myfunc);
var f = queue.dequeue();
f();
how can I have the Queue do its own thing independently
JavaScript is essentially single-threaded, meaning only one thing can execute at any instant of time. So the queue can't really operate "independently" of the rest of your code, if that is what you mean.
You basically have two choices:
Run all the queued functions, one after the other, in a single go -- this doesn't even require a queue since it is the same as simply putting the function calls directly in your code.
Use timed events: run one function at a time and once it completes, set a timeout to execute the next queued function after a certain interval. An example of this follows.
function run() {
var func = this.dequeue();
func();
var self = this;
setTimeout(function() { self.run(); }, 1000);
}
If func is an asynchronous request, you'll have to move setTimeout into the callback function.

**The main functions**
**From there we can define the main elements required:**
var q=[];//our queue container
var paused=false; // a boolean flag
function queue() {}
function dequeue() {}
function next() {}
function flush() {}
function clear() {}
**you may also want to 'pause' the queue. We will therefore use a boolean flag too.
Now let's see the implementation, this is going to be very straightforward:**
var q = [];
var paused = false;
function queue() {
for(var i=0;i< arguments.length;i++)
q.push(arguments[i]);
}
function dequeue() {
if(!empty()) q.pop();
}
function next() {
if(empty()) return; //check that we have something in the queue
paused=false; //if we call the next function, set to false the paused
q.shift()(); // the same as var func = q.shift(); func();
}
function flush () {
paused=false;
while(!empty()) next(); //call all stored elements
}
function empty() { //helper function
if(q.length==0) return true;
return false;
}
function clear() {
q=[];
}
**And here we have our basic queue system!
let's see how we can use it:**
queue(function() { alert(1)},function(){ alert(2)},function(){alert(3)});
next(); // alert 1
dequeue(); // the last function, alerting 3 is removed
flush(); // call everything, here alert 2
clear(); // the queue is already empty in that case but anyway...