We Keep Coding

Is it safe to use Array.sort() in synchronous way?

I am not sure whether the Array.sort(callback) is synchronous or asyncronous. But I have used the Array.sort(callback) to sort the date (updateddate which is stored in db as string type) in the following code snippet. Do i need to include await for Array.sort(callback) to ensure that the rest of the code is executed only when the array sorting is completed. Is it right way to use sorting above the synchronous code? Should I write the rest of the code inside the callback of data.sort?
modify_data(data,likesData){
data.sort(function(a,b){
return new Date(b.updateddate) - new Date(a.updateddate);
})
var nomination_group_id = _.groupBy(data,"submissionid")
var likes_group
var refined_arr = [];
var likesData = likesData
_.each(nomination_group_id,function(eachObj){
var mapObj = new Map()
mapObj.set('category',eachObj[0] ? eachObj[0].question : " ")
mapObj.set('submitter',eachObj[0] ? eachObj[0].email : " ")
refined_arr.push([ ...mapObj.values() ])
})
return refined_arr
}

I am not sure whether the Array.sort(callback) is synchronous or asyncronous
Its safe, and its synchronous.
Array.sort(sortFunction) argument sortFunction is synchronously applied on each element of the array. Also, Javascript is single-threaded.
const names = ['adelaine','ben', 'diana', 'carl']
const namesSortedAlphabetically = names.sort() // default is alphabetical sort
console.log('ALPHA SORT', namesSortedAlphabetically)
const namesSortedByLength = names.sort((a, b) => a.length > b.length) // custom sort
console.log('CUSTOM SORT', namesSortedByLength)
Do i need to include await for Array.sort(callback)
Nope, since Array.sort is synchronous and doesn't return a Promise.

It's synchronous. Callback itself doesn't mean owner function is asynchronous.
Callback is just for deciding how to sort. So you can put your next logic outside of callback safely.

It is safe & synchronous, because it is not a call back, it is just a comparator (compare function) see here
It is not calling this function at the end of doing something, it is using that function. Hope it calrifies

.sort() is a function from the JavaScript language, and it is determined to be synchronous by the ECMAScript specification.
(Functions of the node.js framework are ususally asynchronous by default, but .sort() is coming from JavaScript language itself, not from the node.js framework.)
It is very likely that you want to use it asynchronously, when it comes to any production level application. You can:
just wrap it around an asynchronous function scope, or
write your own asynchronous implementation, or
you can find several ready-to-use implementations for this issue, for example as an npm package.
You are right, you need to pay attention that the code which is counting on the sorting to be done needs to be executed after sorting. If you have chosen the first option in the 3 options before, you can:
put stuff inside one and only callback (likely minor things), or
build up a callback chain with .then()
use the await keyword as you have mentioned.
Please note that I didn't want to confuse you with so many solutions, but it's necessary to be aware of them. They will become second nature over time. If all of this feels a little confusing at this moment, I'd suggest you to start with the basic .sort() wrapped around an asynchronous function, and chained with .then() functions.
EDIT
As my peer mentioned above, .sort() itself does not accept a callback function, but a sorting function instead. That is correct. In order to use .sort() asynchronously, you have to wrap it around your own asynchronous function, as I've suggested.

Modify input is a bad idea and you don't need await to make sure data.sort is synchronous, just declare a new variable to hold the result of sort like:
var sorted_data = data.sort(function(a,b){
return new Date(b.updateddate) - new Date(a.updateddate);
})
var nomination_group_id = _.groupBy(sorted_data, "submissionid")

Should I always return promises in all functions in JavaScript? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I am thinking could it be a good approach to always return promises in functions in JavaScript?
Let's imagine the case where we have a function that validates a username. A master function simply utilises 2 other functions that perform different checks.
Please note, all functions names are just examples.
// Returns a boolean
function validateUsername (username) {
return validateUsernameFormat(username) &&
isUsernameReserved(username);
}
// Returns a boolean
function validateUsernameFormat (username) {
return typeOf(username) === 'string' &&
username.match(/^\[a-z0-9]{8,20}$/);
}
// Returns a boolean
function isUsernameNotReserved (username) {
return ['igor', 'kristina'].indexOf(username) === -1;
}
Now let's imagine we augment our validation in the future by calling API to check if a given username already exists in our database.
// Now returns a promise
function isUsernameNotReserved (username) {
return API.checkIfUserNameAlreadyExists(username);
}
This would mean we will also now have to change the master validateUsername function since it now also needs to return promise. This would also probably mean we will have to modify all functions that use validateUsername function.
But what If we had all function in promises from scratch?
Option A - All functions return promises
// Returns a promise
function validateUsername (username) {
return validateUsernameFormat(username)
.then(() => {
return isUsernameReserved(username);
});
}
// Returns a promise
function validateUsernameFormat (username) {
return (
typeOf(username) === 'string' && username.match(/^\[a-z0-9]{8,20}$/) ?
Promise.resolve() : Promise.reject()
);
}
// Returns a promise
function isUsernameNotReserved (username) {
return (
['igor', 'kristina'].indexOf(username) === -1 ?
Promise.resolve() : Promise.reject()
);
}
Now if we want to augment isUsernameNotReserved with asynchronous API call, we don't need to change anything else.
Option B - Only functions calling another functions return promises
Also, another option would be write functions in promises that call another functions. In that case, only validateUsername should be written as a promise from scratch.
Is this a good approach? What could be the drawbacks apart from performance?
Upd: ran a simple performance test and though running consequent promises is slower, it practically should not make any difference since running 100000 consequent functions takes ~200ms, while running 1000 takes ~3ms in Chrome. Fiddle here https://jsfiddle.net/igorpavlov/o7nb71np/2/

Should I always return promises in all functions in JavaScript?
No.
If you have a function that performs an asynchronous operation or may perform an asynchronous operation, then it is reasonable and generally good design to return a promise from that function.
But, if your function is entirely synchronous and no reasonable amount of forethought thinks this will sometime soon contain an asynchronous operation, then there are a bunch of reason why you should not return a promise from it:
Asynchronous code writing and testing is more complicated than synchronous code writing and testing. So, you really don't want to make code harder to write and test than it needs to be. If your code can be synchronous (and just return a normal value), then you should do so.
Every .then() handler gets called on the next tick (guaranteed asynchronously) so if you take a whole series of synchronous operations and force each function to wait until the next tick of the event loop, you're slowing down code execution. In addition, you're adding to the work of the garbage collector for every single function call (since there's now a promise object associated with every single function call).
Losing the ability to return a normal value from a synchronous function is a huge step backwards in the language tools that you can conveniently use to write normal code. You really don't want to give that up on every single function.
Now if we want to augment isUsernameNotReserved with asynchronous API call, we don't need to change anything else.
A good API design would anticipate whether an asynchronous API is relevant or likely useful in the near future and make the API async only in that case. Because asynchronous APIs are more work to write, use and test, you don't want to unnecessarily just make everything async "just in case". But, it would be wise to anticipate if you are likely to want to use an async operation in your API in the future or not. You just don't want to go overboard here. Remember, APIs can be added to or extended in the future to support new async things that come along so you don't have to over complicate things in an effort to pre-anticipate everything that ever might happen in the future. You want to draw a balance.
"Balance" seems like the right word here. You want to balance the likely future needs of your API with developer simplicity. Making everything return a promise does not draw a proper balance, as it chooses infinite flexibility in the future while over complicating things that don't need to be as complicated.
In looking at a couple of your particular examples:
validateUsernameFormat() does not seem likely to ever need to be asynchronous so I see no reason for it to return a promise.
isUsernameNotReserved() does seem like it may need to be asynchronous at some point since you may need to look some data up in a database in order to determine if the user name is available.
Though, I might point out that checking if a user name is available before trying to just create it can create a race condition. This type of check may still be used as UI feedback, but when actually creating the name, you generally need to create it in an atomic way such that two requests looking for the same name can't collide. Usually this is done by creating the name in the database with settings that will cause it to succeed if the name does not already exist, but fail if it does. Then, the burden is on the database (where it belongs) to handle concurrency issues for two requests both trying to create the same user name.

Are you supposed to be able to set a variable to a function return if Javascript is asynchronous?

var identifier = '0';
var serviceCodes = parseServiceCodes(identifier);
console.log(serviceCodes);
function parseServiceCodes(id){
var serviceCodes = id + 'HYJSIXNS';
return serviceCodes
}
0HYJSIXNS is returned in the console. But I thought that since JavaScript is asynchronous, the variable would be assigned before the function is returned, making it null.

Javascript isn't asynchronous by default, and code is executed in order unless you specify that it shouldn't be. You're never making a call to an external resource, so there's nothing to be async here.

JS, like any interpreted language, is run line-by-line. In your example, at the parseServiceCodes callsite, execution goes from line 2 (the callsite) to line 6 (the function body).
Async behavior is introduced for time consuming operations, like HTTP requests. In this case, the expensive function either takes a callback parameter (code to run once the expensive operation is over) or returns a Promise which "resolves" with the computed data when ready.
To illustrate the first case, let's say we have a function getFromServer which needs a URI and a callback:
function getFromServer(uri, callback){ // ...
It doesn't matter exactly what happens under the hood, just that execution is given back to the main program after invocation. This means the program isn't blocked for the duration of this expensive operation. When getFromServer is done, it will come back and call callback with some data.
To learn more about the second case, Promises, I encourage you to read the documentation I linked to earlier in this response.

You seem to misunderstand what it means to say that "JavaScript is asynchronous".
JavaScript is asynchronous because it supports and benefits from usage of asynchronicity, which is considered a common design pattern throughout the language by using features such as asynchronous callbacks, promises and even async/await, but this does not mean that all code is asynchronous.
You can easily write synchronous code, as you just demonstrated in your question, and there is no issue specifying a return value that is not a Promise.
This is because the code invoking it expects the return value to contain a string, which it then assigns to serviceCodes and uses that in the same tick.
If any of these terms above confuse you, please refer to their respective documentation and usage I've provided the links for.

Design pattern to implement plugins that depend on other plugins

I'm writing a tool that takes some data manually input by the user every 10 minutes, and runs a set of calculations on each input. Plugin A provides one calculation, Plugin B provides another, and so on. Mostly these plugins are independent of one another, i.e. order doesn't matter, because each plugin's calculation returns an integer that gets summed with the other plugins' integers.
But suppose now, I do have a Plugin C that depends on, say, whether Plugin A's return was non-zero. Data-wise, let's say I know how to make Plugin A's states available to Plugin C. (If it were C++, I'd make Plugin A a friend of Plugin C, for example. However, I'm writing this in Javascript, so I may take a looser approach.) My question is more about the pattern for ordering / dependence. How do I ensure that Plugin A's calculation runs before Plugin C's?
Of course, the simplest approach is to simplly "install" the plugins in the order they need to run, i.e. insert the plugins in the right order into an array so the loop looping through said array doesn't need to think.
But this may become fragile as I add more and more plugins (upwards of 20, maybe 30, depending on the scenario). I'd like something more robust.
The best idea I have right now is:
On "installing" a plugin, I supply an array of plugins it depends on.
Each plugin will have a static member, say, _complete, that indicates whether it was run, and gets reset on every new iteration (user input).
As I loop through each plugin, I check each plugin's dependency's _complete states; and if one isn't complete, then I don't run the calculation yet; the loop will be a while-loop that comes back to retry this plugin after attempting all the others. I'll also have a maximum-retries guard to prevent infinite loops.
How can this be improved?

As Gothdo suggested, Promises would work well on a problem like this. However, your code does not need to be async. There are no constraints in mixing and matching async and sync code when using promises. You will end up paying a performance overhead if your running loads of small synchronous functions, but with your use case of tens of functions the overhead is negligible.
Promises provide control flow abstraction for things that happen in the future: Run this, when that completes. Mostly used for async code. An argument might be made that this is hunting ducks with a minigun. I justify the choice with an argument about reinventing the wheel (or finding an obscure hexagonal wheel from the depths of GitHub), having a support for async ready, if the need arises and the fact that most JS programmers are already familiar with Promises and libraries are well supported. And the biggest one: Making the necessary wrapper around promises is extremely simple.
I made a quick sketch of what such a wrapper might look like. The code uses somewhat ugly deferred-pattern to enable adding tasks in any order. Feel free to add error handling, or otherwise modify to suit your needs.
function TaskRunner () {
this.tasks = {};
// name: String
// taskFn: fn(dep_1_result, dep_1_result...) -> result or Promise(result),
// deps: Optional, array of names or a name.
// Return: Promise over added task
TaskRunner.prototype.add = function (name, taskFn, deps) {
var self = this;
deps = (deps === undefined ? [] : (deps instanceof Array ? deps : [deps]));
name = name.toString();
self.tasks[name] = self.tasks[name] || {};
if(self.tasks[name].fn)
throw "Task " + name + " exists."
deps = deps.map(function (d) {
// Create result handler for deps, if none exist
self.tasks[d] = self.tasks[d] || {};
self.tasks[d].result = self.tasks[d].result || defer();
return self.tasks[d].result.promise;
});
// Create result handler for this task if none was created when
// handling deps of formely created tasks
self.tasks[name].result = self.tasks[name].result || defer();
// Excecute when all deps are done
self.tasks[name].fn = Promise.all(deps).spread(taskFn)
.then(function(res, err) {
// Trigger own result handler
if(err) {
self.tasks[name].result.reject(err);
throw err;
}
else {
self.tasks[name].result.resolve(res);
return res;
}
});
return self.tasks[name].fn;
}
}
Example usage: https://jsfiddle.net/3uL9chnd/4/
Bluebird promise lib: http://bluebirdjs.com/docs/api-reference.html
Edit, disclaimer: There's another point when considering overhead: reset efficiency. If you're running light calculations on a tight interval, creating a promise object for each task on each cycle makes this approach less than optimal.

Is Twisted's Deferred the same as a Promise in JavaScript?

I started using Twisted in a project that require asynchronous programming and the docs are pretty good.
So my question is, is a Deferred in Twisted the same as a Promise in Javascript? If not, what are the differences?

The answer to your question is both Yes and No depending on why you're asking.
Yes:
Both a Twisted Deferred and a Javascript Promise implement a mechanism for queuing synchronous blocks of code to be run in a given order while being decoupled from other synchronous blocks of code.
No:
So Javascript's Promise is actually more similar to Python's Future, and the airy-fairy way to explain this is to talk about the Promise and the Resolver being combined to make a Deferred, and to state that this affects what you can do with the callbacks.
This is all very well and good in that it's accurate, however it doesn't really make anything any clearer, and without typing thousands of words where I'm almost guaranteed to make a mistake, I'm probably better quoting someone who knows a little something about Python.
Guido van Rossum on Deferreds:
Here's my attempt to explain Deferred's big ideas (and there are a lot
of them) to advanced Python users with no previous Twisted experience.
I also assume you have thought about asynchronous calls before. Just
to annoy Glyph, I am using a 5-star system to indicate the importance
of ideas, where 1 star is "good idea but pretty obvious" and 5 stars
is "brilliant".
I am showing a lot of code snippets, because some ideas are just best
expressed that way -- but I intentionally leave out lots of details,
and sometimes I show code that has bugs, if fixing them would reduce
understanding the idea behind the code. (I will point out such bugs.)
I am using Python 3.
Notes specifically for Glyph: (a) Consider this a draft for a blog
post. I'd be more than happy to take corrections and suggestions for
improvements. (b) This does not mean I am going to change Tulip to a
more Deferred-like model; but that's for a different thread.
Idea 1: Return a special object instead of taking a callback argument
When designing APIs that produce results asynchronously, you find that
you need a system for callbacks. Usually the first design that comes
to mind is to pass in a callback function that will be called when the
async operation is complete. I've even seen designs where if you don't
pass in a callback the operation is synchronous -- that's bad enough
I'd give it zero stars. But even the one-star version pollutes all
APIs with extra arguments that have to be passed around tediously.
Twisted's first big idea then is that it's better to return a special
object to which the caller can add a callback after receiving it. I
give this three stars because from it sprout so many of the other good
ideas. It is of course similar to the idea underlying the Futures and
Promises found in many languages and libraries, e.g. Python's
concurrent.futures (PEP 3148, closely following Java Futures, both of
which are meant for a threaded world) and now Tulip (PEP 3156, using a
similar design adapted for thread-less async operation).
Idea 2: Pass results from callback to callback
I think it's best to show some code first:
class Deferred:
def __init__(self):
self.callbacks = []
def addCallback(self, callback):
self.callbacks.append(callback) # Bug here
def callback(self, result):
for cb in self.callbacks:
result = cb(result)
The most interesting bits are the last two lines: the result of each
callback is passed to the next. This is different from how things work
in concurrent.futures and Tulip, where the result (once set) is fixed
as an attribute of the Future. Here the result can be modified by each
callback.
This enables a new pattern when one function returning a Deferred
calls another one and transforms its result, and this is what earns
this idea three stars. For example, suppose we have an async function
that reads a set of bookmarks, and we want to write an async function
that calls this and then sorts the bookmarks. Instead of inventing a
mechanism whereby one async function can wait for another (which we
will do later anyway :-), the second async function can simply add a
new callback to the Deferred returned by the first one:
def read_bookmarks_sorted():
d = read_bookmarks()
d.addCallback(sorted)
return d
The Deferred returned by this function represents a sorted list of
bookmarks. If its caller wants to print those bookmarks, it must add
another callback:
d = read_bookmarks_sorted()
d.addCallback(print)
In a world where async results are represented by Futures, this same
example would require two separate Futures: one returned by
read_bookmarks() representing the unsorted list, and a separate Future
returned by read_bookmarks_sorted() representing the sorted list.
There is one non-obvious bug in this version of the class: if
addCallback() is called after the Deferred has already fired (i.e. its
callback() method was called) then the callback added by addCallback()
will never be called. It's easy enough to fix this, but tedious, and
you can look it up in the Twisted source code. I'll carry this bug
through successive examples -- just pretend that you live in a world
where the result is never ready too soon. There are other problems
with this design too, but I'd rather call the solutions improvements
than bugfixes.
Aside: Twisted's poor choices of terminology
I don't know why, but, starting with the project's own name, Twisted
often rubs me the wrong way with its choice of names for things. For
example, I really like the guideline that class names should be nouns.
But 'Deferred' is an adjective, and not just any adjective, it's a
verb's past participle (and an overly long one at that :-). And why is
it in a module named twisted.internet?
Then there is 'callback', which is used for two related but distinct
purposes: it is the preferred term used for a function that will be
called when a result is ready, but it is also the name of the method
you call to "fire" the Deferred, i.e. set the (initial) result.
Don't get me started on the neologism/portmanteau that is 'errback',
which leads us to...
Idea 3: Integrated error handling
This idea gets only two stars (which I'm sure will disappoint many
Twisted fans) because it confused me a lot. I've also noted that the
Twisted docs have some trouble explaining how it works -- In this case
particularly I found that reading the code was more helpful than the
docs.
The basic idea is simple enough: what if the promise of firing the
Deferred with a result can't be fulfilled? When we write
d = pod_bay_doors.open()
d.addCallback(lambda _: pod.launch())
how is HAL 9000 supposed to say "I'm sorry, Dave. I'm afraid I can't
do that" ?
And even if we don't care for that answer, what should we do if one of
the callbacks raises an exception?
Twisted's solution is to bifurcate each callback into a callback and
an 'errback'. But that's not all -- in order to deal with exceptions
raised by callbacks, it also introduces a new class, 'Failure'. I'd
actually like to introduce the latter first, without introducing
errbacks:
class Failure:
def __init__(self):
self.exception = sys.exc_info()[1]
(By the way, great class name. And I mean this, I'm not being
sarcastic.)
Now we can rewrite the callback() method as follows:
def callback(self, result):
for cb in self.callbacks:
try:
result = cb(result)
except:
result = Failure()
This in itself I'd give two stars; the callback can use
isinstance(result, Failure) to tell regular results apart from
failures.
By the way, in Python 3 it might be possible to do away with the
separate Failure class encapsulating exceptions, and just use the
built-in BaseException class. From reading the comments in the code,
Twisted's Failure class mostly exists so that it can hold all the
information returned by sys.exc_info(), i.e. exception class/type,
exception instance, and traceback but in Python 3, exception objects
already hold a reference to the traceback.There is some debug stuff
that Twisted's Failure class does which standard exceptions don't, but
still, I think most reasons for introducing a separate class have been
addressed.
But let's not forget about the errbacks. We change the list of
callbacks to a list of pairs of callback functions, and we rewrite the
callback() method again, as follows:
def callback(self, result):
for (cb, eb) in self.callbacks:
if isinstance(result, Failure):
cb = eb # Use errback
try:
result = cb(result)
except:
result = Failure()
For convenience we also add an errback() method:
def errback(self, fail=None):
if fail is None:
fail = Failure()
self.callback(fail)
(The real errback() function has a few more special cases, it can be
called with either an exception or a Failure as argument, and the
Failure class takes an optional exception argument to prevent it from
using sys.exc_info(). But none of that is essential and it makes the
code snippets more complicated.)
In order to ensure that self.callbacks is a list of pairs we must also
update addCallback() (it still doesn't work right when called after
the Deferred has fired):
def addCallback(self, callback, errback=None):
if errback is None:
errback = lambda r: r
self.callbacks.append((callback, errback))
If this is called with just a callback function, the errback will be a
dummy that passes the result (i.e. a Failure instance) through
unchanged. This preserves the error condition for a subsequent error
handler. To make it easy to add an error handler without also handling
a regular resullt, we add addErrback(), as follows:
def addErrback(self, errback):
self.addCallback(lambda r: r, errback)
Here, the callback half of the pair will pass the (non-Failure) result
through unchanged to the next callback.
If you want the full motivation, read Twisted's Introduction to
Deferreds; I'll just end by noting that an errback and substitute a
regular result for a Failure just by returning a non-Failure value
(including None).
Before I move on to the next idea, let me point out that there are
more niceties in the real Deferred class. For example, you can specify
additional arguments to be passed to the callback and errback. But in
a pinch you can do this with lambdas, so I'm leaving it out, because
the extra code for doing the administration doesn't elucidate the
basic ideas.
Idea 4: Chaining Deferreds
This is a five-star idea! Sometimes it really is necessary for a
callback to wait for an additional async event before it can produce
the desired result. For example, suppose we have two basic async
operations, read_bookmarks() and sync_bookmarks(), and we want a
combined operation. If this was synchronous code, we could write:
def sync_and_read_bookmarks():
sync_bookmarks()
return read_bookmarks()
But how do we write this if all operations return Deferreds? With the
idea of chaining, we can do it as follows:
def sync_and_read_bookmarks():
d = sync_bookmarks()
d.addCallback(lambda unused_result: read_bookmarks())
return d
The lambda is needed because all callbacks are called with a result
value, but read_bookmarks() takes no arguments.