We Keep Coding

JS - Map/Array choice when both can do the same task

This is probably more of an opinion based question. But recently on a project I have seen that Map is always used over array.
My question is when would you ever use an array over a Map?
Note I mean in cases where an array would work as well as a Map.
i.e. storing data, checking it exists, and deleting it.
example here:
// Array
const teams = [];
if (teams.includes(team.id)) {
return;
}
teams.push(team.id);
// do stuff
teams = teams.filter(
(id) => id !== team.id
);
// Map
const teams = new Map();
if (teams.has(team.id)) {
return;
}
teams.set(team.id, team.id);
// do stuff
teams(team.id);
As I understand Map is more performant, you also get methods like get, set, delete which are useful.
If faced with the task above what would people use and why?

Update HTML data without creating more elements on the webpage

Have an interval that loads the html every 3 secs, but want to refresh it and not keep adding more under the already made code.
async function Products(){
setInterval(async function() {
const response = await fetch('http://localhost:3000/api/products');
const data = await response.json();
const mainContainer = document.getElementById("myData");
for (let obj of data) {
const div = document.createElement("div");
div.innerHTML = `${obj["sku"]}: ${obj["name"]}`;
mainContainer.appendChild(div);
}
}, 10000)
}
When I click a start button everything works, but how do i make it refresh the already made HTML rather than repeatedly recreating it with the interval. Trying to figure out a good approach to this. Thanks

Create and append a <div> immediately, and in the interval, assign to its innerHTML:
function Products() {
const container = document.getElementById("myData").appendChild(document.createElement("div"));
setInterval(async function () {
const response = await fetch('http://localhost:3000/api/products');
const data = await response.json();
container.innerHTML = '';
for (let obj of data) {
container.innerHTML += `${obj["sku"]}: ${obj["name"]}`;
}
}, 10000)
}

I think what you are trying to do is implement a Serie of observable elements that you can look only for those who have changed instead of all the data, something like React does with the virtual DOM.
Considering the code tout already posted, refreshing elements at a set interval is a bad idea. What if you have 1000 user refreshing at the same time? What if it cause your response tone to be more then 3 seconds?
That said, if you really want to work on creating something like that, you have to find a way to load not all the products from the api, but only the ones that are different. Again, if you want to keep it that way, here are, in my opinion, what you could do:
Start by loading all the product on the page, but set an interval to check a new Endpoint which would tell you what products have been added after the last one.
Use either an index or a key to identify which product is which, so tout know the ones you have.
You need a way to know quick product was updated since the last load.
That's a start. You can implement these in different way. I would suggest having a timestamp for the time created and updated, so you can then query only those who fall after this timestamp.
Add a dynamic ID to your product elements, (e.g. <div id=${sku} >**Product Here**<\div>
That way, you can track your product and recreate only the ones who changed/are new.
That's obviously a complicated way of implementing an open connection, if you want another solution, you could also open a Socket with your api, which would send event for data updated and created and would ultimately make your 3 second ping obsolete, resulting in a better scalability in my opinion.

More efficient way to handle an extremely specific response from a Rest API in JavaScript

I'm working with a Dungeons & Dragons 5e API and want that an especific result be treated in a special way. The user can choose what to search from a range of options, and in only one of them would I need to take care of the answer in a different way. In this option, I get the answer in JSON that contains a 'name' field, which stores a String, but in this specific case this String comes with an acronym, and I would like to transform it into the full name.I'm afraid to just put am 'if' statement in the middle of the code and deal with the situation inefficiently, even more so that I did not find similar situations to have any reference.
This is part of the result of the API I want to handle in a special way:
{"count":6,
"results":[
{"name":"STR",
"url":"http://www.dnd5eapi.co/api/ability-score/1"},
{"name":"DEX",
"url":"http://www.dnd5eapi.co/api/ability-scores2"},
....
]
}
This is how I handle the answer:
fetch(fullAPIURL)
.then(result => result.json())
.then(data => {
let resultContainer = document.getElementById('resultContainer');
//Cleaning the result container from previous results
document.querySelectorAll('#resultContainer article').forEach(container =>
resultContainer.removeChild(container));
spanSearchResult.classList.remove('invisible', 'searchFail');
spanSearchResult.classList.add('searchSucess');
spanSearchResult.innerHTML = `Search returned ${data.count} results`;
for (element of data.results) {
let containerTitle = element.name != undefined ? element.name : element.class;
resultContainer.appendChild(createResultContainer(containerTitle));
}
})
.catch(err => {
spanSearchResult.classList.remove('invisible');
spanSearchResult.classList.add('searchFail');
spanSearchResult.innerHTML = 'Something went wrong! Details in the console';
console.log(err);
});
Is putting a condition in this snippet of code really the most efficient way to solve this situation?
Thanks in advance.

You could just make a lookup call, actually. In fact, that'd be preferable if you ever want to port your application to another language, for example.
Define the following:
var retrieve = (function() {
var items = {
"STR": "Strength",
"DEX": "Dexterity"
};
return function(item) {
return items[item] || item;
}
})();
console.log(retrieve("DEX"));
With this, you can simply call retrieve(element.name) to retrieve its "actual" name. You can add elements to the object to create new translations, and if you ever need to support multiple languages, you can even replace the function entirely.

Insert multiple entries to SQL Server with Node.js

I am rewriting an old API for which I am trying to insert multiple values at once into a MSSQL-Server (2008) database using the node module mssql. Now, I am capable of doing this somehow, but I want to this following best practices. I've done my research and tried a lot of things to accomplish my target. However, I was not able to find a single solution which works just right.
Before
You may wonder:
Well, you are rewriting this API, so there must be a way this has been done before and that was working?
Sure, you're right, it was working before, but... not in a way I'd feel comfortable with using in the rewrite. Let me show you how it was done before (little bit of abstraction added of course):
const request = new sql.Request(connection);
let query = "INSERT INTO tbl (col1, col2, col3, col4) VALUES ";
for (/*basic for loop w/ counter variable i*/) {
query += "(1, #col2" + [i] + ", #col3" + [i] + ", (SELECT x FROM y WHERE z = #someParam" + [i] + "))";
// a check whether to add a comma or not
request.input("col2" + [i], sql.Int(), values[i]);
// ...
}
request.query(query, function(err, recordset) {
// ...
}
While this is working, again, I don't quite think this could be called anything like 'best practice'. Also this shows the biggest problem: a subselect is used to insert a value.
What I tried so far
The easy way
At first I tried the probably easiest thing:
// simplified
const sQuery = "INSERT INTO tbl (col1, col2, col3, col4) VALUES (1, #col2, #col3, (SELECT x FROM y WHERE z = #col4));";
oPool.request().then(oRequest => {
return oRequest
.input("col2", sql.Int(), aValues.map(oValue => oValue.col2))
.input("col3", sql.Int(), aValues.map(oValue => oValue.col3))
.input("col4", sql.Int(), aValues.map(oValue => oValue.col4))
.query(sQuery);
});
I'd say, this was a pretty good guess and actually working relative fine.
Except for the part, that ignores every item after the first one... which makes this pretty useless. So, I tried...
Request.multiple = true
...and I thought, it would do the job. But - surprise - it doesn't, still only the first item is inserted.
Using '?' for parameters
At this point I really started the search for a solution, as the second one was only a quick search in the modules documentation.
I stumbled upon this answer and tried it immediately.
Didn't take long for my terminal to spit out a
RequestError: Incorrect syntax near '?'.
So much for that.
Bulk inserting
Some further research led to bulk inserting.
Pretty interesting, cool feature and excellent updating of the question with the solution by the OP!
I had some struggle getting started here, but eventually it looked really good: Multiple records were inserted and the values seemed okay.
Until I added the subquery. Using it as value for a column declared didn't cause any error, however when checking the values of the table, it simply displayed a 0 as value for this column. Not a big surprise at all, but everybody can dream, right?
The lazy way
I don't really know what to think about this:
// simplified
Promise.all(aValues.map(oValue => {
return oPool.request().then(oRequest =>
oRequest
.input("col2", sql.Int, oValue.col2)
.input("col3", sql.Int, oValue.col3)
.input("col4", sql.Int, oValue.col4)
.query(sQuery);
});
});
It does the job, but if any of the request fails for whichever reason, the other, non-failing inserts, will still be executed, even though this should not be possible.
Lazy + Transaction
As continuing even if some fail was the major problem with the last method, I tried building a transaction around it. All querys are successful? Good, commit. Any query has an errpr? Well, just rollback than. So I build a transaction, moved my Promise.all construct into it and tried again.
Aaand the next error pops up in my terminal:
TransactionError: Can't acquire connection for the request. There is another request in progress.
If you came this far, I don't need to tell you what the problem is.
Summary
What I didn't try yet (and I don't think I will try this) is using the transaction way and calling the statements sequentially. I do not believe that this is be the way to go.
And I also don't think the lazy way is the one that should be used, as it uses single requests for every record to insert, when this could somehow be done using only one request. It's just that this somehow is, I don't know, not in my head right now. So please, if you have anything that could help me, tell me.
Also, if you see anything else that's wrong with my code, feel free to point it out. I am not considering myself as a beginner, but I also don't think that learning will ever end. :)

The way I solved this was using PQueue library with concurrency 1. Its slow due to concurrency of one but it works with thousands of queries:
const transaction = sql.transaction();
const request = transaction.request();
const queue = new PQueue({ concurrency: 1 });
// being transaction
await transaction.begin();
for (const query of queries) {
queue.add(async () => {
try {
await request.query(query);
} catch (err) {
// stop pending transactions
await queue.clear();
await queue.onIdle();
// rollback transaction
await transaction.rollback();
// throw error
throw err;
}
});
}
// await queue
await queue.onIdle();
// comit transaction
await transaction.commit();

Better performance when saving large JSON file to MySQL

I have an issue.
So, my story is:
I have a 30 GB big file (JSON) of all reddit posts in a specific timeframe.
I will not insert all values of each post into the table.
I have followed this series, and he coded what I'm trying to do in Python.
I tried to follow along (in NodeJS), but when I'm testing it, it's way too slow. It inserts one row every 5 seconds. And there 500000+ reddit posts and that would literally take years.
So here's an example of what I'm doing in.
var readStream = fs.createReadStream(location)
oboe(readStream)
.done(async function(post) {
let { parent_id, body, created_utc, score, subreddit } = data;
let comment_id = data.name;
// Checks if there is a comment with the comment id of this post's parent id in the table
getParent(parent_id, function(parent_data) {
// Checks if there is a comment with the same parent id, and then checks which one has higher score
getExistingCommentScore(parent_id, function(existingScore) {
// other code above but it isn't relevant for my question
// this function adds the query I made to a table
addToTransaction()
})
})
})
Basically what that does, is to start a read stream and then pass it on to a module called oboe.
I then get JSON in return.
Then, it checks if there is a parent saved already in the database, and then checks if there is an existing comment with the same parent id.
I need to use both functions in order to get the data that I need (only getting the "best" comment)
This is somewhat how addToTransaction looks like:
function addToTransaction(query) {
// adds the query to a table, then checks if the length of that table is 1000 or more
if (length >= 1000) {
connection.beginTransaction(function(err) {
if (err) throw new Error(err);
for (var n=0; n<transactions.length;n++) {
let thisQuery = transactions[n];
connection.query(thisQuery, function(err) {
if (err) throw new Error(err);
})
}
connection.commit();
})
}
}
What addToTransaction does, is to get the queries I made and them push them to a table, then check the length of that table and then create a new transaction, execute all those queries in a for loop, then comitting (to save).
Problem is, it's so slow that the callback function I made doesn't even get called.
My question (finally) is, is there any way I could improve the performance?
(If you're wondering why I am doing this, it is because I'm trying to create a chatbot)
I know I've posted a lot, but I tried to give you as much information as I could so you could have a better chance to help me. I appreciate any answers, and I will answer the questions you have.