We Keep Coding

Combining two data sets based on programme name

Hi I am trying to work out the best way to achieve something. I am essentially making two database calls
const [emails] = await dbConnection.execute('SELECT name, programme, timestamp FROM emails');
const [emailsCancelled] = await dbConnection.execute('SELECT data FROM emails where name = "email.cancelled"');
The reason I am making two calls is that I am processing over hundred thousand rows, and the data field contains quite a bit of JSON data, so don't want to retrieve that for all the rows.
So with the emails, I get data back in the following format
[
{
name: 'email.sent',
programme: 'Email One',
timestamp: 2022-03-24T18:06:02.000Z
},
{
name: 'email.sent',
programme: 'Email Two',
timestamp: 2022-03-24T18:06:02.000Z
},
{
name: 'email.sent',
programme: 'Email One',
timestamp: 2022-03-24T18:06:02.000Z
},
...
]
So what I needed to do is group by programme, to identify how many were sent and the total count. I do obtain some other details but reduced for this post. To do this I do
const emailsReduced = await emails.reduce((acc, o) => {
const name = o.name?.replace('email.', '');
if (!acc[o.programme]) {
acc[o.programme] = {
count: 0,
sent: 0,
};
}
acc[o.programme].count = (acc[o.programme].count || 0) + 1;
acc[o.programme][name] = (acc[o.programme][name]) + 1;
return acc;
}, {});
And that will return something like this
'Email One': {
count: 2,
sent: 2,
},
'Email Two': {
count: 1,
sent: 1,
},
Now emailsCancelled returns JSON data. So what I can do is loop it and show an example out the part I need
Object.entries(emailsCancelled).forEach(([key, value]) => {
const data = JSON.parse(value.data);
if (data?.payload?.body?.toUpperCase() === 'STOP') {
console.log(data?.payload?.correlation?.metadata);
}
});
And that will produce rows like this
[
{ customerId: '12345', programName: 'Email One' },
{ customerId: '2321', programName: 'Email Two' },
{ customerId: '33321', programName: 'Email Two' }
]
Now what I need to do is get that into the original array as a count. So you can see that there was 1 cancelled for Email One, and 2 for Two. So I need to add this in like so, matching it based on the programme name.
'Email One': {
count: 2,
sent: 2,
cancelled: 1,
},
'Email Two': {
count: 1,
sent: 1,
cancelled: 2,
},
How can I achieve something like this?
Thanks
Actual format
{
"name":"email.cancelled",
"payload":{
"body":"STOP",
"correlation":{
"metadata":{
"customerId":"232131232113",
"programName":"Email One"
}
},
"id":"123454323343232",
"receivedOn":"2022-05-15T12:51:54.403Z"
},
}

From emailsCancelled, you can reduce your array to a lookup Map before your perform your .reduce() on on emails. The lookup will store the programName as the keys, and the count of that program as the values:
const emails = [
{ customerId: '12345', programName: 'Email One' },
{ customerId: '2321', programName: 'Email Two' },
{ customerId: '33321', programName: 'Email Two' }
];
const lut = emails.reduce((map, {programName}) =>
map.set(programName, (map.get(programName) || 0) + 1)
, new Map);
console.log(lut.get("Email One"));
console.log(lut.get("Email Two"));
You can build this Map directly from your .forEach() loop also, note that I'm using Object.values() instead of .entries() as you're only intrested in the values and not the keys:
const lut = new Map();
Object.values(emailsCancelled).forEach(value => {
const data = JSON.parse(value.data);
if (data?.payload?.body?.toUpperCase() === 'STOP') {
const programName = data.payload.correlation?.metadata?.programName; // if `correcltation`, or `metadata` or `programName` don't exist, use optional chaining and an if-statement to check for `undefined` before updating the map.
lut.set(programName, (map.get(programName) || 0) + 1)
}
});
You can then use this lookup lut Map when you use .reduce() on emails to work out the cancelled value, defaulting cancelled to 0 if the programme can't be found in the Map:
const emailsReduced = await emails.reduce((acc, o) => {
const name = o.name?.replace('email.', '');
if (!acc[o.programme]) {
acc[o.programme] = {
count: 0,
sent: 0,
cancelled: lut.get(o.programme) || 0 // default to zero if program can't be found
};
}
acc[o.programme].count = acc[o.programme].count + 1;
acc[o.programme][name] = acc[o.programme][name] + 1;
return acc;
}, {});

Assuming your data structures look like these, you can map and filter according to emails keys:
const emails = [
{ 'Email One': {
count: 2,
sent: 2,
}},
{'Email Two': {
count: 1,
sent: 1,
}}
]
const canceled = [
{ customerId: '12345', programName: 'Email One' },
{ customerId: '2321', programName: 'Email Two' },
{ customerId: '33321', programName: 'Email Two' }
]
const newmails = emails.map(mail => {
let strmail = Object.keys(mail)
let ncanceled = canceled.filter(item => {
return item.programName == strmail
}).length
mail[strmail].canceled = ncanceled
return mail
})
console.log(newmails)

Try this!
const emails = [{
'Email One': {
count: 2,
sent: 2,
cancelled: 0,
},
},
{
'Email Two': {
count: 1,
sent: 1,
cancelled: 0,
},
},
];
const cancelled_emails = [{
customerId: '12345',
programName: 'Email One'
},
{
customerId: '2321',
programName: 'Email Two'
},
{
customerId: '33321',
programName: 'Email Two'
},
];
for (let cancelled_email of cancelled_emails) {
let prg_name = cancelled_email.programName;
for (email of emails) {
if (Object.keys(email)[0] === prg_name) {
email[prg_name].cancelled += 1;
}
}
}
console.log(emails);

Check if a element in array exists but not perform any task javascript

So I have been tasked with making a basic workflow engine. I have it checking the config file for dependencies and then performing the next function. what I would like to is check to see if the element already exists and not push it to the output array. I have tried .includes() and .indexOf() but I cant seem to get them to work.
const TestWorkflowConfig = {
insertDetails: {
dependencies: [],
workflow: { name: "updateRow", id: 10 },
description: "This is a test function where details need to be entered (row update)",
data: {},
taskName: "insertDetails",
},
detailsConfirmed: {
{ insertDetails: { isRequired: true; } }
dependencies: ["insertDetails"],
workflow: { name: "updateRow", id: 10; },
description: "this is to confirm details (update row status)",
data: {},
taskName: "detailsConfirmed",
},
sendConfirmationEmail: {
dependencies: ["detailsConfirmed"],
workflow: { name: "sendEmail", id: 8; },
description: "this is a test email to send out to confirm details (send email workflow)",
data: { name: "james", email: "james#email.com", body: "this is a test email please ignore"; },
taskName: "sendConfirmationEmail",
},
};
const taskQueue = [
{
"processName": "sendConfirmationEmail",
"isComplete": false
},
{
"processName": "detailsConfirmed",
"isComplete": false
},
{
"processName": "insertDetails",
"isComplete": true
}
];
const workflowTaskQueue = [];
const config = TestWorkflowConfig;
const completedJobs = [];
for (const element of taskQueue) {
if (element.isComplete === true) {
completedJobs.push(element.processName);
}
}
for (const element in config) {
if (config[element].dependencies <= 0) {
// I would like to check if the "config[element]" is in the completedJobs array and only push if it is not there.
workflowTaskQueue.push({ json: config[element] });
} else {
const dependencies = config[element].dependencies;
const dep = [];
for (const el of dependencies) {
dep.push(el);
const result = dep.every((i) => completedJobs.includes(i));
if (result === true) {
// and again I would like to check if the "config[element]" is in the completedJobs array and only push if it is not there
workflowTaskQueue.push({ json: config[element] });
}
}
console.log("taskQueue: " + taskQueue);
console.log("completedJobs: " + completedJobs);
console.log("output:" + workflowTaskQueue);
}
}
as always any help is greatly appreciated.

Filter an array of objects using multiple values from the object

So I have an array of objects that looks like this :
let medicines = [
{
id:3340,
name:nutraplus,
description:"some medicine",
ingredients: [{
ingredient:"glycerol"
},
{
ingredient:"Morphine"
}
]
},
{
id:3320,
name:Panadol,
description:"tablet",
ingredients: [{
ingredient:"Paracetamol"
},
{
ingredient:"Some stuff"
}
]
}
]
I want to to be able to filter by name and by ingredient name I have acheived the former by doing this :
computed: {
medicines() {
return this.$store.state.medicines.filter(med => {
//this.search is the what comes after typing in search bar
return med.name.toLowerCase().includes(this.search.toLowerCase())
})
},
}
Its vue.js so the computed() stuff anyways this works perfectly when searching by name however i also want to be able to search by ingredients from the same search bar. I tried something like this :
edicines() {
return this.$store.state.medicines.filter(med => {
return med.name.toLowerCase().includes(this.search.toLowerCase()) || med.ingredients.map(ing=>{
ing.ingredient.name.toLowerCase().includes(this.search.toLower)
})
})
}
But it didn't work. Any ideas on how to get this working? Thank you for your time.

Haven't used vue in the example, you just need to extract the logic behind the filtering that I have done (Simple JS filtering)
As example -
Try searching for 'Para' - It must return the entries with name/ingredient containing Para
Try searching for 'stuff' - It should return two entries (since both medicine in that array consist of 'some stuff' as ingredient)
let medicines = [{
id: 3340,
name: 'nutraplus',
description: "some medicine",
ingredients: [{
ingredient: "glycerol"
},
{
ingredient: "Morphine"
},
{
ingredient: "Some stuff"
}
]
},
{
id: 3320,
name: 'Panadol',
description: "tablet",
ingredients: [{
ingredient: "Paracetamol"
},
{
ingredient: "Some stuff"
}
]
},
{
id: 3311,
name: 'Amazin',
description: "tablet"
}
];
const form = document.querySelector('form')
form.addEventListener('submit', (e) => {
e.preventDefault();
const searchValue = form.searchText.value.toLowerCase();
const matchValue = medicines.filter(medicine => {
return medicine.name.toLowerCase().includes(searchValue) || (medicine.ingredients ? medicine.ingredients.filter(ingredientObj => {
return ingredientObj.ingredient.toLowerCase().includes(searchValue);
}).length > 0 : false);
});
document.querySelector('.result').textContent = JSON.stringify(matchValue, null, 4);
});
pre {
background: #c5c5c5;
}
<form>
<label for="searchText"></label>
<input type="text" id="searchText" name="searchText">
<button>Search</button>
</form>
<pre class='result'></pre>

This should work.
let medicines = [
{
id:3340,
name:"nutraplus",
description:"some medicine",
ingredients: [{
ingredient:"glycerol"
},
{
ingredient:"Morphine"
}
]
},
{
id:3320,
name:"Panadol",
description:"tablet",
ingredients: [{
ingredient:"Paracetamol"
},
{
ingredient:"Some stuff"
}
]
}
];
const searchPhrase = "Paracetamol";
const filteredByName = medicines.filter((medicine) => {
return medicine.name.toLowerCase() === searchPhrase.toLowerCase();
});
const filteredByIngredient = medicines.filter((medicine) => {
return medicine.ingredients.some((item) => item.ingredient.toLowerCase() === searchPhrase.toLowerCase());
})
const result = [...filteredByName, ...filteredByIngredient];
console.log(result)

Sequelize - After findAll, delete with a single query

I have a table with an associated table. I am using beforeDestroy to remove any associated records up deletion.
Model.beforeDestroy(async (category: any) => {
const items = await Category.findAll({
where: {
category_id: category.id,
},
attributes: ['id'],
raw: true,
});
console.log(items); // [ { id: 2 }, { id: 3364 }, { id: 3365 } ]
items.map((item: any) => {
Category.destroy({ where: { id: item.id } });
});
});
}
I am trying to delete the matching items with a single destroy query rather than mapping through.

try:
Category.destroy({ where: { id:items.map(item=>item.id)} });

MongoDB move element of sub-document to another sub-document

I need to change my schema
const dataSchema = new Schema({
trackingPrice: [{
timeCheck: { type: Date, default: Date.now },
lowestPrice: Number,
salePrice: Number,
saleRank: Number
}]
})
to this
const dataSchema = new Schema({
trackingPrice: [{
timeCheck: { type: Date, default: Date.now },
lowestPrice: Number,
salePrice: Number
}],
trackingRank: [{
timeCheck: { type: Date, default: Date.now },
saleRank: Number
}]
})
How can I transfer my data from one to another under document and then delete "saleRank"?

Based on this very good answer, the cursor in your case would be derived from running the aggregate pipeline:
const pipeline = [
{
"$project": {
"trackingPrice": {
"$map": {
"input": "$trackingPrice",
"as": "el",
"in": {
"timeCheck": "$$el.timeCheck",
"lowestPrice": "$$el.timeCheck",
"salePrice": "$$el.salePrice"
}
}
},
"trackingRank": {
"$map": {
"input": "$trackingPrice",
"as": "el",
"in": {
"timeCheck": "$$el.timeCheck",
"saleRank": "$$el.saleRank"
}
}
}
}
}
];
const cursor = Data.aggregate(pipeline).exec();
Running the bulk update:
let bulkUpdateOps = [];
cursor.then(results => {
results.forEach(doc => {
const { _id, trackingPrice, trackingRank } = doc;
bulkUpdateOps.push({
"updateOne": {
"filter": { _id },
"update": { "$set": { trackingPrice, trackingRank } },
"upsert": true
}
});
});
if (bulkUpdateOps.length === 1000) {
bulkUpdateOps = [];
return Data.bulkWrite(bulkUpdateOps);
}
}).then(console.log).catch(console.error);
if (bulkUpdateOps.length > 0) {
Data.bulkWrite(bulkUpdateOps).then(console.log).catch(console.error);
}

The best way I see is to find all of them, and foreach one create a new one
I'm using mongoose by the way
dataSchema.find({}, (err,all) => {
var array = [];
all.foreach( (ds) => {
array.push({
trackingPrice: ds.trackingPrice,
trackingRank: //whatever way you want to update the old data
})
dataSchema.remove({}).exec(() => {
array.foreach((a) => {
var toadd = new dataSchema({
trackingPrice:a.trackingPrice,
trackingRank:a.trackingRank});
toadd.save();
})
})
})}
);