We Keep Coding

Combining two data sets based on programme name

Hi I am trying to work out the best way to achieve something. I am essentially making two database calls
const [emails] = await dbConnection.execute('SELECT name, programme, timestamp FROM emails');
const [emailsCancelled] = await dbConnection.execute('SELECT data FROM emails where name = "email.cancelled"');
The reason I am making two calls is that I am processing over hundred thousand rows, and the data field contains quite a bit of JSON data, so don't want to retrieve that for all the rows.
So with the emails, I get data back in the following format
[
{
name: 'email.sent',
programme: 'Email One',
timestamp: 2022-03-24T18:06:02.000Z
},
{
name: 'email.sent',
programme: 'Email Two',
timestamp: 2022-03-24T18:06:02.000Z
},
{
name: 'email.sent',
programme: 'Email One',
timestamp: 2022-03-24T18:06:02.000Z
},
...
]
So what I needed to do is group by programme, to identify how many were sent and the total count. I do obtain some other details but reduced for this post. To do this I do
const emailsReduced = await emails.reduce((acc, o) => {
const name = o.name?.replace('email.', '');
if (!acc[o.programme]) {
acc[o.programme] = {
count: 0,
sent: 0,
};
}
acc[o.programme].count = (acc[o.programme].count || 0) + 1;
acc[o.programme][name] = (acc[o.programme][name]) + 1;
return acc;
}, {});
And that will return something like this
'Email One': {
count: 2,
sent: 2,
},
'Email Two': {
count: 1,
sent: 1,
},
Now emailsCancelled returns JSON data. So what I can do is loop it and show an example out the part I need
Object.entries(emailsCancelled).forEach(([key, value]) => {
const data = JSON.parse(value.data);
if (data?.payload?.body?.toUpperCase() === 'STOP') {
console.log(data?.payload?.correlation?.metadata);
}
});
And that will produce rows like this
[
{ customerId: '12345', programName: 'Email One' },
{ customerId: '2321', programName: 'Email Two' },
{ customerId: '33321', programName: 'Email Two' }
]
Now what I need to do is get that into the original array as a count. So you can see that there was 1 cancelled for Email One, and 2 for Two. So I need to add this in like so, matching it based on the programme name.
'Email One': {
count: 2,
sent: 2,
cancelled: 1,
},
'Email Two': {
count: 1,
sent: 1,
cancelled: 2,
},
How can I achieve something like this?
Thanks
Actual format
{
"name":"email.cancelled",
"payload":{
"body":"STOP",
"correlation":{
"metadata":{
"customerId":"232131232113",
"programName":"Email One"
}
},
"id":"123454323343232",
"receivedOn":"2022-05-15T12:51:54.403Z"
},
}

From emailsCancelled, you can reduce your array to a lookup Map before your perform your .reduce() on on emails. The lookup will store the programName as the keys, and the count of that program as the values:
const emails = [
{ customerId: '12345', programName: 'Email One' },
{ customerId: '2321', programName: 'Email Two' },
{ customerId: '33321', programName: 'Email Two' }
];
const lut = emails.reduce((map, {programName}) =>
map.set(programName, (map.get(programName) || 0) + 1)
, new Map);
console.log(lut.get("Email One"));
console.log(lut.get("Email Two"));
You can build this Map directly from your .forEach() loop also, note that I'm using Object.values() instead of .entries() as you're only intrested in the values and not the keys:
const lut = new Map();
Object.values(emailsCancelled).forEach(value => {
const data = JSON.parse(value.data);
if (data?.payload?.body?.toUpperCase() === 'STOP') {
const programName = data.payload.correlation?.metadata?.programName; // if `correcltation`, or `metadata` or `programName` don't exist, use optional chaining and an if-statement to check for `undefined` before updating the map.
lut.set(programName, (map.get(programName) || 0) + 1)
}
});
You can then use this lookup lut Map when you use .reduce() on emails to work out the cancelled value, defaulting cancelled to 0 if the programme can't be found in the Map:
const emailsReduced = await emails.reduce((acc, o) => {
const name = o.name?.replace('email.', '');
if (!acc[o.programme]) {
acc[o.programme] = {
count: 0,
sent: 0,
cancelled: lut.get(o.programme) || 0 // default to zero if program can't be found
};
}
acc[o.programme].count = acc[o.programme].count + 1;
acc[o.programme][name] = acc[o.programme][name] + 1;
return acc;
}, {});

Assuming your data structures look like these, you can map and filter according to emails keys:
const emails = [
{ 'Email One': {
count: 2,
sent: 2,
}},
{'Email Two': {
count: 1,
sent: 1,
}}
]
const canceled = [
{ customerId: '12345', programName: 'Email One' },
{ customerId: '2321', programName: 'Email Two' },
{ customerId: '33321', programName: 'Email Two' }
]
const newmails = emails.map(mail => {
let strmail = Object.keys(mail)
let ncanceled = canceled.filter(item => {
return item.programName == strmail
}).length
mail[strmail].canceled = ncanceled
return mail
})
console.log(newmails)

Try this!
const emails = [{
'Email One': {
count: 2,
sent: 2,
cancelled: 0,
},
},
{
'Email Two': {
count: 1,
sent: 1,
cancelled: 0,
},
},
];
const cancelled_emails = [{
customerId: '12345',
programName: 'Email One'
},
{
customerId: '2321',
programName: 'Email Two'
},
{
customerId: '33321',
programName: 'Email Two'
},
];
for (let cancelled_email of cancelled_emails) {
let prg_name = cancelled_email.programName;
for (email of emails) {
if (Object.keys(email)[0] === prg_name) {
email[prg_name].cancelled += 1;
}
}
}
console.log(emails);

Check if a element in array exists but not perform any task javascript

So I have been tasked with making a basic workflow engine. I have it checking the config file for dependencies and then performing the next function. what I would like to is check to see if the element already exists and not push it to the output array. I have tried .includes() and .indexOf() but I cant seem to get them to work.
const TestWorkflowConfig = {
insertDetails: {
dependencies: [],
workflow: { name: "updateRow", id: 10 },
description: "This is a test function where details need to be entered (row update)",
data: {},
taskName: "insertDetails",
},
detailsConfirmed: {
{ insertDetails: { isRequired: true; } }
dependencies: ["insertDetails"],
workflow: { name: "updateRow", id: 10; },
description: "this is to confirm details (update row status)",
data: {},
taskName: "detailsConfirmed",
},
sendConfirmationEmail: {
dependencies: ["detailsConfirmed"],
workflow: { name: "sendEmail", id: 8; },
description: "this is a test email to send out to confirm details (send email workflow)",
data: { name: "james", email: "james#email.com", body: "this is a test email please ignore"; },
taskName: "sendConfirmationEmail",
},
};
const taskQueue = [
{
"processName": "sendConfirmationEmail",
"isComplete": false
},
{
"processName": "detailsConfirmed",
"isComplete": false
},
{
"processName": "insertDetails",
"isComplete": true
}
];
const workflowTaskQueue = [];
const config = TestWorkflowConfig;
const completedJobs = [];
for (const element of taskQueue) {
if (element.isComplete === true) {
completedJobs.push(element.processName);
}
}
for (const element in config) {
if (config[element].dependencies <= 0) {
// I would like to check if the "config[element]" is in the completedJobs array and only push if it is not there.
workflowTaskQueue.push({ json: config[element] });
} else {
const dependencies = config[element].dependencies;
const dep = [];
for (const el of dependencies) {
dep.push(el);
const result = dep.every((i) => completedJobs.includes(i));
if (result === true) {
// and again I would like to check if the "config[element]" is in the completedJobs array and only push if it is not there
workflowTaskQueue.push({ json: config[element] });
}
}
console.log("taskQueue: " + taskQueue);
console.log("completedJobs: " + completedJobs);
console.log("output:" + workflowTaskQueue);
}
}
as always any help is greatly appreciated.

How to use .each to build an array

So we're working in this system and buildig our own page. We built a form to insert timeline data using a .xwd file. We use javascript to retrieve the data and fill it in a variable to store it. The main page (title:) just has single values, but the actual events should be in an array.
I'm want to use V to fill the array.
$(x_currentPageXML).children().each(function(index, elem){
});
Right now what I have is this and I want to fill the "events" array using the foreach I showed above. Putting the .each inside in the var didn't work and I wouldn't know how else to do it.
var SictTimeline = new function() {
this.loadJS = function () {
$.getScript(x_templateLocation + 'common_html5/js/timeline.js')
.done(function (script, textStatus) {
var make_the_json = $(x_currentPageXML).children().map(function (element) {
return {
title: {
media: {
url: element.getAttribute("url"),
caption: element.getAttribute("tip"),
},
text: {
headline: element.getAttribute("name"),
text: '<p>' + element.getAttribute("text") + '</p>'
}
},
events: {
media: {
url: element.getAttribute("url"),
caption: element.getAttribute("tip"),
},
start_date: {
month: '8',
day: '9',
year: '1963'
},
text: {
headline: element.getAttribute("name"),
text: element.getAttribute("text")
}
}
}
})
var timeline_json = make_the_json; // replace make_the_json() with the JSON object you created
// two arguments: the id of the Timeline container (no '#')
// and the JSON object or an instance of TL.TimelineConfig created from
// a suitable JSON object
window.timeline = new TL.Timeline('timeline-embed', timeline_json);
})
.fail(function (jqxhr, settings, exception) {
console.log('Failed to load the script for the timeline');
});
}
// function called every time the page is viewed after it has initially loaded
this.pageChanged = function() {
}
// function called every time the size of the LO is changed
this.sizeChanged = function() {
}
this.init = function() {
this.loadJS();
// call this function in every model once everything's loaded
x_pageLoaded();
}
}
An example of the xml-file with the values
<?xml version="1.0"?>
<learningObject editorVersion="3" targetFolder="Nottingham" name="Learning Object Title" language="en-GB" navigation="Linear" textSize="12" theme="default" displayMode="fill window" responsive="true">
<SictTimeline linkID="PG1592486441661" name="My page" media="SictTimeline" text="<p>Text for my page</p>
" url="FileLocation + 'media/https___images.genius.com_53c4575fa3f97a8d4b18d69a600afaf0.900x900x1.jpg'" tip="Description for Image 1"></SictTimeline></learningObject>

I guess what you are trying to achieve is to generate an array of objects based on the number (and properties) of elements inside $(x_currentPageXML). For that purpose you need to use the .map() method:
events: $(x_currentPageXML).children().map(function (index, element) {
return {
media: {
url: element.getAttribute("url"),
caption: element.getAttribute("tip"),
},
start_date: {
month: '8',
day: '9',
year: '1963'
},
text: {
headline: element.getAttribute("name"),
text: element.getAttribute("text")
}
}
}).get()

I'm not sure I completely understand the question, but you want to extract stuff from each element in the loop right?
Replace x_currentPageXML with elem inside the loop
var result = []
$(x_currentPageXML).children().each(function (index, elem) {
var make_the_json = {
title: {
media: {
url: elem.getAttribute("url"),
caption: elem.getAttribute("tip"),
},
text: {
headline: elem.getAttribute("name"),
text: '<p>' + elem.getAttribute("text") + '</p>'
}
},
events: [
{
media: {
url: elem.getAttribute("url"),
caption: elem.getAttribute("tip"),
},
start_date: {
month: '8',
day: '9',
year: '1963'
},
text: {
headline: elem.getAttribute("name"),
text: elem.getAttribute("text")
}
}
]
};
result.push(make_the_json)
})

Calling Values from Functions with PouchDB using Find() within a loop

I am trying to loop through the results from a Find() Mango Query and make a all to another function to get extra data to use in my report.
I am looping a list of patient documents from a Find() query but I want to pull in the "last visit" from another list of "visit" documents by calling a function that performs a query but I am having problems.
I can call the function "Get_Static_Value()" and it will return a value however when I send the Patient_ID to the function "Get_Last_Visit(Patient_ID)" then the value comes back as "undefined" although the function is called and will write the "Vist_Date" to the console.
I believe my issue is caused because the promise in the query is not resolving but I am unsure of the syntax to get the value back into my loop once the function has processed.
I read the documement https://pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html and in the section "Rookie mistake #2: WTF, how do I use forEach() with promises?" I think it identifies my problem with the syntax:
db.allDocs({include_docs: true}).then(function (result) {
return Promise.all(result.rows.map(function (row) {
return db.remove(row.doc);
}));
}).then(function (arrayOfResults) {
// All docs have really been removed() now!
});
However the code above is for alldocs and not find() so I am a little bit stuck on how I process the same method on results from a Find() query.
I have created a JSfiddle to show my code and demonstrate my issue.
https://jsfiddle.net/movitico/gkb89uyf/
// Create the Database
var db = new PouchDB('patient_test');
// Add Patient Documents
function Add_Patients() {
db.bulkDocs([{
_id: '1',
type: 'patient',
Patient_Name: 'Patient 1',
Patient_Status: 'Active'
},
{
_id: '2',
type: 'patient',
Patient_Name: 'Patient 2',
Patient_Status: 'Active'
},
{
_id: '3',
type: 'patient',
Patient_Name: 'Patient 3',
Patient_Status: 'Active'
}
]);
}
function Add_Visits() {
// Add Visit Documents
db.bulkDocs([{
_id: 'v1',
type: 'visit',
Patient_ID: '1',
Visit_Date: "06/01/2018"
},
{
_id: 'v2',
type: 'visit',
Patient_ID: '1',
Visit_Date: "05/01/2018"
},
{
_id: 'v3',
type: 'visit',
Patient_ID: '1',
Visit_Date: "02/22/2018"
},
{
_id: 'v4',
type: 'visit',
Patient_ID: '2',
Visit_Date: "02/22/2014"
},
{
_id: 'v5',
type: 'visit',
Patient_ID: '2',
Visit_Date: "02/22/2000"
},
{
_id: 'v6',
type: 'visit',
Patient_ID: '2',
Visit_Date: "02/22/1987"
}
]);
}
function Load_Patients() {
$('#patient_list').empty();
db.createIndex({
index: {
fields: ['Patient_Name', 'type', 'Patient_Status']
}
}).then(function(result) {
db.find({
selector: {
Patient_Name: {
$gt: true
},
type: {
$eq: 'patient'
},
Patient_Status: {
$eq: 'Active'
}
},
sort: [{
"Patient_Name": "asc"
}]
}, function(error, response) {
console.log(response);
for (i in response.docs) {
var Static_Value = Get_Static_Value();
var Last_Visit = Get_Last_Visit(response.docs[i]._id);
$('#patient_list').append('<li>' + response.docs[i]._id + ' ' + response.docs[i].Patient_Name + ' [' + Static_Value + ']' + ' ' + Last_Visit + '</li>');
}
})
});
}
Add_Patients();
Add_Visits();
$('#button_load_patients').unbind().click(function(e) {
Load_Patients();
});
function Get_Static_Value() {
return 'I am static';
}
function Get_Last_Visit(Patient_ID) {
db.createIndex({
index: {
fields: ["Visit_Date", "type"]
}
}).then(function(result) {
db.find({
selector: {
Visit_Date: {
$gt: true
},
type: {
$eq: 'visit'
},
Patient_ID: {
$eq: Patient_ID
}
},
sort: [{
"Visit_Date": "desc"
}],
limit: 1
}).then(function(response) {
console.log(response);
if (response.docs.length > 0) {
Visit_Date = response.docs[0].Visit_Date;
} else {
Visit_Date = 'Never';
}
console.log(Visit_Date);
return Visit_Date;
});
})
}
Once I have returned the "Visit_Date" value then I would manipulate it using MomentJS and include or exclude it from the results that are appended to the div.
I would appreciate any advice on what I am doing wrong.

With the help of a colleague I got the solution to this problem and will post the solution as I am sure there are people out there who are as confused as me.
The things that needed to change in my code were that the call to the "Get_Last_Visit" function needed a .then to return the value of the promise. I also needed to add "let i" to make the "i" variable a global variable and available within the function:
for (let i in response.docs) {
var Static_Value = Get_Static_Value();
Get_Last_Visit(response.docs[i]._id)
.then(function(Last_Visit) {
$('#patient_list').append('<li>' + response.docs[i]._id + ' ' + response.docs[i].Patient_Name + ' [' + Static_Value + ']' + ' ' + Last_Visit + '</li>');
})
}
Here is an updated jsfiddle.
https://jsfiddle.net/movitico/9xorLham/

Recursive Function Causing Overflow

I am trying to write a recursive function in JavaScript. My function needs to search a tree of items. I have created a JSFiddle. When I run the JavaScript in Chrome, I get an error that says:
RangeError: Maximum call stack size exceeded
I assume this means that I'm not returning my value at the correct time. However, I continue to review the function and it looks correct to me. What am I doing wrong?
var sitemap = [
{
name: 'dashboards', children: [
{ name: 'dashboard 1', route: '/dashboards/dashboard1', html: '' }
]
},
{
name: 'objects', children: [
{ name: 'players', route: '/objects/players', html: '/objects/players.html' },
{ name: 'teams', route: '/objects/teams', html: '/objects/teams.html' },
{ name: 'coaches', route: '/objects/coaches', html: '/objects/coaches.html' },
{ name: 'cities', children: [
{ name: 'Chicago', route: '/cities/chicago',
html: '/objects/cities/chicago.html' },
{ name: 'Philadelphia', route: '/cities/philadelphia', html: '/objects/cities/philadelphia.html' }
]},
]
}
];
var getFromSitemap = function (path, entries) {
var sitemapItem = null;
if (entries) {
angular.forEach(sitemap, function (entry, key) {
if (entry.hasOwnProperty("children")) {
sitemapItem = getFromSitemap(path, entry.children);
} else if (entry.route === path) {
sitemapItem = entry;
}
});
}
return sitemapItem;
};
var getItem = function() {
var item = getFromSitemap('/cities/chicago', sitemap);
console.log(item);
}
Thank you!

You are calling foreach on the same object (sitemap) everytime:
angular.forEach(sitemap, function ...
It seems like you want to be calling it on entries recursively
angular.forEach(entries, function ....