We Keep Coding

Why is my program only detecting integer tokens in NodeJS?

I decided to try and make a language tokenizer (don't even know if that's a real word) and made around 4 tokens that successfully tokenized a full program with line breaks and multiple spaces etc, but I just started from scratch and am running into a problem; I have two tokens currently, int and variableSet. The program being read has the content of 1 sv 1 2 as just a test, and the tokenizer returns an array of int, int, int, int with sv having a value of 1.
const code = `1 sv 1 2`
var validTokens = require("./tokens"); // just an object with the structure tokenName: RegExp object
function reverseTokenSearch(regex){
for (const [index, [key, value]] of Object.entries(Object.entries(validTokens))) {
if (value === regex){
return key;
}
}
return false;
}
function throughTokens (code,lastidx=0) {
for (const tokentype in validTokens){ // loop through all of the valid tokens
validTokens[tokentype].lastIndex = lastidx;
const searchresult = validTokens[tokentype]
const tokenresult = searchresult.exec(code.toString());
if (tokenresult) {
return [searchresult, tokenresult[0], tokenresult.index, lastidx+tokenresult[0].length+1, tokenresult.groups]
}
}
}
function resetIndexes (){
for (const tt in validTokens){
validTokens[tt].lastidx = 0;
}
}
resetIndexes();
var lst = 0
var tokens = []
var res = 1;
console.log("\ntokenizer; original input:\n"+code+"\n");
while (lst !== undefined && lst !== null){
if (lst > code.length){
console.error("Fatal error: tokenizer over-reached program length.")
process.exit(1)
}
const res = throughTokens(code,lst);
if(res){
console.log(res,lst)
const current = []
current[0] = reverseTokenSearch(res[0])
current[1] = res[1]
const currentidx = 2
for (const x in res[4]) {
current[currentidx] = x;
}
tokens.push(current)
lst = res[3]
} else {
lst = null
}
}
console.log(tokens)
// What outputs:
/*
tokenizer; original input:
1 sv 1 2
[ /\d+/g { lastidx: 0 }, '1', 0, 2, undefined ] 0
[ /\d+/g { lastidx: 0 }, '1', 5, 4, undefined ] 2
[ /\d+/g { lastidx: 0 }, '1', 5, 6, undefined ] 4
[ /\d+/g { lastidx: 0 }, '2', 7, 8, undefined ] 6
[ [ 'int', '1' ], [ 'int', '1' ], [ 'int', '1' ], [ 'int', '2' ] ]
*/
I think it's because of the order of the array but I have no idea where to start fixing it and would greatly appreciate a push in the right direction.
(edit): I tried removing the "g" flag on the RegExp object and all it did was broke the program into an infinite loop.

The problem is that you are silently assuming that every match found by the regex will start at lastidx which is not always the case. If you log tokenresult and lastidx before returning from throughTokens, you will see:
0
[ '1', index: 0, input: '1 sv 1 2', groups: undefined ]
2
[ '1', index: 5, input: '1 sv 1 2', groups: undefined ]
4
[ '1', index: 5, input: '1 sv 1 2', groups: undefined ]
6
[ '2', index: 7, input: '1 sv 1 2', groups: undefined ]
In the second iteration, the match is at index 5, but you assume it to be at index 2, which it is not (whereby you also incorrectly increment lastidx to 4). You also at the end of throughTokens assume that every match is followed by a space, which is also incorrect for the last token.
Simplest way to fix this code is to replace
//if (tokenresult) { // replace in throughTokens with below
if (tokenresult && tokenresult.index === lastidx) {
to be sure that you're matching at the right place and then in the main loop
//while (lst !== undefined && lst !== null){ // replace with below
while (lst !== undefined && lst !== null && lst < code.length){
to handle the end of the input correctly.
With these changes, the printouts that we added earlier will be
0
[ '1', index: 0, input: '1 sv 1 2', groups: undefined ]
2
[ 'sv', index: 2, input: '1 sv 1 2', groups: undefined ]
5
[ '1', index: 5, input: '1 sv 1 2', groups: undefined ]
7
[ '2', index: 7, input: '1 sv 1 2', groups: undefined ]
which is correct and the output would be
[
[ 'int', '1' ],
[ 'variableSet', 'sv' ],
[ 'int', '1' ],
[ 'int', '2' ]
]
Recommendations
There are a lot of other logical and programmatical problems with this code which I will not go into but my advice is to go through every piece of the code and understand what it does and whether it could be done in a simpler way.
On a general level instead of returning an array with data [d1, d2, d3, ...] return an object with named properties { result: d1, index: d2, ... }. Then it is much easier for someone else to understand your code. Also go through naming of methods.
As far as this approach is concerned, if you know that there will be a space after each token, then extract only the current token and send to throughToken. Then you can make that function both more efficient and robust against errors.

Calendar date ordering: Occupation overview

I'm working on a calendar plugin that would show my occupation overview ordered as following:
Now I've got the event on the calendar with moment.js, so they are formatted like this (sorted on Start date):
let events = [{
name: 'Event A',
start: '01-11-2021 00:00',
end: '08-11-2021 00:00',
},{
name: 'Event C',
start: '03-11-2021 00:00',
end: '06-11-2021 00:00',
},{
name: 'Event E',
start: '05-11-2021 00:00',
end: '08-11-2021 00:00',
},{
name: 'Event D',
start: '07-11-2021 00:00',
end: '12-11-2021 00:00',
},{
name: 'Event B',
start: '10-11-2021 00:00',
end: '17-11-2021 00:00',
},]
Expected occupationOverview array would be something like:
let ooArray = [
{ // Longest/bottom bar
start: '01-11-2021 00:00',
end: '17-11-2021 00:00',
group: 1
},
{ // Middle bar 1
start: '03-11-2021 00:00',
end: '08-11-2021 00:00',
group: 2
},
{ // Middle bar 2
start: '10-11-2021 00:00',
end: '12-11-2021 00:00',
group: 2
},
{ // Top bar 1
start: '05-11-2021 00:00',
end: '06-11-2021 00:00',
group: 3
},
{ // Top bar 2
start: '07-11-2021 00:00',
end: '08-11-2021 00:00',
group: 3
},]
I have honestly no clue how to group the calendar events so they give back an array with the start and end times as resulted in the red box.
Anybody that can help me figure this out?
Thanks!

Based on the updated question, sort all the dates keeping the attribute for start and end. Process them in order so that the first date (that must be a start) starts level 1, which is single booked.
If the next date is an end, that ends the bar. However, if the next date is a start, that increases the level (i.e. double booked). The following is an implementation, you might want to sort the bars by level or start date.
The function below firstly gets all the dates sorted in ascending order with their type - start or end. It then processes each date - start dates create a new bar, end dates end the most recent bar. When ended, the last bar is popped off starts and added to bars, which is an array of finished bars.
This depends on the source data being valid, i.e. it must start with a start and end with an end, they must be in the right order and of equal number.
One enhancement would be to ensure that where a start and end have the same date, the start is always sorted before the end so zero length events (milestones?) don't get ordered end-start, which would cause the level to be decremented before it's incremented. There may be other issues with starts and ends that have the same date and time, please test.
// Parse date in D-M-Y H:m format
function parseDMY(s) {
let [D, M, Y, H, m] = s.split(/\D/);
return new Date(Y, M - 1, D, H, m);
}
// Format as DD-MM-YYYY HH:mm
function formatDate(d) {
let z = n => ('0'+n).slice(-2);
return `${z(d.getDate())}-${z(d.getMonth()+1)}-${d.getFullYear()} ` +
`${z(d.getHours())}:${z(d.getMinutes())}`;
}
// Generates "occupation" bars
function calcBookingLevels(events) {
// Get sorted array of [{type, Date}]
let dates = events.reduce( (dates, event) => {
dates.push({type: 'start', date: parseDMY(event.start)},
{type: 'end', date: parseDMY(event.end)});
return dates;
}, []).sort((a, b) => a.date - b.date);
// Process dates to get occupation bars with levels
let bars = [];
let starts = [];
let level = 0;
dates.forEach(date => {
// If it's a start, start a new bar
if (date.type == 'start') {
let bar = {level: ++level, start: formatDate(date.date)};
starts.push(bar);
// Otherwise it's an end, close the most recent bar and
// move to bars array
} else {
let t = starts.pop();
t.end = formatDate(date.date);
--level;
bars.push(t);
}
})
return bars;
}
// Sample data
let events = [{
name: 'Event A',
start: '01-11-2021 00:00',
end: '08-11-2021 00:00',
},{
name: 'Event C',
start: '03-11-2021 00:00',
end: '06-11-2021 00:00',
},{
name: 'Event E',
start: '05-11-2021 00:00',
end: '08-11-2021 00:00',
},{
name: 'Event D',
start: '07-11-2021 00:00',
end: '12-11-2021 00:00',
},{
name: 'Event B',
start: '10-11-2021 00:00',
end: '17-11-2021 00:00',
},];
// Run it...
console.log(calcBookingLevels(events));
If the dates were in YYYY-MM-DD HH:mm format then they could be sorted as strings without converting to Dates.

Looks like you need to get the diff in milliseconds, and sort from greatest to least. You'll have to play with this to see which way it sorts (might need to flip the 'a' and 'b' on the diff statement, as I didn't run it against your array)
const sortedArray = array.sort((a, b) => a.diff(b))

With the help of #RobG I figured it out. Here is my answer for interested people:
// Not needed for answer, but result gave warnings so yeah
moment.suppressDeprecationWarnings = true;
let newItems = []
let dateArray = []
let events = [{
name: 'Event A',
start: '01-11-2021 00:00',
end: '08-11-2021 00:00',
},{
name: 'Event C',
start: '03-11-2021 00:00',
end: '06-11-2021 00:00',
},{
name: 'Event E',
start: '05-11-2021 00:00',
end: '08-11-2021 00:00',
},{
name: 'Event D',
start: '07-11-2021 00:00',
end: '12-11-2021 00:00',
},{
name: 'Event B',
start: '10-11-2021 00:00',
end: '17-11-2021 00:00',
},]
// Make one big array with all start and end dates
events.forEach(event=> {
dateArray.push({
type: 'start',
date: event.start
}, {
type: 'end',
date: event.end
})
})
// Sort the dates from first to last
dateArray = dateArray.sort((left, right) => {
return moment(left.date).diff(moment(right.date))
})
let groupID = -1
// Loop through the array with all dates and add them to a group
// based on if the current date is a start or end
for (let i = 0; i < dateArray.length; i++) {
if (dateArray[i].type === 'start') groupID++
else groupID--
for (let ii = 0; ii <= groupID; ii++) {
if (dateArray[i + 1] &&
!moment(dateArray[i].date).isSame(dateArray[i + 1].date)) {
newItems.push({
group: 1,
start: dateArray[i].date,
end: dateArray[i + 1].date,
subgroup: 'sg_' + ii,
subgroupOrder: ii
})
}
}
}
console.log(newItems)
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.1/moment.min.js"></script>

Vanilla Javascript merge JSON array objects into nested array if match a common value

I was unable to find a solution that is identical with the issue I am trying to solve. If is a duplicate kindly provide a link to a solution.
Using vanilla Javascript, merge array of objects that has a common value. I have a JSON file and it will create the following javascript object (I am unable to change the original structure). Notice that each object will have different nested name and nested value. However the common value is the name[0]
var data = [
{
name: [
'Data 1', // common value
'1 Jan, 2019', // same value therefore not overwrite/merge
'hotfix dec'
],
value: [
'hotfix1.fsfix',
'hotfix1.txt'
]
},
{
name: [
'Data 1', // common value
'1 Jan, 2019' // same value therefore not overwrite/merge
],
value: 'data1.jar'
},
{
name: [
'Data 2',
'1 Feb, 2019'
],
value: 'data2.fsfix'
},
{
name: [
'Data 2',
'1 Feb, 2019'
],
value: 'data2.jar'
},
{
name: [
'Data 3',
'1 Mar, 2018'
],
value: 'data3.fsfix'
}
]
The desire output will be merging the nested object that has the same name[0].
var data = [
{
name: [
'Data 1', // common value
'1 Jan, 2019', // same value therefore not overwrite/merge
'hotfix dec'
],
value: [
'data1.fsfix',
'data1.txt',
'data1.jar' // This was added after the merge
]
},
{
name: [
'Data 2',
'1 Feb, 2019'
],
value: [
'data2.fsfix',
'data2.jar' // This was added after the merge
]
},
{
name: [
'Data 3',
'1 Mar, 2018'
],
value: 'data3.fsfix'
}
]
Using this new merged structure, I would create a function to loop each array set. Thanks in advance

You could key the data by the first name entry using a Map. Then populate the data into the value property within the corresponding Map value, also collect all the extra values in the name arrays (beyond the first two entries), and finally extract the Map values.
This works with linear time complexity:
var data = [{name: ['Data 1', '1 Jan, 2019', 'hotfix dec'],value: ['hotfix1.fsfix','hotfix1.txt']},{name: ['Data 1', '1 Jan, 2019'],value: 'data1.jar'},{name: ['Data 2','1 Feb, 2019'],value: 'data2.fsfix'},{name: ['Data 2','1 Feb, 2019'],value: 'data2.jar'},{name: ['Data 3','1 Mar, 2018'],value: 'data3.fsfix'}];
const map = new Map(data.map(o =>
[o.name[0], { name: o.name.slice(0,2), value: [] }]));
data.forEach(o => map.get(o.name[0]).value.push(...[].concat(o.value)));
data.forEach(o => map.get(o.name[0]).name.push(...o.name.slice(2)));
const result = Array.from(map.values(), o => o.value.length === 1
? { ...o, value: o.value[0] } : o);
console.log(result);
The callback function that is passed to Array.from is a map-function. It is only needed to turn value arrays with only one string into that string only. If this is not required, you can omit that callback and just do Array.from(map.values()).

With some Array methods, I think you can make this work (untested code below, but I think it should do what you want):
// Merge matching entries by pushing values from matching objects to each other
data = data.map(entry => {
let matchingObjects = data.filter(match => {
return match.name[0] === entry.name[0];
});
matchingObjects.forEach(match => {
if (match !== entry) {
var flatValue = [entry.value].flat();
entry.value = flatValue.push.apply(flatValue, [match.value].flat());
}
});
});
// Remove duplicates by filtering out all but the first entry of each name[0]
data = data.filter((entry, index) => {
return index === data.findIndex(match => {
return match.name[0] === entry.name[0];
});
});

You can also use reduce and map to create your desired outcome. I think it is quite powerful combination in general.
const data = [{name: ['Data 1', '1 Jan, 2019', 'hotfix dec'],value: ['hotfix1.fsfix','hotfix1.txt']},{name: ['Data 1', '1 Jan, 2019'],value: 'data1.jar'},{name: ['Data 2','1 Feb, 2019'],value: 'data2.fsfix'},{name: ['Data 2','1 Feb, 2019'],value: 'data2.jar'},{name: ['Data 3','1 Mar, 2018'],value: 'data3.fsfix'}];
const dataMap = data.reduce((acc, curr)=>{
id = curr.name[0];
acc[id] = acc[id] || {
name: [],
value: []
};
const value = Array.isArray(curr.value) ? curr.value : [curr.value]
acc[id].name = [...acc[id].name, ...curr.name.filter((i)=>acc[id].name.indexOf(i)===-1)]
acc[id].value = [...acc[id].value, ...value.filter((i)=>acc[id].value.indexOf(i)===-1)]
return acc
},{})
const result = Object.keys(dataMap).map(key=> {
const d = dataMap[key];
d.value = d.value.length===1 ? d.value[0] : d.value
return d;
})

Loop and Combine Time Ranges in an Array

I am trying to read through an array of objects (that are start/end times) and combine two (or more) of those times if they are back to back.
ie. the first objects end time is the same as the next objects start time. If they are, combine them. Then check the newly combined objects end time with the next object in the array
Here is a simplified array of times:
var times = [
{
start: 1,
end: 2
},{
start: 2,
end: 3
},{
start: 4,
end: 5
},{
start: 6,
end: 7
},
]
I would like that (or have a diff Array) to output like the below:
var newTimes = [
{
start: 1,
end: 3
},{
start: 4,
end: 5
},{
start: 6,
end: 7
},
]
It gets trickier if there are 3 times in a row.
var threeTime = [
{
start: 1,
end: 2
},{
start: 2,
end: 3
},{
start: 3,
end: 5
},{
start: 6,
end: 7
},
]
The above should turn into:
var newThreeTimes = [
{
start: 1,
end: 5
},{
start: 6,
end: 7
},
]
The original array of times will always be sorted from oldest (smallest start time) to newest (largest start time). The output does not need to be in any specific order. All time objects will be moment times.
Can someone help me solve this?

This is the code i have come up with
function mergeArr(arr) {
// Sort the array in descending order
arr.sort(function(a, b) {
return b.start - a.start;
});
// Traverse from the top as you will need to remove the elements
// Merge the elements based on start of one and end of the previous
for (var i = arr.length - 1; i > 0; i--) {
if (arr[i].end == arr[i - 1].start) {
arr[i].end = arr[i - 1].end;
arr.splice(i - 1, 1);
}
}
// Sort it again in reverse order.
return arr.sort(function(a, b) {
return a.start - b.start;
});
}
Comments are making the code self explanatory.
Working Fiddle

Javascript: Creating a Summary Object from a Series of Detail Objects

I'm iterating over a bunch of objects that look like this:
var tests = [{ date: 1379082017487,
summary:
{'Item A': 2069,
'Item B': 8987,
'Item C': 1890
}},
{date: 1379082015968,
summary:
{'Item A': 1824,
'Item B': 6758,
'Item C': 3857
}}];
I need to combine the data into a single object with the same keys, and arrays containing the combined values:
{ date: [1379082017487, 1379082015968],
summary:
{ 'Item A': [2069, 1824],
'Item B': [8987, 6758],
'Item C': [1890, 3857]
}
}
My strategy thus far is to create a nested loop; iterate over the top-level objects and add their values to a 'date' array. Then iterate over the 'summary' object, pushing those values to an array representing each key. In the end, I'd add the arrays back together to form my single output object.
My brain freeze is on the inner array. It can have a varying number of keys ('Items'). While these keys will have the same names for all objects in any given run of the code, they can have different names between runs. The inner loop isn't producing an array (like [2069, 1824]).
So far I have:
var testDates = [];
var itemVals = [];
// iterate over the outer objects
for (var record = 0; record < tests.length; record++) {
testDates.push(tests[record].date); // this works fine
// iterate over the inner 'summary' object:
for (var itemName in tests[record]['summary']) {
// doesn't produce the appropriate array
itemVals.push([tests[record]['summary'][itemName]]);
}
}
Thanks for any advice!

I have written a sample code for your problem. Please try this and let me know if it work
var tests = [{
date: 1379082017487,
summary: {
'Item A': 2069,
'Item B': 8987,
'Item C': 1890
}
}, {
date: 1379082015968,
summary: {
'Item A': 1824,
'Item B': 6758,
'Item C': 3857
}
}];
var json = {
date: [],
summary: {}
};
for (var i = 0; i < tests.length; i++) {
var dateJSON = tests[i];
json.date.push(dateJSON.date);
for (var key in dateJSON.summary) {
if (dateJSON.summary.hasOwnProperty(key)) {
if (json.summary[key] == null) {
json.summary[key] = [];
}
json.summary[key].push(dateJSON.summary[key]);
}
}
}