We Keep Coding

Why is my program only detecting integer tokens in NodeJS?

I decided to try and make a language tokenizer (don't even know if that's a real word) and made around 4 tokens that successfully tokenized a full program with line breaks and multiple spaces etc, but I just started from scratch and am running into a problem; I have two tokens currently, int and variableSet. The program being read has the content of 1 sv 1 2 as just a test, and the tokenizer returns an array of int, int, int, int with sv having a value of 1.
const code = `1 sv 1 2`
var validTokens = require("./tokens"); // just an object with the structure tokenName: RegExp object
function reverseTokenSearch(regex){
for (const [index, [key, value]] of Object.entries(Object.entries(validTokens))) {
if (value === regex){
return key;
}
}
return false;
}
function throughTokens (code,lastidx=0) {
for (const tokentype in validTokens){ // loop through all of the valid tokens
validTokens[tokentype].lastIndex = lastidx;
const searchresult = validTokens[tokentype]
const tokenresult = searchresult.exec(code.toString());
if (tokenresult) {
return [searchresult, tokenresult[0], tokenresult.index, lastidx+tokenresult[0].length+1, tokenresult.groups]
}
}
}
function resetIndexes (){
for (const tt in validTokens){
validTokens[tt].lastidx = 0;
}
}
resetIndexes();
var lst = 0
var tokens = []
var res = 1;
console.log("\ntokenizer; original input:\n"+code+"\n");
while (lst !== undefined && lst !== null){
if (lst > code.length){
console.error("Fatal error: tokenizer over-reached program length.")
process.exit(1)
}
const res = throughTokens(code,lst);
if(res){
console.log(res,lst)
const current = []
current[0] = reverseTokenSearch(res[0])
current[1] = res[1]
const currentidx = 2
for (const x in res[4]) {
current[currentidx] = x;
}
tokens.push(current)
lst = res[3]
} else {
lst = null
}
}
console.log(tokens)
// What outputs:
/*
tokenizer; original input:
1 sv 1 2
[ /\d+/g { lastidx: 0 }, '1', 0, 2, undefined ] 0
[ /\d+/g { lastidx: 0 }, '1', 5, 4, undefined ] 2
[ /\d+/g { lastidx: 0 }, '1', 5, 6, undefined ] 4
[ /\d+/g { lastidx: 0 }, '2', 7, 8, undefined ] 6
[ [ 'int', '1' ], [ 'int', '1' ], [ 'int', '1' ], [ 'int', '2' ] ]
*/
I think it's because of the order of the array but I have no idea where to start fixing it and would greatly appreciate a push in the right direction.
(edit): I tried removing the "g" flag on the RegExp object and all it did was broke the program into an infinite loop.

The problem is that you are silently assuming that every match found by the regex will start at lastidx which is not always the case. If you log tokenresult and lastidx before returning from throughTokens, you will see:
0
[ '1', index: 0, input: '1 sv 1 2', groups: undefined ]
2
[ '1', index: 5, input: '1 sv 1 2', groups: undefined ]
4
[ '1', index: 5, input: '1 sv 1 2', groups: undefined ]
6
[ '2', index: 7, input: '1 sv 1 2', groups: undefined ]
In the second iteration, the match is at index 5, but you assume it to be at index 2, which it is not (whereby you also incorrectly increment lastidx to 4). You also at the end of throughTokens assume that every match is followed by a space, which is also incorrect for the last token.
Simplest way to fix this code is to replace
//if (tokenresult) { // replace in throughTokens with below
if (tokenresult && tokenresult.index === lastidx) {
to be sure that you're matching at the right place and then in the main loop
//while (lst !== undefined && lst !== null){ // replace with below
while (lst !== undefined && lst !== null && lst < code.length){
to handle the end of the input correctly.
With these changes, the printouts that we added earlier will be
0
[ '1', index: 0, input: '1 sv 1 2', groups: undefined ]
2
[ 'sv', index: 2, input: '1 sv 1 2', groups: undefined ]
5
[ '1', index: 5, input: '1 sv 1 2', groups: undefined ]
7
[ '2', index: 7, input: '1 sv 1 2', groups: undefined ]
which is correct and the output would be
[
[ 'int', '1' ],
[ 'variableSet', 'sv' ],
[ 'int', '1' ],
[ 'int', '2' ]
]
Recommendations
There are a lot of other logical and programmatical problems with this code which I will not go into but my advice is to go through every piece of the code and understand what it does and whether it could be done in a simpler way.
On a general level instead of returning an array with data [d1, d2, d3, ...] return an object with named properties { result: d1, index: d2, ... }. Then it is much easier for someone else to understand your code. Also go through naming of methods.
As far as this approach is concerned, if you know that there will be a space after each token, then extract only the current token and send to throughToken. Then you can make that function both more efficient and robust against errors.

Mongodb: Get sample of nested array using Aggreagate

$slice allows me to get a slice of nested array. I used it successfully like this:
const user = await User.aggregate([
{ $match: { _id: ObjectId(user_id) } },
{
$lookup: {
from: "users",
let: { friends: "$friends" },
pipeline: [
{ $match: { $expr: { $in: ["$_id", "$$friends"] } } },
{
$lookup: {
from: "profiles",
localField: "profile",
foreignField: "_id",
as: "profile",
},
},
{
$match: {
"profile.online": true,
},
},
{
$project: {
name: "$name",
surname: "$surname",
profile: { $arrayElemAt: ["$profile", 0] },
},
},
],
as: "friends",
},
},
{
$addFields: {
friends: {
$slice: ["$friends", skip, limit],
},
},
},
]);
Now, instead of taking a slice, I would like to take a random sample of the array field friends.
I could not find a way to do this. But, in the group stage I can use something like this:
const pipeline = [
{
$lookup: {
from: "profiles",
let: { profiles_id: "$profile" },
pipeline: [
{
$match: {
online: true,
$expr: { $eq: ["$_id", "$$profiles_id"] },
},
},
],
as: "profile",
},
},
{ $unwind: "$profile" },
{ $sample: { size: 10 } },
];
const users = await User.aggregate(pipeline);

Change the last $addFields stage to this.
Pros: It "works."
Cons: You are not guaranteed unique random entries in the list. To get that is a lot more work. If you have a LOT more friends than the range then you are probably OK.
,{$addFields: {friends: {$reduce: { // overwrite friends array...
// $range is the number of things you want to pick:
input: {$range:[0,4]},
initialValue: [],
in: {
$let: {
// qq will be a random # between 0 and size-1 thanks to mult
// and floor, so we do not have to do qq-1 to get to zero-based
// indexing on the $friends array
vars: {qq: {$floor:{$multiply:[{$rand: {}},{$size:"$friends"}]}} },
// $concat only works for strings, but $concatArrays can be used
// (creatively) on other types. Here $slice returns an array of
// 1 item which we easily pass to $concatArrays to build the
// the overall result:
in: {$concatArrays: [ "$$value", {$slice:["$friends","$$qq",1]} ]}
}}
}}
UPDATED
This version exploits keeping state in the $reduce chain and will not pick dupes. It does so by iteratively shrinking the input candidate list of items as each item is randomly chosen. The output is a little nested (i.e. friends is not set to picked random sample but rather to an object containing picks and the remnant aa list) but this is something easily reformatted after the fact. In MongoDB 5.0 we could finish it off with:
{$addFields: {friends: {$getField: {field: "$friends.picks", input: {$reduce: {
but many people are not yet on 5.0.
{$addFields: {friends: {$reduce: {
// $range is the number of things you want to pick:
input: {$range:[0,6]},
// This is classic use of $reduce to iterate over something AND
// preserve state. We start with picks as empty and aa being the
// original friends array:
initialValue: {aa: "$friends", picks: []},
in: {
$let: {
// idx will be a random # between 0 and size-1 thanks to mult
// and floor, so we do not have to do idx-1 to get to zero-based
// indexing on the $friends array. idx and sz will be eval'd
// each time reduce turns the crank through the input range:
vars: {idx: {$floor:{$multiply:[{$rand: {}},{$size:"$$value.aa"}]}},
// cannot set sz and then use it in same vars; oh well
sz: {$size:"$$value.aa"}
},
in: {
// Add to our picks list:
picks: {$concatArrays: [ "$$value.picks", {$slice:["$$value.aa","$$idx",1]} ]},
// And now shrink up the input candidate array.
// Sadly, we cannot do $slice:[array,pos,0] to yield an empty
// array and keep the $concat logic tight; thus we have to test
// for front and end special conditions.
// This whole bit is to extract the chosen item from the aa
// array by splicing together a new one MINUS the target.
// This will change the value of $sz (-1) as we crank thru
// the picks. This ensures we only pick UNPICKED items from
// $$value.aa!
aa: {$cond: [{$eq:["$$idx",0]}, // if
// idx 0: Take from idx 1 and count size - 1:
{$slice:["$$value.aa",1,{$subtract:["$$sz",1]}]}, // then
// idx last: Take from idx 0 and ALSO count size - 1:
{$cond: [ // else
{$eq:["$$idx",{$subtract:["$$sz",1]}]}, // if
{$slice:["$$value.aa",0,{$subtract:["$$sz",1]}]}, // then
// else not 0 or last item, i.e. idx = 3
{$concatArrays: [
// Start at 0, count idx; this will land
// us BEFORE the target item (because idx
// is n-1:
{$slice:["$$value.aa",0,"$$idx"]},
// Jump over the target (+1), and go n-2
// (1 for idx/n conversion, and 1 for the
// fact we jumped over:
{$slice:["$$value.aa",{$add:["$$idx",1]},{$subtract:["$$sz",2]}]}
]}
]}
]}
}
}}
}}
}}
]);
Starting in MongoDB v4.4 (Jan 2021), you may opt to use the $function operator. The splice function in javascript does all the work of the multiple $slice operations in the previous example.
{$addFields: {friends: {$function: {
body: function(candidates, npicks) {
var picks = []
for(var i = 0; i < npicks; i++) {
var idx = Math.floor(Math.random() * candidates.length);
picks.push(candidates.splice(idx,1)[0]);
}
return picks;
},
args: [ "$friends", 4], // 4 is num to pick
lang: "js"
}}

Can someone explain me how to use paraments correctly in this example?

I'm trying JS. I have used only Python recently.
And there is a problem, look at this example:
var myCharacter = {
name: "character",
avatarImage: "avatar.png",
experience: 1,
level: 1,
items: {helmetSlot: blank, armorSlot: blank}, //'items' will be changed into 'inventory'
backpack: [],
strength: 0,
//It is corrent 'method'?
takeOff: function(slot) {
if (this.backpack.length < 10) { //setting backpack length
this.backpack.push(this.items.slot)
this.items.slot = blank
}
else {}
}
}
myCharacter.takeOff(helmetSlot) //why it doesn't work? I know it did't declared but ... You know I want to use 'word' helmetSlot to make block 'takeOff' way I want, I have used to do things like this in python
Problem is in method takeOff. Can you explain me how to use it like I can do it in Python?
Here is full code: https://pastebin.com/NP1KLPie
I know I have done it wrong, but how to use it the way I want?

this.items.slot is looking for a property literally named "slot". Instead you can use square brackets to use a variable property name: this.items[slot] (passing a string value into slot.)
var myCharacter = {
name: "character",
avatarImage: "avatar.png",
experience: 1,
level: 1,
items: { helmetSlot: "helmet", armorSlot: "armor" }, // using strings instead of the `blank` variable, just so the results will be visible
backpack: [],
strength: 0,
takeOff: function(slot) {
if (this.backpack.length < 10) {
this.backpack.push(this.items[slot]) // not this.items.slot
this.items[slot] = "empty" // string instead of `blank` again
} else {}
}
}
myCharacter.takeOff("helmetSlot") // pass a string, not a bare variable name
console.log(myCharacter) // check the results

in your code values you are using for helmetSlot and armorSlot is not valid .Because in javascript you can can assign one of the below values to variablesor properties of objects
a) String ('' or 'somevalue')
b Number
c) undefined
d) null
e) {}
f) function (){
....
}
var myCharacter = {
name: "character",
avatarImage: "avatar.png",
experience: 1,
level: 1,
items: {helmetSlot:'', armorSlot: ''}, //'items' will be changed into 'inventory'
backpack: [],
strength: 0,
//It is corrent 'method'?
takeOff: function(slot) {
if (this.backpack.length < 10) { //setting backpack length
this.backpack.push(this.items.slot)
this.items.slot = undefined;
}
else {}
}
};
console.log(myCharacter);
myCharacter.takeOff(10);

Sorting array of object inside reduce in JavaScript

I am trying to do sort inside the reduce and I thought I have everything correct but still my result is not sorted as desired.
Here is the code snippet I have:
var studentInfo = [
{
studentId: 1,
addresses: [
{street: '123 Main St'},
]
},
{
studentId: 2,
addresses: [
{street: '456 Front St'}
]
},
{
studentId: 3,
addresses: [
{street: '100 MLK St'}
]
}
];
function appendAddress(studentId, newAddress) {
return studentInfo.reduce(function (info, student) {
if (student.studentId === studentId) {
student = {
studentId: student.studentId,
addresses: student.addresses.concat(newAddress).sort(function (address1, address2) {
return address2.street - address1.stree;
})
};
}
info.push(student);
return info;
}, []);
}
var newAddress = {
street: '166 Devil St'
}
console.log('Result: ' + JSON.stringify(appendAddress(2, newAddress)));
I am getting result as
Result: [{"studentId":1,"addresses":[{"street":"123 Main St"}]},{"studentId":2,"addresses":[{"street":"456 Front St"},{"street":"166 Devil St"}]},{"studentId":3,"addresses":[{"street":"100 MLK St"}]}]
instead of
Result: [{"studentId":1,"addresses":[{"street":"123 Main St"}]},{"studentId":2,"addresses":[{"street":"166 Devil St"},{"street":"456 Front St"}]},{"studentId":3,"addresses":[{"street":"100 MLK St"}]}]
Am I missing anything?

As to the sorting issue, if that was the main thing you were wondering about, you do indeed have a typo as the comment above noted, and also, performing subtraction on non-numeric strings won't get you very far. I used .localeCompare in the solution above.
If you wanted to copy the objects as you were appending, that can still be done more simply, but I don't know if that's what you actually want.
var studentInfo = [
{studentId: 1,addresses: [{street: '123 Main St'}]},
{studentId: 2,addresses: [{street: '456 Front St'}]},
{studentId: 3,addresses: [{street: '100 MLK St'}]}
];
console.log(addAddress(2, {street: "1234 56th Ave"}));
function addAddress(studentId, address) {
const idx = studentInfo.findIndex(o => o.studentId === studentId);
if (idx !== -1) {
return [...studentInfo.slice(0, idx), {
studentId,
addresses: [...studentInfo[idx].addresses, address].sort((a,b) => a.street.localeCompare(b.street))
}, ...studentInfo.slice(idx+1)];
} else {
return [...studentInfo, {studentId, addresses:[address]}];
}
}
But now you're having two different copies of the data with some shared objects.

Trying to get a variable to equal an object from a set of objects...i think?

OK so the way i worded this question before must have been too convoluted so here is a simplified version:
I have a variable which equals:
{
txid: 'f0315ffc38709d70ad5647e22048358dd3745f3ce3874223c80a7c92fab0c8ba',
version: 1,
locktime: 0,
vin:
[ { coinbase: '0420e7494d017f062f503253482f',
sequence: 4294967295 } ],
vout: [ { value: 50, n: 0, scriptPubKey: [Object] } ],
blockhash: '00000000b873e79784647a6c82962c70d228557d24a747ea4d1b8bbe878e1206',
confirmations: 39288,
time: 1296688928,
blocktime: 1296688928 }
I want to make another variable equal just:
vout: [ { value: 50, n: 0, scriptPubKey: [Object] } ],
How do i do this?
When i try:
varible2 = variable1.vout
I get back an error saying ".vout" is undefined

var avariable = {/* your object [literal] here */};
var anothervariable = avariable.vout; // or avariable["vout"]
// or did you want the following?
var anothervariable = {
vout: avariable.vout
};