Related
I wrote a simple iterator to loop trough a JSON object and parse the object into a new form.
This works as expected in almost any JS environment. (For example in the console(
However, the function below returns an empty array when executed in AppScript.
returns : [[], [], [], [], []]
This problem seems to be AppScript specific. Before I submit a bug through Google Developer Group I like to understand if this might be an App Script specific and intended behavior.
function parseBQData(/*tableData, request*/) {
var tableData = [["20220301","(none)","(direct)","3","1","1"],["20220301","organic","google","3","1","1"],["20220302","(none)","(direct)","4","2","2"],["20220302","organic","bing","1","1","1"],["20220303","(none)","(direct)","1","1","1"]]
try {
// store the array of dimensions and metrics in a variable called 'fields'
var fields = ["date", "medium", "source", "pageviews", "sessions", "users"]
// create a new empty array to store the parsed data
var newtableData = new Array();
// loop through each row in the tableData
for (var i = 0; i < tableData.length; i++) {
Logger.log(tableData[i]) // This returns: [20220301, (none), (direct), 3, 1, 1], [2022]
// create a new empty array to store the current row
var wrapper = new Array();
// loop through each column in the row
for (var j = 0; j < fields.length; j++) {
wrapper[fields[j]] = tableData[i][j]; /// <-is this not working?
Logger.log("Test Log:")
Logger.log("This is the updated field stored in the wrapper:"+wrapper[fields[j]]) // Returns : "20220301"
Logger.log("Lets check the wrapper if has the date/first value : " + wrapper.date ) // Returns "20220301"
// the wrapper does not dissapear but appears as empty when accessed as in root, the assignment abovew worked and the key value pair is accessible
Logger.log("Wrapper : " + JSON.stringify(wrapper)) // This returns always "Wrapper : []" Stringify is solely used to see that atleast something is returned and the wrapper is accesible
Logger.log("This is the current cell: "+fields[j]+" : "+tableData[i][j]) // This returns : "This is the current cell: date : 20220301" ... This is the current cell: medium : (none)
// So in conclusion All values and Arrays are accessible
// store the column data in the current row array, using the column header as the key
}
// add the current row to the new array of parsed data
newtableData.push(wrapper);
}
// return the new array of parsed data
Logger.log(newtableData) //This returns: "[[], [], [], [], []]""
return newtableData;
// if there is an error parsing the data, print the error to the log
} catch (e) {
Logger.log("Error parsing data")
Logger.log(e)
}
}
#Edit: Added some logging and comments
did not try in App Script but works fine with browser JS console :
var TableData = [["20220301","(none)","(direct)","3","1","1"],["20220301","organic","google","3","1","1"],["20220302","(none)","(direct)","4","2","2"],["20220302","organic","bing","1","1","1"],["20220303","(none)","(direct)","1","1","1"]]
function parseBQData(TableData, request) {
try {
var fields = ["date", "medium", "source", "pageviews", "sessions", "users"];
var newTableData = new Array();
for (var i = 0; i < TableData.length; i++) {
var wrapper = new Array();
for (var j = 0; j < fields.length; j++) {
wrapper[fields[j]] = TableData[i][j];
console.log("Wrapper : " + wrapper)
}
newTableData.push(wrapper);
}
return newTableData;
} catch (e) {
console.log("Error parsing data")
console.log(e)
}
}
parseBQData(TableData, 0);
Logs :
Wrapper :
(5) [Array(0), Array(0), Array(0), Array(0), Array(0)]
0: [date: '20220301', medium: '(none)', source: '(direct)', pageviews: '3', sessions: '1', …]
1: [date: '20220301', medium: 'organic', source: 'google', pageviews: '3', sessions: '1', …]
2: [date: '20220302', medium: '(none)', source: '(direct)', pageviews: '4', sessions: '2', …]
3: [date: '20220302', medium: 'organic', source: 'bing', pageviews: '1', sessions: '1', …]
4: [date: '20220303', medium: '(none)', source: '(direct)', pageviews: '1', sessions: '1', …]
length: 5
So my best guess, check Logger print args types, and maybe Logger.log("Wrapper : ", wrapper); ?
In App Script TableData is defined in the global scope. However the parameter TableData passed to parseBQData is in the function scope. If I try to run your script function parseBQData(TableData, request) including TableData in global scope I get TypeError: Cannot read property 'length' of undefined because TableData is undefined. You can fix this by simply:
function parseBQData(request) {
This is not a bug.
I also found this article JavaScript function parameter and scope
Same output but much simpler
function parseBQData() {
const iA = [["20220301", "(none)", "(direct)", "3", "1", "1"], ["20220301", "organic", "google", "3", "1", "1"], ["20220302", "(none)", "(direct)", "4", "2", "2"], ["20220302", "organic", "bing", "1", "1", "1"], ["20220303", "(none)", "(direct)", "1", "1", "1"]]
const fields = ["date", "medium", "source", "pageviews", "sessions", "users"];
iA.unshift(fields)
Logger.log(JSON.stringify(iA));
}
Execution log
10:25:24 AM Notice Execution started
10:25:25 AM Info [["date","medium","source","pageviews","sessions","users"],["20220301","(none)","(direct)","3","1","1"],["20220301","organic","google","3","1","1"],["20220302","(none)","(direct)","4","2","2"],["20220302","organic","bing","1","1","1"],["20220303","(none)","(direct)","1","1","1"]]
10:25:26 AM Notice Execution completed
I'm trying to create an advance search function to use on a multi-dimensional array of product objects. On the form, the user will have the option of narrowing down product results based on data in several different attributes (e.g. "'price' < 20 && 'country' == 'France'", "region == 'New York' && 'vintage' == '2016' && 'qty > 20"). The data, within variable data is formatted as follows:
0:
name: "Kedem Cream Red Concord"
appellation: "New York"
country: "United States"
price: "10.99"
primarygrape: "Concord"
qty: "1"
region: "New York"
regprice: "10.99"
sku: "230"
vintage: "NV"
1:
name: "Kendall Jackson Vintner's Chardonnay"
appellation: ""
country: "United States"
price: "14.99"
primarygrape: "Chardonnay"
qty: "35"
region: "California"
regprice: "18.99"
sku: "345"
vintage: "2016"
... continuing on over 10,000 product records
I figured I could create a string based on the user form input fairly easily that sums up what they're searching for. However, I need a safe way to evaluate that string within my function that returns the filtered results in resultsArray
I was able to get it to work with the eval() function, but I know that's not safe or efficient. I also tried fooling around with putting the conditions into an array and trying to incorporate that into the for loop, but I couldn't figure out how to make it work. The tricky part is, the conditions could be any number of things involving the following product attributes: appellation, country, price, primarygrape, qty, region, vintage. And different attributes use different operators (e.g. country, region, and others use '==' while qty and price use '>' or '<').
This code works, but uses the taboo eval() function:
var conditional = "data[i].price < 20 && data[i].country == 'France' && data[i].vintage == '2016'";
var resultArray = [];
console.log(data.length);
for (var i = 0; i < data.length; i++)
{
if (eval(conditional))
{
resultArray.push(data[i]);
}
}
}
This doesn't work, as it results in a Uncaught ReferenceError: data is not defined error:
var conditional = "return data[i].price < 20 && data[i].country == 'France' && data[i].vintage == '2016'";
var resultArray = [];
console.log(data.length);
for (var i = 0; i < data.length; i++)
{
if (new Function(conditional)())
{
resultArray.push(data[i]);
}
}
}
Any idea how to dynamically create that if conditional statement so that it recognizes the data variable? Or, is there a better solution?
Javascript Arrays have a function called filter which can be used to do things like you want to achieve.
For example you have an array with numbers:
var arr = [1,3,6,2,6]
And your plan is to filter all even numbers.You can use array.filter().
you need to pass an anonymous function into it. This function gets called once every entry in which you have access to the current element.
You need to return a boolean in this function - true means
that the element will be in the new, filtered array
Now the completed example to filter all even numbers:
arr.filter(function(element) {return element%2 == 0})
I hope that's enough for you to continue. :-) In the next snippet I made a small example with your data array
var your_data_array = [
{
name: "Kendall Jackson Vintner's Chardonnay",
appellation: "",
country: "United States",
price: "14.99",
primarygrape: "Chardonnay",
qty: "35",
region: "California",
regprice: "18.99",
sku: "345",
vintage: "2016" },
{
name: "Kendall Jackson Vintner's Chardonnay",
appellation: "",
country: "United States",
price: "14.99",
primarygrape: "Chardonnay",
qty: "35",
region: "California",
regprice: "18.99",
sku: "345",
vintage: "2016" },
{
name: "Kendall Jackson Vintner's Chardonnay",
appellation: "",
country: "United States",
price: "14.99",
primarygrape: "Chardonnay",
qty: "35",
region: "California",
regprice: "18.99",
sku: "345",
vintage: "2016" }
]
var filtered_array = your_data_array.filter(function(dataElement) {
return dataElement.country == "United States";
})
console.log(filtered_array)
If you are getting the conditions based on user input, you can format it how you like, it doesn't need to be a string. I would recommend an array that looks something like this:
var conditions = [
{
property: 'price',
comparator: '<',
value: 20
},
{
property: 'country',
comparator: '=',
value: 'France'
}
];
You could then use the filter method to generate your results array:
var resultArray = data.filter(function(item) {
for (var i = 0; i < conditions.length; i++) {
if (conditions[i].comparator === '=' &&
item[conditions[i].property] !== conditions[i].value) {
return false;
}
if (conditions[i].comparator === '>' &&
item[conditions[i].property] <= conditions[i].value) {
return false;
}
if (conditions[i].comparator === '<' &&
item[conditions[i].property] >= conditions[i].value) {
return false;
}
}
})
Below are my two arrays.
let clientCollection = ["1","ABC","X12","OE2","PQ$"];
let serverCollection = [{
"Id": "1",
"Name": "Ram",
"Other": "Other properties"
},
{
"Id": "ABC",
"Name": "Shyam",
"Other": "Other properties"
},
{
"Id": "OE2",
"Name": "Mohan",
"Other": "Other properties"
}]
Now I am in need to compare the above two collections & create two sub arrays
let matchedIds = [];
let unMatchedIds = [];
Now this is what I am doing currently.
for(let i =0 ; i < clientsCollection.length;i++)
{
if(_.indexOf(serverCollection, clientCollection[i]) >= 0)
{
matchedIds.push(clientCollection[i]);
}
else
{
unMatchedIds.push(clientCollection[i]);
}
}
In my application, the size of these arrays can increase to 1000 or more. This could be have efficieny issues
I am using underscore & tried if I can get some better solution but couldn't find yet.
Can someone please suggest if I can do the same in better efficient way using underscore + ES6??
I think, this would be a good way for matchedIds population:
for(let i = serverCollection.length - 1; i >= 0; i--) {
const id = serverCollection[i]['Id'];
if(clientCollection.indexOf(id) !== -1) {
matchedIds.push(id);
}
}
And this one is for unMatchedIds after the matchedIds is done:
for (var i = clientCollection.length - 1; i >= 0; i--) {
if (matchedIds.indexOf(clientCollection[i]) === -1) {
unMatchedIds.push(clientCollection[i]);
}
}
None of filter, reduce etc is faster than basic indexOf!
UPD
I created a plunker: https://plnkr.co/edit/UcOv6SquUgC7Szgfn8Wk?p=preview. He says that for 10000 items this solution is up to 5 times faster than other 2 solutions suggested here.
I would make a Set from all the server ids. Then just loop through and see if the id is in the Set and add it to the two arrays:
let serverCollection = [
{
Id: '1',
Name: 'Ram',
Other: 'Other properties'
},
{
Id: 'ABC',
Name: 'Shyam',
Other: 'Other properties'
},
{
Id: 'OE2',
Name: 'Mohan',
Other: 'Other properties'
}
];
let clientCollection = ['1', 'ABC', 'X12', 'OE2', 'PQ$'];
const serverIds = new Set(serverCollection.map((server) => server.Id));
let matchedIds = [];
let unmatchedIds = [];
for (let id of clientCollection) {
if (serverIds.has(id)) {
matchedIds.push(id);
} else {
unmatchedIds.push(id);
}
}
console.log('matched', matchedIds);
console.log('unmatched', unmatchedIds);
As the length of clientCollection and serverCollection increases, the cost of looping through each item becomes more and more apparent.
See a plunkr measuring performance
Create a Set of server ids. Use Array#reduce to iterate the clients, and assign the id to a sub array according to it's existence in the server's set. Extract the sub arrays to variables using destructuring assignment.
const clientCollection = ["1","ABC","X12","OE2","PQ$"];
const serverCollection = [{"Id":"1","Name":"Ram","Other":"Other properties"},{"Id":"ABC","Name":"Shyam","Other":"Other properties"},{"Id":"OE2","Name":"Mohan","Other":"Other properties"}];
// create a set of Ids. You can use underscore's pluck instead of map
const serverSet = new Set(serverCollection.map(({ Id }) => Id));
// reduce the ids to an array of two arrays (matchedIds, unMatchedIds), and then get assign to variables using destructuring assignment
const [matchedIds, unMatchedIds] = clientCollection.reduce((r, id) => {
r[serverSet.has(id) ? 0 : 1].push(id); // push to a sub array according to existence in the set
return r;
}, [[], []])
console.log(matchedIds);
console.log(unMatchedIds);
Please note that the following scenario is for the demonstration purposes only.
Lets assume I have a following array of object:
var obj = [{
id: 4345345345,
cat: [{
id: 1,
cat: "test1"
}, {
id: 2,
cat: "test2"
}]
}, {
id: 3453453421,
cat: [{
id: 1,
}, {
id: 2,
}]
}];
My goal is to :
Find an object within an array with #id 4345345345, add property selected : true to it
Then within this object with #id 4345345345, find cat with #id 2, add property
selected : true to it
The below works, however should my array have 1000+ objects it's feels somehow wasteful, can you please suggest any cleaner/clever solution ( possible using underscore)?
for (var i = 0; i < obj.length; i++) {
var parent = obj[i];
if (parent.id === 4345345345) {
parent.selected = true;
for (var j = 0; j < parent.cat.length; j++) {
var sub = parent.cat[j];
if(sub.id === 2) {
sub.selected = true;
}
};
}
};
Here are a few approaches I can think of
1) change your data structure to use the id's as the key. ie.
4345345345 :
cat: { 1 :{
cat: "test1"
}, 2 : {
cat: "test2"
}}
2) Alternatively you can create a temporary lookup table based on the id to directly look the object actual objects; obviously you would only create the look up table one time, or whenever the data changes. This will bring your runtime from O(n) to O(1).
I need to merge two objects in a code path that is going to be heavily used. The code works, but I am concerned it is not optimized enough for speed and I am looking for any suggestions to improve/replace what I have come up with. I originally started working off an example at the end of this issue: How can I merge properties of two JavaScript objects dynamically?. That solution works well for simple objects. However, my needs have a twist to it which is where the performance concerns come in. I need to be able to support arrays such that
an array of simple values will look for values in the new object and add those to the end of the existing object and
an array of objects will either merge objects (based off existence of an id property) or push new objects (objects whose id property does not exist) to the end of the existing array.
I do not need functions/method cloning and I don't care about hasOwnProperty since the objects go back to JSON strings after merging.
Any suggestions to help me pull every last once of performance from this would be greatly appreciated.
var utils = require("util");
function mergeObjs(def, obj) {
if (typeof obj == 'undefined') {
return def;
} else if (typeof def == 'undefined') {
return obj;
}
for (var i in obj) {
// if its an object
if (obj[i] != null && obj[i].constructor == Object)
{
def[i] = mergeObjs(def[i], obj[i]);
}
// if its an array, simple values need to be joined. Object values need to be remerged.
else if(obj[i] != null && utils.isArray(obj[i]) && obj[i].length > 0)
{
// test to see if the first element is an object or not so we know the type of array we're dealing with.
if(obj[i][0].constructor == Object)
{
var newobjs = [];
// create an index of all the existing object IDs for quick access. There is no way to know how many items will be in the arrays.
var objids = {}
for(var x= 0, l= def[i].length ; x < l; x++ )
{
objids[def[i][x].id] = x;
}
// now walk through the objects in the new array
// if the ID exists, then merge the objects.
// if the ID does not exist, push to the end of the def array
for(var x= 0, l= obj[i].length; x < l; x++)
{
var newobj = obj[i][x];
if(objids[newobj.id] !== undefined)
{
def[i][x] = mergeObjs(def[i][x],newobj);
}
else {
newobjs.push(newobj);
}
}
for(var x= 0, l = newobjs.length; x<l; x++) {
def[i].push(newobjs[x]);
}
}
else {
for(var x=0; x < obj[i].length; x++)
{
var idxObj = obj[i][x];
if(def[i].indexOf(idxObj) === -1) {
def[i].push(idxObj);
}
}
}
}
else
{
def[i] = obj[i];
}
}
return def;}
The object samples to merge:
var obj1 = {
"name" : "myname",
"status" : 0,
"profile": { "sex":"m", "isactive" : true},
"strarr":["one", "three"],
"objarray": [
{
"id": 1,
"email": "a1#me.com",
"isactive":true
},
{
"id": 2,
"email": "a2#me.com",
"isactive":false
}
]
};
var obj2 = {
"name" : "myname",
"status" : 1,
"newfield": 1,
"profile": { "isactive" : false, "city": "new York"},
"strarr":["two"],
"objarray": [
{
"id": 1,
"isactive":false
},
{
"id": 2,
"email": "a2modified#me.com"
},
{
"id": 3,
"email": "a3new#me.com",
"isactive" : true
}
]
};
Once merged, this console.log(mergeObjs(obj1, obj2)) should produce this:
{ name: 'myname',
status: 1,
profile: { sex: 'm', isactive: false, city: 'new York' },
strarr: [ 'one', 'three', 'two' ],
objarray:
[ { id: 1, email: 'a1#me.com', isactive: false },
{ id: 2, email: 'a2modified#me.com', isactive: false },
{ id: 3, email: 'a3new#me.com', isactive: true } ],
newfield: 1 }
I'd check out: https://github.com/bestiejs/lodash
_.merge is not on the list of 'optimized' functions, but this is a battle tested, battle hardened. He also has a performance suite, could ask how you might contribute to the perf suite to get some visibility into the merge implementation.
https://github.com/bestiejs/lodash/blob/master/lodash.js#L1677-1738
Edit: As an aside, I wouldn't prematurely optimize. I would see if this is actually a problem in your use case and then move on to actual data. I would look at something like: https://github.com/felixge/faster-than-c
Basic tenets:
Collect data
Analyze it
Find problems
Fix them
Repeat
He's got tips on each of those.
If you don't use Lo-Dash, and just want a tool to merge two objects including their arrays, use deepmerge: https://github.com/nrf110/deepmerge
npm install deepmerge