I have a long list of chat rooms
let chatRooms = {
"general": ChatRoom,
"myRoomA": ChatRoom,
"bobsRoom": ChatRoom,
...
}
ChatRoom has a serialize method
ChatRoom.serialize = function(){
return {
name: this.name,
clients: this.clients,
...
}
}
In order to list all ChatRooms to a user, I must send this data to them
ChatRoomManager.serialize = function(){
let serializedObjects = [];
Util.each(this.chatRooms, function(i, e){
if(e.serialize){
serializedObjects.push(e.serialize());
}
});
return serializedObjects;
}
This becomes a performance issue as people regularly request to list all chat rooms and it gets serialized so often so I want to do paging. But if an object has no guaranteed order, how can I possibly say "here are the next 10 chat rooms"? Even if I could guarantee order, how could I start at index 11 without looping through all of the objects? Imagine if I was at index 1000, etc..
TLDR: is it possible to do paging with an object of objects efficiently and accurately.
You coulf just take the values of the objects which returns an array, so the order is guaranteed:
const ordered = Object.values(chatRooms);
You could now also apply a custom sort order, e.g.:
ordered.sort((roomA, roomB) => roomA.name.localeCompare(roomB.name));
To now serialize only one chunk it is as easy as:
let index = 0, chunk = 100;
const result = ordered.slice(index * chunk, (index + 1) * chunk).map(room => room.serialize());
Related
Building a script in google apps script.
I get values from an invoice data sheet with multiple lines per invoice so as to account for line items.
My progress so far has been to extract individual invoice numbers from the column (each invoice number occurs as many line items the individual invoice has).
The array todaysInvoices looks like this: [35033817, 35033818, 35033819, 35033820, 35033821]
Now, I need a way to create an object for each of these invoice numbers that has different properties (such as invoiceDate and customerName etc.). The initial invoice number as in the array should thereby be assigned as 'id' property to the new invoice object.
I need help to use objects in javascript.
If you require additional information, please let me know.
Below is a screenshot of a simplified version of my order sheet:
This is a clipping of my order sheet. Before and after the shown columns there are many more with more details but the hierarchies of information are already in the image
Below is the code I have so far:
const orderSheet = SpreadsheetApp.openById('SPREADSHEETID').getSheetByName('SHEETNAME');
const invoiceTemplate = DriveApp.getFileById('DOCUMENTID');
const tempFolder = DriveApp.getFolderById('FOLDERID');
const invoiceData = orderSheet.getRange(4,7, orderSheet.getLastRow() - 1, 57).getDisplayValues().filter(function (rows){ return rows[0] === 'INVOICED'});
const invDataRepo = SpreadsheetApp.openById('SPREADSHEETID2');
var timestamp = new Date();
function printBulkInvoices() {
logLineItems ();
var todaysInvoices = uniqueInvIDs ();
todaysInvoices.sort();
todaysInvoices.map(String);
//fetchInvData (todaysInvoices);
Logger.log (todaysInvoices)
}
function fetchInvData (invoiceIDs) {
let invoices = {
}
Logger.log(invoices)
invoiceIDs.forEach
}
function fetchLineItems (invoiceDataArray) {
}
// send array of todays unique invoice numbers (later all inv data?) to invdata sheet and log them
function logTodaysInvoices (invIDArr){
invIDArr.forEach
invDataRepo.getSheetByName('invdata').getRange(invDataRepo.getSheetByName('invdata').getLastRow()+1,1,invIDArr.length,1).setValue(invIDArr);
}
// return an array of unique invoice ids from todays invoice data
function uniqueInvIDs (){
let singleArray = invoiceData.map(row => row[5]);
let unique = [...new Set(singleArray)];
return unique;
}
//log incoicedata to invdatarepo-sheet 'lineitems'
function logLineItems (){
invDataRepo.getSheetByName('lineitems').getRange(invDataRepo.getSheetByName('lineitems').getLastRow()+1,2,invoiceData.length,invoiceData[0].length).setValues(invoiceData);
}
It's hard to say exactly what you need since we cannot see your Invoice Data Sheet.
But here's something that might give you a start:
let iobj = {idA:[]};
[35033817, 35033818, 35033819, 35033820, 35033821].forEach((id => {
if(!iobj.hasOwnProperty(id)) {
iobj[id]={date: invoiceDate, name: customName, items:[]};
iobj.idA.push(id);//I find it handy to have an array of object properties to loop through when I wish to reorganize the data after it's all collected
} else {
iobj[id].items.push({item info properties});//I am guessing here that you may wish to addition additional information about the items which are on the current invoice
}
});
Javascript Object
To follow up from your question:
Your loop to collect object data would start to look something like this:
function getInvoiceData() {
const ss = SpreadsheetApp.getActive();
const ish = ss.getSheetByName('Invoice Data');
const isr = 2;
const hA = ish.getRange(1, 1, 1, ish.getLastColumn()).getValues()[0];
let idx = {};//object return head index into row array based on header title which in this case I assume invoice number is labeled 'Invoicenumber'
hA.forEach((h, i) => {idx[h] = i});
const vs = ish.getRange(isr, 1, ish.getLastRow() - isr + 1, ish.getLastColumn()).getValues();
let iobj = { idA: [] };
vs.forEach(r => {
if (!iobj.hasOwnProperty(r[idx['invoicenumber']])) {
iobj[r[idx['invoicenumber']]] = { date: r[idx['invoicedate']], name: r[idx['customername']], items: [] };
iobj.idA.push(r[idx['invoicenumber']]);
} else {
iobj[r[idx['invoicenumber']]].items.push({ iteminfoproperties:'' });
}
});
}
I have a huge list of items about almost all the crops and these data is to be plotted using maps and charts. I would like to count the number of each crop, say how many times was cabbage planted. I use Firebase database to store the data and I retrieve it using this function below:
database = firebase.database()
var ref = database.ref('Planting-Calendar-Entries');
ref.on('value', gotData, errData);
function gotData(data){
console.log(data.val())
var veggie = data.val();
var keys = Object.keys(veggie);
console.log(keys);
let counter = 0
for (var i = 0; i < keys.length; i++){
var k = keys[i];
var Veg_planted = veggie[k].Veg_planted;
var coordinates = veggie[k].coordinates;
if (Veg_planted == 'Cabbage'){
counter++;
}
// vegAll = Veg_planted.count()
console.log(Veg_planted, coordinates)
}
console.log(counter)
}
function errData(err){
console.log('Error!');
console.log(err)
}
This data I retrieve it from the database where it gets updated whenever someone submits their planting information. The code I used above will only apply if my list is small, but I have a list of about 170 items and it would be hard to write code to count each crop individually using something like let counter = 0, counter++. Is there a way I could navigate around this?
I'm assuming data.val() returns an array, not an object, and you're misusing Object.keys() on an array instead of just looping over the array itself. If that's true, then it sounds like you want to group by the Veg_planted key and count the groupings:
const counts = Object.values(veggie).reduce((counts, { Veg_planted }) => ({
...counts,
[Veg_planted]: (counts[Veg_planted] || 0) + 1
}), {});
Usage:
const veggie = [{ Veg_planted: 'Cabbage' }, { Veg_planted: 'Cabbage' }, { Veg_planted: 'Corn' }];
// result of counts:
// {Cabbage: 2, Corn: 1}
Actually: the code to count the items is probably going to be the same, no matter how many items there are. The thing that is going to be a problem as you scale though is the amount of data that you have to retrieve that you're not displaying to the user.
Firebase does not support aggregation queries, and your approach only works for short lists of items. For a more scalable solution, you should store the actual count itself in the database too.
So:
Have a blaCount property for each bla that exists.
Increment/decrement the counter each time your write/remove a bla to/from the database.
Now you can read only the counters, instead of having to read the individual items.
Firestore would be better option. You can query based on the field value.
var plantingRef = db.collection("PlantingCalendarEntries");
var query = plantingRef.where("Veg_planted", "==", "Cabbage");
if you still want to stuck with realtime database.
Save Counters to database.
Or use cloud dunctions to count.
I'm working with MeteorJS (aned MongoDB).
I have two collections :
events, with idEvent
eventsType, with idEventType (finite list of
type of events)
The link between two collections must be realized with idEvent == idEventType.
The goal is to have an array of events, with eventstype object associed.
This following code is functionnal, but I find it horrible... What did you think about ?
events() {
// Type of event
const eventsType = EventsType.find();
const eventsTypeArray = [];
eventsType.forEach((ev) => {
eventsTypeArray[ev.idEventType] = ev;
});
// List of events
const eventsList = Events.find();
const eventsListArray = [];
// Merge both data
eventsList.forEach((ev) => {
const evObj = ev;
evObj.type = eventsTypeArray[ev.idEvent];
eventsListArray.push(evObj);
});
return eventsListArray;
}
Thanks ! :D
You could map your eventsList and use Object.assign to enrich the original item :
eventsListArray = eventsList.map(ev => Object.assign({type: eventsTypeArray[ev.idEvent]}, ev))
Test run :
originalArray = [{a:"1"}, {a:"2"}];
dataMap = { "1": 10, "2": 100 };
mappedArray = originalArray.map(i=>Object.assign({b:dataMap[i.a]}, i));
console.log(originalArray);
console.log(mappedArray);
Result :
[{a:"1"}, {a:"2"}] //original array left untouched
[{a:"1", b:10}, {a:"2", b:100}] // mappedArray contains the extra data
I actually had a similar problem recently where I wanted to join data from two collections.
My solution was to create a new local collection (this is a collection that lives on the client only).
client:
const LocalEvents = new Mongo.Collection(null);
From there, instead of pushing your joined objects in to an array, you can join them and push the new objects in to the LocalEvents collection. This gives you the benefit of being able to query the new objects from the local minimongo collection. You'll need to make sure you clear the local collection when the template/component is destroyed. Also run a tracker function to empty the LocalCollection if your cursor changes.
Tracker.autorun((eventsType) => {
LocalEvents.remove({});
});
I would like to store product information in a key, value array, with the key being the unique product url. Then I would also like to store the visit frequency of each of these products. I will store these objects as window.localStorage items, but that's not very important.
The thing I had in mind was two key value arrays:
//product information
prods["url"] = ["name:product_x,type:category_x,price:50"]
//product visits frequency
freq["url"] = [6]
Then I would like to sort these prods based on the frequency.
Is that possible?
Hope you guys can help! Thanks a lot
Well you seem to have made several strange choices for your data format/structure. But assuming the format of the "prod" is beyond your control but you can choose your data structure, here's one way to do it.
Rather than two objects both using url as a key and having one value field each I've made a single object still keyed on url but with the product and frequency information from each in a field.
Objects don't have any inherent order so rather than sorting the table object I sort the keys, your "url"s ordered by ascending frequency.
To show that it's sorted that way I print it out (not in the same format).
For descending frequency, change data[a].freq - data[b].freq to data[b].freq - data[a].freq
var data = {
"url": {
prod: "name:product_x,type:category_x,price:50",
freq: 6
},
"url2": {
prod: "name:product_y,type:category_y,price:25",
freq: 3
}
};
var sorted = Object.keys(data).sort((a, b) => data[a].freq - data[b].freq);
console.log(sorted.map(k => [data[k].freq, k, data[k].prod]));
There's more than one way to format the data, which would change the shape of the code here.
maybe something like this:
var prods = [
{url:1, val:[{name:'a',type:'x',price:60}]},
{url:2, val:[{name:'b',type:'x',price:30}]},
{url:3, val:[{name:'c',type:'x',price:50}]},
{url:4, val:[{name:'c',type:'x',price:20}]},
{url:5, val:[{name:'c',type:'x',price:10}]},
{url:6, val:[{name:'c',type:'x',price:40}]}
];
var freq = [
{url:1, freq:6},
{url:2, freq:3},
{url:3, freq:5},
{url:4, freq:2},
{url:5, freq:1},
{url:6, freq:4}
];
prods.sort(function (a, b) {
var aU = freq.filter(function(obj) {
return obj.url === a.url;
});
var bU = freq.filter(function(obj) {
return obj.url === b.url;
});
if (aU[0].freq > bU[0].freq) {
return 1;
}
if (aU[0].freq < bU[0].freq) {
return -1;
}
return 0;
});
I'm working with a large dataset that needs to be efficient with its Mongo queries. The application uses the Ford-Fulkerson algorithm to calculate recommendations and runs in polynomial time, so efficiency is extremely important. The syntax is ES6, but everything is basically the same.
This is an approximation of the data I'm working with. An array of items and one item being matched up against the other items:
let items = ["pen", "marker", "crayon", "pencil"];
let match = "sharpie";
Eventually, we will iterate over match and increase the weight of the pairing by 1. So, after going through the function, my ideal data looks like this:
{
sharpie: {
pen: 1,
marker: 1,
crayon: 1,
pencil: 1
}
}
To further elaborate, the value next to each key is the weight of that relationship, which is to say, the number of times those items have been paired together. What I would like to have happen is something like this:
// For each in the items array, check to see if the pairing already
// exists. If it does, increment. If it does not, create it.
_.each(items, function(item, i) {
Database.upsert({ match: { $exist: true }}, { match: { $inc: { item: 1 } } });
})
The problem, of course, is that Mongo does not allow bracket notation, nor does it allow for variable names as keys (match). The other problem, as I've learned, is that Mongo also has problems with deeply nested $inc operators ('The dollar ($) prefixed field \'$inc\' in \'3LhmpJMe9Es6r5HLs.$inc\' is not valid for storage.' }).
Is there anything I can do to make this in as few queries as possible? I'm open to suggestions.
EDIT
I attempted to create objects to pass into the Mongo query:
_.each(items, function(item, i) {
let selector = {};
selector[match] = {};
selector[match][item] = {};
let modifier = {};
modifier[match] = {};
modifier[match]["$inc"] = {};
modifier[match]["$inc"][item] = 1
Database.upsert(selector, modifier);
Unfortunately, it still doesn't work. The $inc breaks the query and it won't let me go more than 1 level deep to change anything.
Solution
This is the function I ended up implementing. It works like a charm! Thanks Matt.
_.each(items, function(item, i) {
let incMod = {$inc:{}};
let matchMod = {$inc:{}};
matchMod.$inc[match] = 1;
incMod.$inc[item] = 1;
Database.upsert({node: item}, matchMod);
Database.upsert({node: match}, incMod);
});
I think the trouble comes from your ER model. a sharpie isn't a standalone entity, a sharpie is an item. The relationship between 1 item and other items is such that 1 item has many items (1:M recursive) and each item-pairing has a weight.
Fully normalized, you'd have an items table & a weights table. The items table would have the items. The weights table would have something like item1, item2, weight (in doing so, you can have asymmetrical weighting, e.g. sharpie:pencil = 1, pencil:sharpie = .5, which is useful when calculating pushback in the FFA, but I don't think that applies in your case.
Great, now let's mongotize it.
When we say 1 item has many items, that "many" is probably not going to exceed a few thousand (think 16MB document cap). That means it's actually 1-to-few, which means we can nest the data, either using subdocs or fields.
So, let's check out that schema!
doc =
{
_id: "sharpie",
crayon: 1,
pencil: 1
}
What do we see? sharpie isn't a key, it's a value. This makes everything easy. We leave the items as fields. The reason we don't use an array of objects is because this is faster & cleaner (no need to iterate over the array to find the matching _id).
var match = "sharpie";
var items = ["pen", "marker", "crayon", "pencil"];
var incMod = {$inc:{}};
var matchMod = {$inc:{}};
matchMod.$inc[match] = 1;
for (var i = 0; i < items.length; i++) {
Collection.upsert({_id: items[i]}, matchMod);
incMod.$inc[items[i]] = 1;
}
Collection.upsert({_id: match}, incMod);
That's the easy part. The hard part is figuring out why you want to use an FFA for a suggestion engine :-P.