Meteor sends entire array when only single element is updated - javascript

I have a collection that has a 10 x 10 2-d array field. When I update an element in this array, the entire field is sent back to me as changed.
Is there a better way to do the update part?
I'm new to noSQL so maybe I just need to rethink my database setup. This would be a little depressing as I love the simplicity of being able to map javascript objects directly to fields in a single collection. I, however, am not willing to lose ~500 bytes of overhead every time I update this thing.
Is there a way to force Meteor to update the client with more fine-grained changes? Or is this a limitation of how Meteor Livequery works?
Thanks!

The problem is that DDP only supports changed messages at the top-level field granularity. With your current structure, any change to a sub-field of squares will result in the entire squares field being transmitted to the client.
One alternative is to store each element of the 2D array as a separate, top-level field. Below is a complete working example using a - character to separate the elements:
var pool2squares = function(pool) {
var squares = [];
_.each(pool, function(value, key) {
var parts = key.split('-');
if (parts[0] === 'squares') {
var i = Number(parts[1]);
var j = Number(parts[2]);
if (squares[i] == null)
squares[i] = [];
squares[i][j] = value;
}
});
return squares;
};
Pools = new Mongo.Collection('pools', {
transform: function(doc) {
return _.extend(doc, {squares: pool2squares(doc)});
}
});
Meteor.startup(function() {
Pools.insert({
'squares-0-0': 'a',
'squares-0-1': 'b',
'squares-1-0': 'c',
'squares-1-1': 'd'
});
console.log(Pools.findOne().squares);
});
Here we are using a collection transform to add a squares property to each Pool document whenever it's fetched. The above code will print:
[ [ 'a', 'b' ], [ 'c', 'd' ] ]
To update a Pool:
Pools.update(id, {$set: {'squares-3-4': 'hello'}});

Related

How to count a huge list of items

I have a huge list of items about almost all the crops and these data is to be plotted using maps and charts. I would like to count the number of each crop, say how many times was cabbage planted. I use Firebase database to store the data and I retrieve it using this function below:
database = firebase.database()
var ref = database.ref('Planting-Calendar-Entries');
ref.on('value', gotData, errData);
function gotData(data){
console.log(data.val())
var veggie = data.val();
var keys = Object.keys(veggie);
console.log(keys);
let counter = 0
for (var i = 0; i < keys.length; i++){
var k = keys[i];
var Veg_planted = veggie[k].Veg_planted;
var coordinates = veggie[k].coordinates;
if (Veg_planted == 'Cabbage'){
counter++;
}
// vegAll = Veg_planted.count()
console.log(Veg_planted, coordinates)
}
console.log(counter)
}
function errData(err){
console.log('Error!');
console.log(err)
}
This data I retrieve it from the database where it gets updated whenever someone submits their planting information. The code I used above will only apply if my list is small, but I have a list of about 170 items and it would be hard to write code to count each crop individually using something like let counter = 0, counter++. Is there a way I could navigate around this?
I'm assuming data.val() returns an array, not an object, and you're misusing Object.keys() on an array instead of just looping over the array itself. If that's true, then it sounds like you want to group by the Veg_planted key and count the groupings:
const counts = Object.values(veggie).reduce((counts, { Veg_planted }) => ({
...counts,
[Veg_planted]: (counts[Veg_planted] || 0) + 1
}), {});
Usage:
const veggie = [{ Veg_planted: 'Cabbage' }, { Veg_planted: 'Cabbage' }, { Veg_planted: 'Corn' }];
// result of counts:
// {Cabbage: 2, Corn: 1}
Actually: the code to count the items is probably going to be the same, no matter how many items there are. The thing that is going to be a problem as you scale though is the amount of data that you have to retrieve that you're not displaying to the user.
Firebase does not support aggregation queries, and your approach only works for short lists of items. For a more scalable solution, you should store the actual count itself in the database too.
So:
Have a blaCount property for each bla that exists.
Increment/decrement the counter each time your write/remove a bla to/from the database.
Now you can read only the counters, instead of having to read the individual items.
Firestore would be better option. You can query based on the field value.
var plantingRef = db.collection("PlantingCalendarEntries");
var query = plantingRef.where("Veg_planted", "==", "Cabbage");
if you still want to stuck with realtime database.
Save Counters to database.
Or use cloud dunctions to count.

Get parent array from child array

In the code below I can easily enter into a subarray within the array with current = current[0].
var root = [
[
'Value 1',
]
];
var current = root;
current = current[0];
current.push('Value 2');
document.write(JSON.stringify(root));
How can I do the reverse and go up a level? In this example, current would become:
[['Value 1', 'Value 2']]
The only way I can think of doing this would be looping, but duplicates would be very probably and problematic. All I've found online is looping, which won't work as a mention in the last sentence.
EDIT
I'm looping through a large string and converting it to a tree. Certain characters will require a indent in the tree or a separate branch, while others will require an outdent and return to the parent function. I've achieved the indents by having a current variable just enter a new subarray as suggested here. I just need a way to exit with a different character.
You can't.
Arrays have no concept of a "parent" or anything. They don't care where their references are being held. However, if you really really need to, you can implement the concept of a "parent" yourself using a simple recursive function setting properties of the arrays - see below:
var root = [
[
'Value 1',
]
];
function setParents(root) {
for(var i = 0; i < root.length; i++) {
if(Array.isArray(root[i])) {
root[i].parent = root;
setParents(root[i]);
}
}
}
setParents(root);
var current = root;
current = current[0];
current.push('Value 2');
document.write(JSON.stringify(root));
current = current.parent;
console.log(current);
This makes use of the fact that array properties don't show up when logged or serialized as JSON, only their elements. So they're kind of "hidden" in a sense. It's a bit hacky but could work for you.
However, I recommend you simply avoid overwriting current - there's not really a great need to use it like a cursor traversing some hierarchy structure.
Simply pushing BEFORE setting current variable will work.
var root = [
[
'Value 1',
]
];
var current = root;
root.push("AAA");
current = current[0];
current.push('Value 2');
document.write(JSON.stringify(root));

Select Random Item (shown in pairs) from Array & Remove It, Restart Once Array is Empty

I'm super newbie in coding and I need help to achieve this code.
I'm trying to get a random item (in pairs) from an array and then remove it from this array until user gets to the last item or 60 days have gone from using the service (cookie?)... I have build a script with the help of other questions here in stackoverflow and here is my results so far.
`<script>
var randomizer = document.getElementById("getImgBut");
var dog1 = '/app/wp-content/mediaApp/yo-creo-mi-realidad/01F.jpg';
var dog2 = '/app/wp-content/mediaApp/yo-creo-mi-realidad/01B.jpg';
var dogpics=[dog1,dog2];
var yourPics = [
dogpics,
[ '/app/wp-content/mediaApp/yo-creo-mi-realidad/02F.jpg', '/app/wp-content/mediaApp/yo-creo-mi-realidad/02B.jpg' ],
[ '/app/wp-content/mediaApp/yo-creo-mi-realidad/03F.jpg', '/app/wp-content/mediaApp/yo-creo-mi-realidad/03B.jpg' ],
[ '/app/wp-content/mediaApp/yo-creo-mi-realidad/04F.jpg', '/app/wp-content/mediaApp/yo-creo-mi-realidad/04B.jpg' ],
[ '/app/wp-content/mediaApp/yo-creo-mi-realidad/05F.jpg', '/app/wp-content/mediaApp/yo-creo-mi-realidad/05B.jpg' ],
[ '/app/wp-content/mediaApp/yo-creo-mi-realidad/06F.jpg', '/app/wp-content/mediaApp/yo-creo-mi-realidad/06B.jpg' ] //This array has 52 cards but I cutted it for example purposes
];
function get_random_number(array){
return Math.floor(Math.random() * array.length |0);
} // here is where I have tried to modify with other scripts like the one in this page https://stackoverflow.com/questions/38882487/select-random-item-from-array-remove-it-restart-once-array-is-empty with no success
randomizer.addEventListener("click", function() {
var rand_number = get_random_number(yourPics);
console.log(rand_number);
document.getElementById('img1').src = yourPics[rand_number][0];
document.getElementById('img2').src = yourPics[rand_number][1];
});
var card = document.querySelector('.card');
card.addEventListener( 'click', function() {
card.classList.toggle('is-flipped');
});
</script>`
Thank you for your help!
I don't fully understand what you mean by "remove in pairs", but I'll answer presuming you mean you wish to remove the image ending in 02F.jpg at the same time as removing the image ending in 02B.jpg, and then 03F.jpg at the same time as 03B.jpg.
The solution to this that I will propose is that we will structure your data a bit differently to begin with. That is, if those images, the "B image" and "F image" are linked, we could keep them in the same `javascript object. This would look like:
var yourPics = [
{
bImage: '/app/wp-content/mediaApp/yo-creo-mi-realidad/02F.jpg',
fImage: '/app/wp-content/mediaApp/yo-creo-mi-realidad/02B.jpg'
},
{
bImage: '/app/wp-content/mediaApp/yo-creo-mi-realidad/03F.jpg',
fImage: '/app/wp-content/mediaApp/yo-creo-mi-realidad/03B.jpg'
}...]
This would then be an array of objects, rather than strings. We can access the bImage property of an object with just
myObject = yourPics[0]
myObject.bImage
We could delete one of those objects those at random via splice.
myRandomlyRemovedObject = yourPics.splice(myIndexToDeleteFrom, 1) would remove 1 object from yourPics at position of myIndexToDeleteFrom, which you presumably would choose randomly. myRandomlyRemovedObject would be assigned to the one object we removed.
I think this object based approach is safer since you will know for a fact that you will removed both matching strings at the same time.

What is the best way to realise "join" with JS objects?

I'm working with MeteorJS (aned MongoDB).
I have two collections :
events, with idEvent
eventsType, with idEventType (finite list of
type of events)
The link between two collections must be realized with idEvent == idEventType.
The goal is to have an array of events, with eventstype object associed.
This following code is functionnal, but I find it horrible... What did you think about ?
events() {
// Type of event
const eventsType = EventsType.find();
const eventsTypeArray = [];
eventsType.forEach((ev) => {
eventsTypeArray[ev.idEventType] = ev;
});
// List of events
const eventsList = Events.find();
const eventsListArray = [];
// Merge both data
eventsList.forEach((ev) => {
const evObj = ev;
evObj.type = eventsTypeArray[ev.idEvent];
eventsListArray.push(evObj);
});
return eventsListArray;
}
Thanks ! :D
You could map your eventsList and use Object.assign to enrich the original item :
eventsListArray = eventsList.map(ev => Object.assign({type: eventsTypeArray[ev.idEvent]}, ev))
Test run :
originalArray = [{a:"1"}, {a:"2"}];
dataMap = { "1": 10, "2": 100 };
mappedArray = originalArray.map(i=>Object.assign({b:dataMap[i.a]}, i));
console.log(originalArray);
console.log(mappedArray);
Result :
[{a:"1"}, {a:"2"}] //original array left untouched
[{a:"1", b:10}, {a:"2", b:100}] // mappedArray contains the extra data
I actually had a similar problem recently where I wanted to join data from two collections.
My solution was to create a new local collection (this is a collection that lives on the client only).
client:
const LocalEvents = new Mongo.Collection(null);
From there, instead of pushing your joined objects in to an array, you can join them and push the new objects in to the LocalEvents collection. This gives you the benefit of being able to query the new objects from the local minimongo collection. You'll need to make sure you clear the local collection when the template/component is destroyed. Also run a tracker function to empty the LocalCollection if your cursor changes.
Tracker.autorun((eventsType) => {
LocalEvents.remove({});
});

Meteor 1.0 - Mongo queries using variables as key, including $inc

I'm working with a large dataset that needs to be efficient with its Mongo queries. The application uses the Ford-Fulkerson algorithm to calculate recommendations and runs in polynomial time, so efficiency is extremely important. The syntax is ES6, but everything is basically the same.
This is an approximation of the data I'm working with. An array of items and one item being matched up against the other items:
let items = ["pen", "marker", "crayon", "pencil"];
let match = "sharpie";
Eventually, we will iterate over match and increase the weight of the pairing by 1. So, after going through the function, my ideal data looks like this:
{
sharpie: {
pen: 1,
marker: 1,
crayon: 1,
pencil: 1
}
}
To further elaborate, the value next to each key is the weight of that relationship, which is to say, the number of times those items have been paired together. What I would like to have happen is something like this:
// For each in the items array, check to see if the pairing already
// exists. If it does, increment. If it does not, create it.
_.each(items, function(item, i) {
Database.upsert({ match: { $exist: true }}, { match: { $inc: { item: 1 } } });
})
The problem, of course, is that Mongo does not allow bracket notation, nor does it allow for variable names as keys (match). The other problem, as I've learned, is that Mongo also has problems with deeply nested $inc operators ('The dollar ($) prefixed field \'$inc\' in \'3LhmpJMe9Es6r5HLs.$inc\' is not valid for storage.' }).
Is there anything I can do to make this in as few queries as possible? I'm open to suggestions.
EDIT
I attempted to create objects to pass into the Mongo query:
_.each(items, function(item, i) {
let selector = {};
selector[match] = {};
selector[match][item] = {};
let modifier = {};
modifier[match] = {};
modifier[match]["$inc"] = {};
modifier[match]["$inc"][item] = 1
Database.upsert(selector, modifier);
Unfortunately, it still doesn't work. The $inc breaks the query and it won't let me go more than 1 level deep to change anything.
Solution
This is the function I ended up implementing. It works like a charm! Thanks Matt.
_.each(items, function(item, i) {
let incMod = {$inc:{}};
let matchMod = {$inc:{}};
matchMod.$inc[match] = 1;
incMod.$inc[item] = 1;
Database.upsert({node: item}, matchMod);
Database.upsert({node: match}, incMod);
});
I think the trouble comes from your ER model. a sharpie isn't a standalone entity, a sharpie is an item. The relationship between 1 item and other items is such that 1 item has many items (1:M recursive) and each item-pairing has a weight.
Fully normalized, you'd have an items table & a weights table. The items table would have the items. The weights table would have something like item1, item2, weight (in doing so, you can have asymmetrical weighting, e.g. sharpie:pencil = 1, pencil:sharpie = .5, which is useful when calculating pushback in the FFA, but I don't think that applies in your case.
Great, now let's mongotize it.
When we say 1 item has many items, that "many" is probably not going to exceed a few thousand (think 16MB document cap). That means it's actually 1-to-few, which means we can nest the data, either using subdocs or fields.
So, let's check out that schema!
doc =
{
_id: "sharpie",
crayon: 1,
pencil: 1
}
What do we see? sharpie isn't a key, it's a value. This makes everything easy. We leave the items as fields. The reason we don't use an array of objects is because this is faster & cleaner (no need to iterate over the array to find the matching _id).
var match = "sharpie";
var items = ["pen", "marker", "crayon", "pencil"];
var incMod = {$inc:{}};
var matchMod = {$inc:{}};
matchMod.$inc[match] = 1;
for (var i = 0; i < items.length; i++) {
Collection.upsert({_id: items[i]}, matchMod);
incMod.$inc[items[i]] = 1;
}
Collection.upsert({_id: match}, incMod);
That's the easy part. The hard part is figuring out why you want to use an FFA for a suggestion engine :-P.

Categories