Mongoose pagination with first item instead of skip [duplicate] - javascript

I'm trying to calculate the skip value for a given record in a mongo db collection using the php driver. So taking a given record, find out the index of that record within the entire collection. Is this possible?
Currently I'm selecting all records and manually doing an index of on the array of results.


This is called "forward paging" which is a concept you can use to "efficiently page" through results in a "forward" direction when using "sorted" results.
JavaScript logic included (because it works in the shell), but not hard to translate.
The concept in general:
{ "_id": 1, "a": 3 },
{ "_id": 2, "a": 3 },
{ "_id": 3, "a": 3 },
{ "_id": 4, "a": 2 },
{ "_id": 5, "a": 1 },
{ "_id": 6, "a": 0 }
Consider those "already sorted" documents ( for convienience ) as an example of results we want to "page" by "two" items per page.
In the first instance you do something like this:
var lastVal = null,
lastSeen = [];
db.collection.find().sort({ "a": -1 }).limit(2).forEach(function(doc) {
if ( lastVal != doc.a ) {
lastSeen = [];
}
lastVal = doc.a;
lastSeen.push( doc._id );
// do something useful with each document matched
});
Now those lastVal and lastSeen are something you store in something like a "session variable" than can be accessed on the next request in terms of web applications, or otherwise something similar where not.
What they should contain though are the very last value you were sorting on and the list of "unique" _id values that were seen since that value did not change. Hence:
lastVal = 3,
lastSeen = [1,2];
The point is that when the request for the "next page" comes around then you want to use those variables for something like this:
var lastVal = 3,
lastSeen = [1,2];
db.collection.find({
"_id": { "$nin": lastSeen },
"a": { "$lte": lastVal }
}).sort({ "a": -1 }).limit(2).forEach(function(doc) {
if ( lastVal != doc.a ) {
lastSeen = [];
}
lastVal = doc.a;
lastSeen.push( doc._id );
// do something useful with each document matched
});
What that does is "exclude" all values of _id that are recorded in lastSeen from the list of results, as well as make sure that all results need to be "less than or equal to" ( descending order ) the lastVal recorded for the sort field "a".
This yields the next two results in the collection:
{ "_id": 3, "a": 3 },
{ "_id": 4, "a": 2 },
But after processing our values now look like this:
lastVal = 2,
lastSeen = [4];
So now the logic follows that you don't need to exclude the other _id values seen before since you are only really looking for values of "a" than are "less than or equal to" the lastVal and since there was only "one" _id value seen at that value then only exclude that one.
This of course yields the next page on using the same code as just above:
{ "_id": 5, "a": 1 },
{ "_id": 6, "a": 0 }
That is the most effiecient way to "forward page" through results in general and is particularly useful for efficient paging of "sorted" results.
If however you want to "jump" to page 20 or similar action at any stage then this is not for you. You are stuck with the traditional .skip() and .limit() approach to be able to do this by "page number" since there is no other rational way to "calculate" this.
So it all depends on how your application is implementing "paging" and what you can live with. The .skip() and .limit() approach suffers the performance of "skipping" and can be avoided by using the approach here.
On the other hand, if you want "jump to page" then "skipping" is your only real option unless you want to build a "cache" of results. But that's another issue entirely.

Related

Javascript Best Practices - Sorting Mixed Data from Array

I just started trying Javascript and I'm struggling. My end result is supposed to be a chronological timeline of activities (Calls, Meetings, Tasks).
I'm receiving a file from an application with different types of records. It contains Calls, Meetings, and Tasks. When I receive the file, they are in no particular order and each record type has different fields. I need to get them into the same table but sorted by date.
Here is a sample file with a Call and a Task but it might have 10 or more records of differing type.
[
{
"Owner": {
"name": "Raymond Carlson",
},
"Check_In_State": null,
"Activity_Type": "Calls",
"Call_Start_Time": "2022-10-23T20:00:00-05:00",
"$editable": true,
"Call_Agenda": "Need to call and discuss some upcoming events",
"Subject": "Call scheduled with Florence"
},
{
"Owner": {
"name": "Raymond Carlson",
},
"Check_In_State": null,
"Activity_Type": "Tasks",
"Due_Date": "2022-10-24",
"$editable": true,
"Description": "-Review Action Items from Last week”,
"Subject": "Complete Onboarding"
}
]
This is what I'm doing now and I know it's not the best way to go about it.
for (var i = 0; i < obj.length; i++) {
var activityType0 = (obj[0].Activity_Type)
var activityOwner0 = (obj[0].Owner.name);
if (activityType0 == "Events") {
start0 = (obj[0].Start_DateTime)
startDate0 = new Date(start0);
activityDate0 = startDate0.toLocaleString();
activityTitle0 = (obj[0].Subject);
activityDesc0 = (obj[0].Description);
}
else if (activityType0 == "Tasks"){
dueDate0 = (obj[0].Due_Date)
activityDate0 = dueDate0;
activityTitle0 = (obj[0].Subject);
activityDesc0 = (obj[0].Description);
}
else if (activityType0 == "Calls"){
callStart0 = (obj[0].Call_Start_Time)
callStartTime0 = new Date(callStart0);
activityDate0 = callStartTime0.toLocaleString();
activityTitle0 = (obj[0].Subject);
activityDesc0 = (obj[0].Call_Agenda);
}
}
So regardless of the type of record, I have an
activityOwner,
activityDate,
activityTitle,
activityDesc,
And that's what I need.
Aside from that code above needing work, now my question is, what do I need to do with these values for each record to put them in order by "activityDate". Do I need to put them back into an array then sort and if so, what's the best approach?
Thank you much!

Right now I'm not really sure what your end goal is, is it sorting by activity type or activity date ?
If it's the latter, you can try referring to this answer, or try to sort activity type by the ASCII number of each starting letter in each type (e.g. "C" in "Call", "T" in "Tasks", etc.)

how to keep track of objects inserted in an array

When clicking add it should keep inserting object to the array and the order Id should be based from the current order id + 1 of the objects on the array so for example if if templatesDto is empty then every order in the object should increment starting from 0 when I click add
but for example there are existing data on the templatesDto
example
[
{
"id": 255,
"order": 0,
},
{
"id": 256,
"order": 1,
},
{
"id": 256,
"order": 2,
},
]
then if I click add the next order value of the new added object should be 3 since the last one is 2.
Any idea guys ? Thank you.
#html code
<button (click)="add()" mat-stroked-button mat-button class="btn-add-entitlement action-btn">
<mat-icon aria-label="Add" class="add-icon">add</mat-icon> Add
</button>
#ts code - to insert object to array
this.templatesDto= []
add() {
this.templatesDto.push({
id: 0,
order : 0,
})
}
#sample result if I click add and there are no data in templatesDto
[
{
"id": 0,
"order": 0,
},
{
"id": 0,
"order": 1,
},
{
"id": 0,
"order": 2,
}
....
]

Set order to call getNextOrder() with the function being:
function getNextOrder() {
if (this.templatesDto.length === 0) {
return 0
}
let maxOrder = 0
for (const template of this.templatesDto) {
if (template.order > maxOrder) {
maxOrder = template.order
}
}
return maxOrder + 1
}
This function goes through all objects in the array, gets the highest order value and then returns it incremented by 1.
This should also work, even if the objects in the array are not ordered according to the order property.

The best way to find the last number if there is one would be :
this.templatesDto.length > 0 ? this.templatesDto[this.templatesDto.length-1].order : 1
This way if there isn't an order already it will start at 1.
add() {
const orderNo = this.templatesDto.length > 0 ? this.templatesDto[this.templatesDto.length-1].order + 1 : 1 ;
this.templatesDto.push({
id: 0,
order : orderNo ,
})
}

The previous answers are all correct and will be sufficient for the current state of the Question.
I however would append the case, that the Order might be shuffled or at any point an element would be remove, would eventually cause some issues.
For the case, that an element in the middle would be deleted/removed, it might not cause any trouble, as the SQL-like auto-increment would still work.
But in the case, that the elements would be sorted or shuffled, the methods previously mentioned would fail, as the last element wouldn't be always the highest orderNum.
For the case, that at any point, the case occurs, i would like to submit a method, that takes this case in consideration.
add(){
// We collect the last Order number
const nextOrder = getLastOrderNum();
// We can safely insert the lastOrderNum with a single increment.
// If it is -1 we then insert 0 and so on.
this.templatesDto.push({
id: 0,
order : (nextOrder + 1)
})
}
getLastOrderNum() {
// We iterate through the entire Array and reduce it
// by returning the higher order.
// If the Array has 0 elements, we will return the beginning Value of -1,
// which will latter be incremented to 0.
// With this we can spare some logic to check if the result were 0 and not
// increment.
return this.templatesDto.reduce((a,b) => Math.(a, b.order), -1);
}

Strange behavior in for-loop in JS/NodeJS

I'm working on a NodeJS/VueJS app and I encounter some strange behavior and I am not sure if it is something I am doing wrong or something else.
I have 2 arrays, one array of objects that contains campaigns and one array of objects that contains all promotions associated to all campaigns.
Each Campaign object contains, at this point, an empty array called promos, where I want to push all the promotions for each campaign.
for (var i = 0; i < campaigns.length; i++) {
for (var j = 0; j < campaignsPromo.length; j++) {
if (campaignsPromo.length > 0) {
if (campaigns[i].IDCampaign == campaignsPromo[j].IDCampaign) {
if ((campaignsPromo[j].associated == 1) && (campaignsPromo[j].validated == 1)) {
campaigns[i].promos.push(campaignsPromo[j]);
console.log("Validated Campaign ID " + campaigns[i].IDCampaign);
console.log("Promo ID " + campaignsPromo[j].IDPromo);
} else if ((campaignsPromo[j].associated == 1) && (campaignsPromo[j].validated == 0)) {
campaigns[i].unvalidatedPromos++;
console.log("Unvalidated Campaign ID " + campaigns[i].IDCampaign);
console.log("Promo ID " + campaignsPromo[j].IDPromo);
}
}
} else {
console.log("No promos!");
}
}
}
At first, the code seems to be doing what is supposed to do and it checks out with my test data set.
However, in the end, all campaigns end up having the same promotions.
Campaigns With No Promos: [{"IDCampaign":7,"campaignName":"dadsadafds","startDate":"2022-02-03","endDate":"2022-02-28","unvalidatedPromos":0,"promos":[]},{"IDCampaign":3,"campaignName":"Tarzan 3","startDate":"2022-02-02","endDate":"2022-02-06","unvalidatedPromos":0,"promos":[]},{"IDCampaign":1,"campaignName":"Tarzan","startDate":"2022-02-01","endDate":"2022-03-01","unvalidatedPromos":0,"promos":[]},{"IDCampaign":2,"campaignName":"Tarzan 2","startDate":"2022-02-01","endDate":"2022-02-08","unvalidatedPromos":0,"promos":[]},{"IDCampaign":4,"campaignName":"Tarzan 4","startDate":"2022-02-01","endDate":"2022-02-05","unvalidatedPromos":0,"promos":[]},{"IDCampaign":5,"campaignName":"Jabe 1","startDate":"2022-02-01","endDate":"2022-02-05","unvalidatedPromos":0,"promos":[]},{"IDCampaign":6,"campaignName":"dadsada","startDate":"2022-02-01","endDate":"2022-02-08","unvalidatedPromos":0,"promos":[]},{"IDCampaign":8,"campaignName":"Black Friday","startDate":"2022-02-01","endDate":"2022-02-28","unvalidatedPromos":0,"promos":[]}]
Validated Campaign ID 1
Promo ID 1119
Unvalidated Campaign ID 1
Promo ID 107
Campaigns With Promos: [{"IDCampaign":7,"campaignName":"dadsadafds","startDate":"2022-02-03","endDate":"2022-02-28","unvalidatedPromos":0,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]},{"IDCampaign":3,"campaignName":"Tarzan 3","startDate":"2022-02-02","endDate":"2022-02-06","unvalidatedPromos":0,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]},{"IDCampaign":1,"campaignName":"Tarzan","startDate":"2022-02-01","endDate":"2022-03-01","unvalidatedPromos":1,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]},{"IDCampaign":2,"campaignName":"Tarzan 2","startDate":"2022-02-01","endDate":"2022-02-08","unvalidatedPromos":0,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]},{"IDCampaign":4,"campaignName":"Tarzan 4","startDate":"2022-02-01","endDate":"2022-02-05","unvalidatedPromos":0,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]},{"IDCampaign":5,"campaignName":"Jabe 1","startDate":"2022-02-01","endDate":"2022-02-05","unvalidatedPromos":0,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]},{"IDCampaign":6,"campaignName":"dadsada","startDate":"2022-02-01","endDate":"2022-02-08","unvalidatedPromos":0,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]},{"IDCampaign":8,"campaignName":"Black Friday","startDate":"2022-02-01","endDate":"2022-02-28","unvalidatedPromos":0,"promos":[{"IDCampaign":1,"IDPromo":1119,"promoType":"CHRISTMASS","associated":1,"validated":1,"promoName":"Test Promo 1","beginDate":"2020-11-26","endingDate":"2020-11-29"}]}]
This is the content of campaignsPromo:
[ { IDCampaign: 1,
IDPromo: 1119,
promoType: 'CHRISTMASS',
associated: 1,
validated: 1,
promoName: 'Test Promo 1',
beginDate: '2020-11-26',
endingDate: '2020-11-29' },
{ IDCampaign: 1,
IDPromo: 107,
promoType: 'CHRISTMASS',
associated: 1,
validated: 0,
promoName: 'Test Promo 2',
beginDate: '2019-12-21',
endingDate: '2019-12-23' } ]
Any ideas? From where I'm standing, I am not doing anything wrong but it wouldn't be the first time I'm missing the obvious.
PS: Please ignore the fact my campaigns are called "Tarzan".

You didn't share the offending code, so I'll give a generic answer.
When the symptom is that a bunch of all objects seem to have a property that is the same array of items, it is likely caused by exactly that. Each object may be sharing the same array instance.
A typical mistake is something like the following:
var items = [];
var containers = [];
for (let i = 0; i < 3; ++i) {
items.push(i);
let container = {};
container.items = items;
containers.push(container);
}
console.log(containers);
Although one might expect, or even have intended to get 3 objects like this:
[
{ items: [ 0 ] },
{ items: [ 0, 1 ] },
{ items: [ 0, 1, 2 ] }
]
The items array is actually the same instance of an array. And what you actually get is more like:
[
{ items: [ 0, 1, 2 ] },
{ items: [ 0, 1, 2 ] },
{ items: [ 0, 1, 2 ] }
]
In fact, the stack snippet visualizer actually does a better job visualizing this because it outputs:
[
{
"items": [
/**id:3**/
0,
1,
2
]
},
{
"items": /**ref:3**/
},
{
"items": /**ref:3**/
}
]
In this way, it tries to inform you that it is actually the same array by giving it a ref:3 label and just marking the values of the other properties with the same ref:3 label as a comment.
Causes I see typically stem from a misunderstanding of what it means to assign an array to a property. Doing so does not create a copy of the array. Both objects refer to the same array.
This can be strange because: it can appear to be correct in a debugger by inspecting the contents of your array as your are stepping through a loop. Even if we had added log messages like console.log(items) or even console.log(container) inside the loop, we'd probably still not have enough of a clue that something went wrong because the contents of that one array instance change with each iteration of the loop and we can dump versions of it as text that appear to be correct, but then unwittingly change the contents of the array on the next iteration.
But if we log the entire containers array after the loop, you'll find that each object has the same instance of the array. Assining an array isn't bad if it's only one object that gets the array assigned to it, but if you assign the same array to properties of multiple objects, or in a loop, you may run into this problem.
One possible habit-breaker you can try is to inspect all objects in a second loop after your primary loop rather than sneaking your logging code directly into the first loop. It's less efficient, but if you're often making these kinds of mistakes, it can help you find problems and achieve correctness.
Another habit breaker is to console.log(JSON.stringify(foo)) rather than console.log(foo) because console.log in the browser actually remembers the reference of the object and shows you its current contents, rather than its contents at the time it was logged. This is different depending on the platform, where node.js will log as text instead of by reference.
Good luck and be careful!

Find index of filtered array

I have a JSON object which looks like this:
var JSON = {
"implants":[{
"familyId": "3",
"reference": "1234",
"quantity": "3"
},{
"familyId": "2",
"reference": "3456",
"quantity": "1"
}]
}
I'm building a function which searches whether an implant with the given "reference" is already in the array. If it's not, it just gets added to the array. Easy. If it is in the array, then I want to add the new quantity to the existing quantity and then update the array.
function returnTheseImplantsWhere(key1,key2){
return function(el){
if(el[key1]==key2) {
return el;
}
}
}
var activeCaseImplants = JSON.parse(getLocal("activeCaseImplants"));
var existingImplant = activeCaseImplants.implants.filter(returnTheseImplantsWhere("reference", window.currentImplantRef));
if(existingImplant.length) {
//previously added implant
newImplantQty = Number(existingImplant[0].quantity) + Number($("#qty").val());
existingImplant.quantity = newImplantQty;
} else {
//new implant
}
My question relates to the "previously added implant" section of the above code (which all works as expected). I now have the updated quantity, but I can I determine which object within the activeCaseImplants.implants array needs it's quantity updated? I feel like I could make use of indexOf somewhere around the line beginning with var existingImplant = ... but I'm not sure where or how. Any ideas?

Javascript Compare Index with Array Value

I have a main array, and an object that contains an array of items. I want to see if the object that contains items matches the main array index. If a match, then write to the console.
var mainItems = [0,1,2,3,4,5,6,7,8,9,10,"car", "boat", "truck", "plane"];
var nonStandardItems =
{
"items": [
{
"value": "8",
"code": "8ic"
},
{
"value": "boat",
"code": "10bt"
}
],
}
if( nonStandardItems.items.slice(-1)[0].value == mainItems.indexOf(nonStandardItems.items.slice(-1)[0].value) ){
console.log("you are right");
}
However, I only get a value of -1, and an error. What am I missing?
EDIT
I am pulling data from a different data sources, all containing junk data. However, the last item in the junk data array is the item I need.
I want to check if the results from my api request match an "expected" or "standard" set of data. For example, my nonStandardItems object is an example of what I get back from the api.
I want to parse it, and check to see if the LAST item in the items array has a value that matches the index of my standard items array. My junk items array contains numeric and text data, but the last item will always have either a numerical or textual code that I expect.
The example was contrived and rushed, so I apologize for the spelling, syntactical and explanation mistakes. The error the console threw was that nonstandardItems.items[-1].value did not exist.

Use filter():
if (nonStandardItems.items.filter(v => mainItems.indexOf(v.code) == v.value).length > 0) {
console.log("you are right")
}

Categories