We Keep Coding

Intermittently database search and drop down filter self population code does not work - seemingly randomly - cannot find a pattern

I see that someone has given me a minus 1. I am a 55 year old mother who has no experience. I have many skills but this is not one of them. I am absolutely desperate and have bust myself to get this far. If you cannot help, I accept that, but please do not be negative towards me. I am now crying. Some encouragement would be much appreciated.
I have a page which displays items from a database on a repeater. The code searches the items using several drop down filters, which are populated from the database. Intermittently, seemingly randomly (no pattern is emerging despite extensive testing) the code is failing to populate random drop down filters (one or more of the drop down filters show the default settings rather than those self populated from the database). I discovered this by either repeatedly visiting the page or by repeatedly refreshing the page. Often the code works, then every 3 or 4 times, one or more of the drop down filters shows its default settings rather than those self populated from the database (then the next time it goes wrong, it might be the same or a different one or set of filters which do not work)
This is the code. On this page, there are 3 drop down filters but I have several pages like this, each displaying and searching a different database, with up to 10 drop down filters on each page, and they all have this intermittent problem...
import wixData from "wix-data";
$w.onReady(function () {
$w('#iTitle')
$w('#iCounty')
$w('#iGeog')
$w('#dataset1')
$w('#text102')
});
let lastFilterTitle;
let lastFilterCounty;
let lastFilterGeog;
export function iTitle_change(event, $w) {
filter($w('#iTitle').value, lastFilterCounty, lastFilterGeog);
}
export function iCounty_change(event, $w) {
filter(lastFilterTitle, $w('#iCounty').value, lastFilterGeog);
}
export function iGeog_change(event, $w) {
filter(lastFilterTitle, lastFilterCounty, $w('#iGeog').value);
}
function filter(title, county, geog) {
if (lastFilterTitle !== title || lastFilterCounty !== county || lastFilterGeog !== geog) {
let newFilter = wixData.filter();
if (title)
newFilter = newFilter.eq('title', title);
if (county)
newFilter = newFilter.eq('county', county);
if (geog)
newFilter = newFilter.eq('geog', geog);
$w('#dataset1').setFilter(newFilter)
.then(() => {
if ($w('#dataset1').getTotalCount() ===0) {
$w('#text102').show();
}
else {
$w('#text102').hide();
}
})
.catch((err) => {
console.log(err);
});
lastFilterTitle = title;
lastFilterCounty = county;
lastFilterGeog = geog;
}
}
// Run a query that returns all the items in the collection
wixData.query("Psychologists")
// Get the max possible results from the query
.limit(1000)
.ascending("title")
.distinct("title")
.then(results => {
let distinctList = buildOptions(results.items);
// unshift() is like push(), but it prepends an item at the beginning of an array
distinctList.unshift({ "value": '', "label": 'All Psychologists'});
//Call the function that builds the options list from the unique titles
$w("#iTitle").options = distinctList
});
function buildOptions(items) {
return items.map(curr => {
//Use the map method to build the options list in the format {label:uniqueTitle, valueuniqueTitle}
return { label: curr, value: curr };
})
}
// Run a query that returns all the items in the collection
wixData.query("Psychologists")
// Get the max possible results from the query
.limit(1000)
.ascending("county")
.distinct("county")
.then(results => {
let distinctList = buildOptions(results.items);
// unshift() is like push(), but it prepends an item at the beginning of an array
distinctList.unshift({ "value": '', "label": 'All Counties'});
//Call the function that builds the options list from the unique titles
$w("#iCounty").options = distinctList
});
function buildOptions1(items) {
return items.map(curr => {
//Use the map method to build the options list in the format {label:uniqueTitle1, valueuniqueTitle1}
return { label: curr, value: curr };
})
}
// Run a query that returns all the items in the collection
wixData.query("Psychologists")
// Get the max possible results from the query
.limit(1000)
.ascending("geog")
.distinct("geog")
.then(results => {
let distinctList = buildOptions(results.items);
// unshift() is like push(), but it prepends an item at the beginning of an array
distinctList.unshift({ "value": '', "label": 'All Regions'});
//Call the function that builds the options list from the unique titles
$w("#iGeog").options = distinctList
});
function buildOptions2(items) {
return items.map(curr => {
//Use the map method to build the options list in the format {label:uniqueTitle2, valueuniqueTitle2}
return { label: curr, value: curr };
})
}
export function button45_click(event, $w) {
//Add your code for this event here:
filter($w('#iTitle').value='', $w('#iCounty').value='', $w('#iGeog').value='');
}
My experience and knowledge is very limited, so the answer may well be very simple. Any help would be much appreciated as I will have to abandon my project if I can't find a solution.Thank you

Each click triggers different API call

I have list and each clicked item triggers different API request. Each request have different duration. On success I'm displaying some data.
Issue is that when I click on item#1 which takes approx 6000 to load, and just after on item#2 which takes 2000 to load, I will have the last clicked item displayed - which is item#2 because it has already loaded and once item#1 has received data my data will change to that. This is wrong as I want to display data from the latest click.
This is how I handle event:
newList.on('click', 'li', (e) => {
let id = $(e.currentTarget).data("id");
store.getCharacterDetails(id).then(docs => {
this.clearDetails();
this.charDetails = docs;
this.displayDetails(this.charDetails);
})
My API is a simulation from store object.
I suppose this works as expected but I do want the last triggered request to be valid.

A crude and simple method can be creating an array and pushing the IDs and after the asynchronous operations you can just check if it is the latest click or not. But pitfall is that if clear and displayDetails takes much time and if someone click while it was clearing and displaying it will not register the latest click.
Anyway, here is the code maybe you can make something better out of it.
var latestClick = [];
newList.on('click', 'li', (e) => {
let id = $(e.currentTarget).data("id");
latestClick.push(id);
store.getCharacterDetails(id).then(docs => {
if(id === latestClick[latestClick.length - 1]){
this.clearDetails();
this.charDetails = docs;
this.displayDetails(this.charDetails);
latestClick = [];
}
})
})

Make charDetails an object that keeps all of the results, keyed by the ids. Keep track of the last clicked id.
// in constructor
this.charDetails = {};
this.lastId = null;
newList.on('click', 'li', (e) => {
let id = $(e.currentTarget).data("id");
this.lastId = id;
if (this.charDetails[id] === id) { // don't cancel requests, cache them!
this.displayDetails(this.charDetails[id])
} else {
store.getCharacterDetails(id).then(docs => {
// this runs later, cache the result
this.charDetails[id] = docs;
if (id === lastId) { // only update UI if the id was last clicked
this.displayDetails(docs)
}
});
}
});

Necessary to pass limit to data when subscription already limits the dataset?

In a example about pagination, different number is passed to the route in terms of how many results should be displayed.
// lib/router.js
Router.route('/:postsLimit?', {
name: 'postsList',
waitOn: function() {
var limit = parseInt(this.params.postsLimit) || 5;
return Meteor.subscribe('posts', {sort: {submitted: -1}, limit: limit});
},
data: function() {
var limit = parseInt(this.params.postsLimit) || 5;
return {
posts: Posts.find({}, {sort: {submitted: -1}, limit: limit})
};
}
});
I am wondering if it's necessary to still give the Posts.find() arguments in the data attribute. It seems kinda redundant here because the subscribe in waitOn already limit the dataset that i am getting back from the server. I know that data is used to provide data context for different parts of my templates. However, providing the same arguments here to data Posts.find() just seems redundant. I have tried just using Posts.find() without the argument and it worked.
Also, I am wondering what is a way to access all the data in router? I am accessing the Posts collections outside the individual route and I thought that I would be able to access all the data in Posts collections. However, it returned with a count 0.
// lib/router.js
var totalPostsCount = Posts.find().count()
console.log('total count?')
console.log(totalPostsCount) // output 0
PostsListController = RouteController.extend({
template: 'postsList',
increment: 5,
postsLimit: function() {
return parseInt(this.params.postsLimit) || this.increment;
},
findOptions: function() {
return {sort: {submitted: -1}, limit: this.postsLimit()};
},
subscriptions: function() {
this.postsSub = Meteor.subscribe('posts', this.findOptions());
},
posts: function() {
// console.log("is find limited to subscriptions too??");
// console.log(Posts.find().count())
return Posts.find({}, this.findOptions());
},
data: function() {
// console.log('is data limited to this?')
// console.log(Posts.find().count())
var hasMore = this.posts().count() === this.postsLimit();
var adjustOrNot = this.posts()
if (!hasMore){
var nextPath = this.route.path({postsLimit: this.posts().count()});
}else{
var nextPath = this.route.path({postsLimit: this.postsLimit() + this.increment});
}
return {
posts: this.posts(),
ready:this.postsSub.ready(),
nextPath: hasMore ? nextPath : null
};
}
});
//...
Router.route('/:postsLimit?', {
name: 'postsList'
});
Edit 1: code for getting all posts:
// server
Meteor.publish('allPosts',function(){
return Posts.find({});
})
// lib/router.js
if (Meteor.isClient){
var handle = Meteor.subscribe('allPosts');
if (handle.ready()) {
// This logs the actual count of posts
console.log(Posts.find().count());
} else {
console.log('not ready yet');
}
}
This only outputs 'not ready yet' in the console and it's not changing even when the page finish loading.
Thanks a lot for the help.
Edit 2: possible solutions
I tried wrapping the reactive source ready() inside a computation such as Tracker.autorun() and now it worked.
if (Meteor.isClient){
var handle = Meteor.subscribe('allPosts');
Tracker.autorun(function(){
var status = handle.ready();
if (status){
console.log(Posts.find().count())
}else{
console.log('not ready yet')
}
})
}

Yes, you should pass limit and other query options to the front-end query too, regardless of your subscription. This is because if you were to have multiple subscriptions to the same collection active at the same time (as is often done in larger apps) the subscribed records end up all in the same Minimongo collection in the front-end. In this case, if you had omitted the query parameters, you would end up with unpredictable results. See more in this excellent explanation. Note also that while iron:router supports doing subscriptions in the route handlers, it's encouraged to handle them in the template lifecycle methods.
For your second question, the reason the Posts collection appears empty is that the code in the top level of the file is run immediately as the file is loaded. At this point, the collection subscription isn't loaded yet, so the front-end collection is empty. You need to check for subscription readiness in some reactive context, such as a template helper or Tracker.autorun, to ensure that the data is loaded:
var handle = Meteor.subscribe('posts');
if (handle.ready()) {
// This logs the actual count of posts
console.log(Posts.find().count());
} else {
// This logs 0 (unless there are other subscriptions to this collection)
console.log(Posts.find().count());
}

RethinkDB: Determine if there is a next document in result set

With the RethinkDB JavaScript driver, is there a way to determine if there is another document available while within a "cursor.each" or a "cursor.on('data')" event?
Via each
cursor.each(function(err, doc) {
if (err) throw err;
// process document here
// Is there a way to tell if there is another doc coming?
}, function() {
// End of data here, no document passed
})
);
or via Event Emitter
cursor.on('data', function(doc) {
// handle doc, check if another doc is coming or not
});
cursor.on('end', function() {
// no document passed, just indicating end of data
})

There are a couple of ways you could go about this. Keep in mind that these would only work if you're not currently in changefeed.
1. Using cursor.toArray()
You could first convert the cursor into an array and then just use the index to determine if there is another row on the cursor.
r
.table('hello')
.filter({ name: 'jorge' })
.run(conn)
.then(function (cursor) {
return cursor.toArray();
})
.then(function (array) {
array.forEach(function (row, i) {
if (i === array.length - 1) {
// There are now more rows after this
} else {
// There are more rows
}
});
});
As coffeemug mentioned above, this approach won't work for large datasets since converting the cursor into an array means that you have to load all the data.
2. Using cursor.next
A less convenient way of doing it, but probably more performant is to use cursor.next. The way this would work is that the .next method on the cursor would throw an error if there are no more rows in the cursor.
Your code would look something like this:
r
.table('hello')
.filter({ name: 'jorge' })
.run(conn)
.then(function (cursor) {
var hasNextRow = function (cb, prevRow) {
cursor.next(function (err, row) {
if (err) cb(false, prevRow);
if (row) cb(true, prevRow, row);
})
};
var consoleRow = function (row) {
hasNextRow(function (hasNextRow, prevRow, row) {
if (prevRow) {
if (!hasNextRow) console.log('isLast');
// This is your row
console.log(i, prevRow);
}
if (hasNextRow) {
consoleRow(row);
}
}, row);
};
consoleRow();
});
Basically, this code saves a reference to the last row and goes on to the next row. At that point, it'll know if the next row exists or not and you can handle any behavior on that row using the prevRow variable.

The each cursor method is really meant to process the elements independently. If you need to do something more sophisticated, I'd suggest using next (http://rethinkdb.com/api/javascript/next/) -- see the examples for details on how to use the method.
If your dataset is short, you can also call cursor.toArray() and just get the array of elements.

Meteor "counts-by-room" example causing template not to load

I used that "counts-by-room" example from Meteor's docs to count the number of "reviews" for an "entry" on my project. Now, when I click through to an entry, the template isn't loading. It requires a refresh. Did I mess the example up somewhere? Here's my code. I don't really understand the "initializing" part of this, and don't know what to play with to get it to work correctly.
Server:
Meteor.publish("counts-by-entry", function (entryId) {
var self = this;
check(entryId, String);
var count = 0;
var initializing = true;
var handle = Reviews.find({entry: entryId}).observeChanges({
added: function (id) {
count++;
if (!initializing)
self.changed("counts", entryId, {count: count});
},
removed: function (id) {
count--;
self.changed("counts", entryId, {count: count});
}
});
initializing = false;
self.added("counts", entryId, {count: count});
self.ready();
self.onStop(function () {
handle.stop();
});
});
Client (iron-router):
this.route('entry', {
path: '/entry/:_id',
layoutTemplate: 'layout',
yieldTemplates: {
'navigation': {to: 'navigation'},
'pt_entry': {to: 'content'}
},
waitOn: function () {
Meteor.subscribe('singleEntry', this.params._id),
Meteor.subscribe('entryReviews', this.params._id),
Meteor.subscribe('counts-by-entry', this.params._id);
},
data: {
singleEntry: function () {return Entries.findOne()},
reviews: function () {return Reviews.find({entry: Session.get('entryId')}, {sort: {date: -1}})},
count: function () {return Counts.findOne(Session.get("entryId")).count + " reviews."}
},
before: function() {
Session.set("activeNav", Router.current().route.name),
Session.set("entryId", this.params._id);
},
notFoundTemplate: 'notFoundTemplate'
});
and also:
Counts = new Meteor.Collection("counts");

I just tried recreating the minimum amount required of your example to get it to work and I don't have the same issue as you.
Do you have the javascript console open in your browser? I would look for an error there, usually this kind of stuff happens when a helper is called upon when the collection data isn't available to the client. This is what iron-router's waitOn fixes which you have made use of.
In my reproduction I only have the one subscription (counts-by-entry) so maybe there is a issue with the other ones.
As for what the initializing part does:
The publish block is a piece of code that will be run for each client subscription. It does 2 things, it provides the client with the initial payload of data which in a traditional publication would be all the documents from a collection query then it reacts to changes that affects the result of the query and sends just those changes to the client.
Here is the most common publication you will see:
Meteor.publish("all-reviews", function() {
return Reviews.find({});
});
Meteor hides the complexities of what is really going on in this publication. This is closer to what is really going on:
Meteor.publish("all-reviews", function() {
var self = this;
//this is the query we want realtime updates for
//when relevant additions, removals or changes happen the correct callbacks will fire...
var handle = Reviews.find({}).observeChanges({
added: function(id, fields) {
//when a document is added it gets sent to the client.
//Note: the initial payload of data happens here, lets say you had 5 reviews
//this would get called 5 times as soon as a user subscribes.
self.added("reviews", id, fields);
},
removed: function(id) {
//when a document is removed the client is told which one
self.removed("reviews", id);
},
changed: function(id, fields) {
//when a document has a change made to its fields the changes get sent
self.changed("reviews", id, fields);
}
});
//letting the client know that the initial payload of data has been sent.
//Stuff like iron-routers waitOn would rely on this message before loading a template
//that relies on this publication
self.ready();
//stops the publication running forever. This will fire when a client no longer needs a
//subscription or when they close the page.
self.onStop(function() {
handle.stop();
});
});
As for what it going on in the docs example with the initializing flag. The initializing flag is used as a way of simply counting all the initial payload of existing reviews in your case then after the observeChanges call telling the client how many there are. This is an optimisation on the other way of doing it which would be to send the client several self.changed messages during the initial payload.
Maybe it will make more sense if I show how it would be done without the flag:
Meteor.publish("counts-by-entry", function (entryId) {
var self = this;
check(entryId, String);
var count = 0;
//we need to initially add the document we are going to increment
self.added("counts", entryId, {count: 0});
var handle = Reviews.find({entry: entryId}).observeChanges({
added: function (id) {
count++;
//So for example if there were 100 reviews for this entry when the user
//subscribes this self.changed would get called 100 times:
//self.changed("counts", entryId, {count: 1})
//self.changed("counts", entryId, {count: 2})
//self.changed("counts", entryId, {count: 3})
//self.changed("counts", entryId, {count: 4}) etc...
//which is a waste when we can just initially tell the client how
//many reviews there are at the point of them subscribing
self.changed("counts", entryId, {count: count});
},
removed: function (id) {
count--;
self.changed("counts", entryId, {count: count});
}
});
//remove the self.added(...) which handles the optimiation as explained above
self.ready();
self.onStop(function () {
handle.stop();
});
});
Regardless it doesn't look like that particular publish is the problem. I would expect the console to make it clear what the issue is

waitOn should return an array with the subscription handles.