On the browser JS console, News.insert({name: 'Test'}) caused {{count}} to increase from 0 to 1.
In mongo console mrt mongo, db.news.find().count() returns 1. However after adding a record via the mongo console db.news.insert({name: 'TestAgain'}), {{count}} remains at 1 while in mongo, there are 2 records now.
Question: What is causing minimongo and the mongodb console to give inconsistent results?
If I replace Meteor.SmartCollection with Meteor.Collection and reload the page, {{count} is now 2. But if I were to change it back to Meteor.SmartCollection, {{count}} goes back to 1!!
collections/news.js
News = new Meteor.SmartCollection('news');
client/views/main.html
<template name="news">
{{ count }}
</template>
client/views/main.js
Template.news.count = function() {
return News.find().count();
}
Using Meteor v6.6.3 with SmartCollection v0.3.2.2
Update
By Cuberto's suggestion, I have enabled Oplog on my Mongodb server.
export MONGO_URL=mongodb://192.168.1.111:27017/myDb
export OPLOG_URL=mongodb://192.168.1.111:27017/local
mrt
mongod runs with --replSet meteor and mongodb was configured with
var config = {_id: "meteor", members: [{_id: 0, host: "127.0.0.1:27017"}]}
rs.initiate(config)
The prompt in mongo also becomes meteor:PRIMARY> and db.local. does contain the collection oplog.rs.
Starting meteor, we see in the console SmartCollection charged with MongoDB Oplog.
Problem: However, nothing is retrieved when we try to do News.find() in the browser JS console. Doing the same query in mongo client returns the correct result. Switching from Meteor.SmartCollection back to Meteor.Collection allows the site to work again.
How can we troubleshoot the problem with SmartCollection?
Make sure you configure your MongoDB to use oplog and set the environment variables, as explained here:
http://meteorhacks.com/lets-scale-meteor.html
Since smart collections removes the periodic database poll, you need to use an oplog-enabled mongodb instance to make it recognize DB changes from outside meteor.
Related
I am making a giveaway command, but whenever I restart all dynos in heroku it seems the giveaway just froze(Never ends the giveaway) and when I do !gdelete {messageid} It says there is no giveaway for {messageid} any idea why and how to fix it. I have tried using quick.db but still the same and I am quite new to heroku and coding discord bot. Im using node.js
const { GiveawaysManager } = require("discord-giveaways");
const manager = new GiveawaysManager(bot, {
storage: "./giveaways.json",
updateCountdownEvery: 10000,
default: {
botsCanWin: false,
embedColor: "#FF0000",
reaction: "🎉"
}
})
bot.giveawaysManager = manager;
Heres the code
And heres the gstart command: https://pastebin.com/9tBjpVEY
The issue is caused by Heroku, which doesn't store local files when you're not running the app. Every time you restart a dyno Heroku deletes everything and rebuilds it: that means that if you save your files locally when it restarts they'll get deleted.
To solve this issue you need either to switch to another service or to create some form of backup for your file.
You could also use a remote database, but I don't know how that could be implemented with the discord-giveaways package.
I had the same issue and I think that it can be solved by doing this:
Instead of using quick.db, you can use quickmongo which just the same as quick.db and discord-giveaways also has an example of it. Although there is one change that you need to make. The example of quickmongo also shows a local way to store the files but instead of typing the localhost string, replace it with the MongoDB Compass connection string of your MongoDB cluster and give the new collection the same name which is giveaways.
In order to get the connection string, log in to your MongoDB account and create a cluster. After creating the cluster, click the connect button on the cluster and then select Connect using MongoDB Compass. From there you will see a connection string. Copy and paste that string in the place where there was the localhost string. Then replace <password> with your account's password which is your password with your username. Also, replace the test at the end with giveaways and you are good to go. After running the code, you would also see a collection named giveaways in the Collections Tab inside your cluster.
Example:
const db = new Database('connectionLink/giveaways');
db.once('ready', async () => {
if ((await db.get('giveaways')) === null) await db.set('giveaways', []);
console.log('Giveaway Database Loaded');
});
I am using Cloud Firestore as my database
This is my form codes on my webpage that creates a new document into my Cloud Firestore collection called "esequiz". So how do I code it in such a way that it always plus 1 to the number of documents there are in the database? And also set a limit to having the amount of documents inside the database
form.addEventListener('submit', (e) => {
e.preventDefault();
db.collection('esequiz').add({
question: form.question.value,
right: form.right.value,
wrong: form.wrong.value
});
form.question.value = '';
form.right.value = '';
form.wrong.value = '';
});
It currently works but it will show up as an auto generated ID. How do I make it carry on from the numbers, like as my current documents? When i save I would like it to read the current last document id, OR simply count the number of documents, then just + 1
Insight from Andrei Cusnir, counting documents in Cloud Firestore is not supported.
Now I am trying Andrei's approach 2, to query documents in descending order, then using .limit to retrieve the first one only.
UPDATED
form.addEventListener('submit', (e) => {
e.preventDefault();
let query = db.collection('esequiz');
let getvalue = query.orderBy('id', 'desc').limit(1).get();
let newvalue = getvalue + 1;
db.collection('esequiz').doc(newvalue).set({
question: form.question.value,
right: form.right.value,
wrong: form.wrong.value
});
form.question.value = '';
form.right.value = '';
form.wrong.value = '';
});
No more error, but instead, the code below returns [object Promise]
let getvalue = query.orderBy('id', 'desc').limit(1).get();
So when my form saves, it saves as [object Promise]1, which I don't know why it is like this. Can someone advise me on how to return the document id value instead of [object Promise]
I think it is because I did specify to pull the document id as the value, how do I do so?
UPDATED: FINAL SOLUTION
Played around with the codes from Andrei, and here are the final codes that works. Much thanks to Andrei!
let query = db.collection('esequiz');
//let getvalue = query.orderBy('id', 'desc').limit(1).get();
//let newvalue = getvalue + 1;
query.orderBy('id', 'desc').limit(1).get().then(querySnapshot => {
querySnapshot.forEach(documentSnapshot => {
var newID = documentSnapshot.id;
console.log(`Found document at ${documentSnapshot.ref.path}`);
console.log(`Document's ID: ${documentSnapshot.id}`);
var newvalue = parseInt(newID, 10) + 1;
var ToString = ""+ newvalue;
db.collection('esequiz').doc(ToString).set({
id: newvalue,
question: form.question.value,
right: form.right.value,
wrong: form.wrong.value
});
});
});
If I understood correctly you are adding data to the Cloud Firestore and each new document will have as name an incremental number.
If you query all the documents and then count how many are of them, then you are going to end up with many document reads as the database increases. Don't forget that Cloud Firestore is charging per document Read and Write, therefore if you have 100 documents and you want to add new document with ID: 101, then with the approach of first reading all of them and then counting them will cost you 100 Reads and then 1 Write. The next time it will cost you 101 Reads and 1 Write. And it will go on as your database increases.
The way I see is from two different approaches:
Approach 1:
You can have a single document that will hold all the information of the database and what the next name should be.
e.g.
The structure of the database:
esequiz:
0:
last_document: 2
1:
question: "What is 3+3?
right: "6"
wrong: "0"
2:
question: "What is 2+3?
right: "5"
wrong: "0"
So the process will go as follows:
Read document "/esequiz/0" Counts as 1 READ
Create new document with ID: last_document + 1 Counts as 1 WRITE
Update the document that holds the information: last_document = 3; Counts as 1 WRITE
This approach cost you 1 READ and 2 WRITES to the database.
Approach 2:
You can load only the last document from the database and get it's ID.
e.g.
The structure of the database (Same as before, but without the additional doc):
esequiz:
1:
question: "What is 3+3?
right: "6"
wrong: "0"
2:
question: "What is 2+3?
right: "5"
wrong: "0"
So the process will go as follows:
Read the last document using the approach described in Order and limit data with Cloud Firestore documentation. So you can use direction=firestore.Query.DESCENDING with combination of limit(1) which will give you the last document. Counts as 1 READ
Now you know the ID of the loaded document so you can create new document with ID: that will use the loaded value and increase it by 1. Counts as 1 WRITE
This approach cost you 1 READ and 1 WRITE in total to the database.
I hope that this information was helpful and it resolves your issue. Currently counting documents in Cloud Firestore is not supported.
UPDATE
In order for the sorting to work, you will also have to include the id as a filed of the document that so you can be able to order based on it. I have tested the following example and it is working for me:
Structure of database:
esequiz:
1:
id: 1
question: "What is 3+3?
right: "6"
wrong: "0"
2:
id:2
question: "What is 2+3?
right: "5"
wrong: "0"
As you can see the ID is set the same as the document's ID.
Now you can query all the documents and order based on that filed. At the same time you can only retrieve the last document from the query:
const {Firestore} = require('#google-cloud/firestore');
const firestore = new Firestore();
async function getLastDocument(){
let query = firestore.collection('esequiz');
query.orderBy('id', 'desc').limit(1).get().then(querySnapshot => {
querySnapshot.forEach(documentSnapshot => {
console.log(`Found document at ${documentSnapshot.ref.path}`);
console.log(`Document's ID: ${documentSnapshot.id}`);
});
});
}
OUTPUT:
Found document at esequiz/2
Document's ID: 2
Then you can take the ID and increase it by 1 to generate the name for your new document!
UPDATE 2
So, the initial question is about "How to store data in the Cloud Firestore with documents having incremental ID", at the moment you are facing issues of setting up Firestore with you project. Unfortunately, the new raised questions should be discussed in another Stackoverflow post as they have nothing to do with the logic of having incremental IDs for the document and it is better to keep one issue per question, to give better community support for members that are looking for a solution about particular issues. Therefore, I will try to help you, in this post, to execute a simple Node.js script and resolve the initial issue, which is storing to Cloud Firestore documents with incremental IDs. Everything else, on how to setup this in your project and how to have this function in your page, should be addressed in additional question, where you also will need to provide as much information as possible about the Framework you are using, the project setup etc.
So, lets make a simple app.js work with the logic described above:
Since you have Cloud Firestore already working, this means that you already have Google Cloud Platform project (where the Firestore relies) and the proper APIs already enabled. Otherwise it wouldn't be working.
Your guide in this tutorial is the Cloud Firestore: Node.js Client documentation. It will help you to understand all the methods you can use with the Firestore Node.js API. You can find helpful links for adding, reading, querying documents and many more operations. (I will post entire working code later in this steps. I just shared the link so you know where to look for additional features)
Go to Google Cloud Console Dashboard page. You should login with your Google account where your project with the Firestore database is setup.
On top right corner you should see 4 buttons and your profile picture. The first button is the Activate Cloud Shell. This will open a terminal on the bottom of the page with linux OS and Google Cloud SDK already install. There you can interact with your resources within GCP projects and test your code locally before using it in your projects.
After clicking that button, you will notice that the terminal will open in the bottom of your page.
To make sure that you are properly authenticated we will set up the project and authenticate the account again, even if it is already done by default. So first execute $ gcloud auth login
On the prompted question type Y and hit enter
Click on the generated link and authenticate your account on the prompted window
Copy the generated string back to the terminal and hit enter. Now you should be properly authenticated.
Then setup the project that contains Cloud Firestore database with the following command: $ gcloud config set project PROJECT_ID. Now you are ready to build a simple app.js script and execute it.
Create a new app.js file: nano app.js
Inside paste my code example that can be found in this GitHub link. It contains fully working example and many comments explaining each part therefore it is better that it is shared through GitHub link and not pasted here. Without doing any modifications, this code will execute exactly what you are trying to do. I have tested it my self and it is working.
Execute the script as: node app.js
This will give you the following error:
Error: Cannot find module '#google-cloud/firestore'
Since we are importing the library #google-cloud/firestore but haven't installed it yet.
Install #google-cloud/firestore library as follows: $ npm i #google-cloud/firestore. Described in DOC.
Execute the script again: $ node app.js.
You should see e.g. Document with ID: 3 is written.
If you execute again, you should see e.g. Document with ID: 4 is written.
All those changes should appear in your Cloud Firestore database as well. As you can see it is loading the ID of the last document, it is creating a new ID and then it creates a new document with the given arguments, while using the new generated ID as document name. This is exactly what the initial issue was about.
So I have shared with you the full code that works and does exactly what you are trying to do. Unfortunately, the other newly raised issues, should be addressed in another Stackoverflow post, as they have nothing to do with the initial issue, which is "How to create documents with incremental ID". I recommend you to follow the steps and have a working example and then try to implement the logic to your project. However, if you are still facing any issues with how to setup Firestore in your project then you can ask another question. After that you can combine both solutions and you will have working app!
Good luck!
I don't think the way you are trying to get the length of the collection is right and I am entirely not sure what is the best way to get that either. Because the method you are trying to implement will cost you a lot more as you are trying to read all the records of the collection.
But there can be alternatives to get the number you require.
Start storing the ID in the record and make the query with limit 1 and a descending sort on ID.
Store the latest number in another collection and increment that every time you create a new record, And fetch the same whenever needed.
These methods might fail if concurrent requests are being made without transactions.
Exception in defer callback: Error: The Mongo server and the Meteor query disagree on how many documents match your query. Maybe it is hitting a Mongo edge case? The query is: {}
I20180319-12:18:14.344(5.5)? at packages/mongo/oplog_observe_driver.js:881:15
I20180319-12:18:14.345(5.5)? at Object.Meteor._noYieldsAllowed (packages/meteor.js:730:12)
I20180319-12:18:14.345(5.5)? at OplogObserveDriver._publishNewResults (packages/mongo/oplog_observe_driver.js:851:12)
I20180319-12:18:14.345(5.5)? at OplogObserveDriver._runQuery (packages/mongo/oplog_observe_driver.js:758:10)
I20180319-12:18:14.346(5.5)? at OplogObserveDriver._runInitialQuery (packages/mongo/oplog_observe_driver.js:658:10)
I20180319-12:18:14.346(5.5)? at packages/mongo/oplog_observe_driver.js:191:10
I20180319-12:18:14.346(5.5)? at packages/mongo/oplog_observe_driver.js:15:9
I20180319-12:18:14.346(5.5)? at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:26)
I20180319-12:18:14.347(5.5)? at packages/meteor.js:502:25
I20180319-12:18:14.347(5.5)? at runWithEnvironment (packages/meteor.js:1238:24)
This is the full exception I'm getting, it started coming as I update meteor to 1.6.1 from 1.6.0, on change in code just updated my meteor, I can't show the code as it's confidential but can some give me any hint on what part of my code should I look for example : on helpers, meteor call, meteor method, router etc.
Or can any body give me idea why this error came.
In case you haven't found a solution yet:
This bug comes from the handling of undefined values in Mongo and JSON/EJSON.
Because undefined doesn't exist in JSON, the key-value pair gets stripped from the query before being passed to Mongo. This is why the error says The query is: {}.
Meteor have a fix for this in the pipeline for the next version, but in the mean time you can update affected queries to test for undefined and replace it with null
reference: https://github.com/meteor/meteor/issues/9619
I have a large application and I am using Headless Chrome, Selenium and Python to test each module. I want to go through each module and get all the JS console errors produced while inside that specific module.
However, since each module is inside a different test case and each case executes in a separate session, the script first has to login on every test. The login process itself produces a number of errors that show up in the console. When testing each module I don't want the unrelated login errors to appear in the log.
Basically, clear anything that is in the logs right now -> go to the module and do something -> get logs that have been added to the console.
Is this not possible? I tried doing driver.execute_script("console.clear()") but the messages in the console were not removed and the login-related messages were still showing after doing something and printing the logs.
State in 2017 and late 2018
The logging API is not part of the official Webdriver specification yet.
In fact, it's requested to be defined for the level 2 specification. In mid 2017 only the Chromedriver has an undocumented non-standard implementation of that command.
In the sources there's no trace of a method for clearing logs:
The public API Webdriver.get_log()
which references internal Command names
which translate to acutal requests in RemoteConnection
Possible Workaround
The returned (raw) data structure is a dictionary that looks like this:
{
u'source': u'console-api',
u'message': u'http://localhost:7071/console.html 8:9 "error"',
u'timestamp': 1499611688822,
u'level': u'SEVERE'
}
It contains a timestamp that can be remembered so that subsequent calls to get_log() may filter for newer timestamps.
Facade
class WebdriverLogFacade(object):
last_timestamp = 0
def __init__(self, webdriver):
self._webdriver = webdriver
def get_log(self):
last_timestamp = self.last_timestamp
entries = self._webdriver.get_log("browser")
filtered = []
for entry in entries:
# check the logged timestamp against the
# stored timestamp
if entry["timestamp"] > self.last_timestamp:
filtered.append(entry)
# save the last timestamp only if newer
# in this set of logs
if entry["timestamp"] > last_timestamp:
last_timestamp = entry["timestamp"]
# store the very last timestamp
self.last_timestamp = last_timestamp
return filtered
Usage
log_facade = WebdriverLogFacade(driver)
logs = log_facade.get_log()
# more logs will be generated
logs = log_facade.get_log()
# newest log returned only
This thread is a few years old, but in case anyone else finds themselves here trying to solve a similar problem:
I also tried using driver.execute_script('console.clear()') to clear the console log between my login process and the page I wanted to check to no avail.
It turns out that calling driver.get_log('browser') returns the browser log and also clears it.
After navigating through pages for which you want to ignore the console logs, you can clear them with something like
_ = driver.get_log('browser')
I am using meteor to do full stack javascript work.And I want to use mysql instead of mongodb.I find numtel:mysql on github, which is a Reactive MySQL for Meteor.
On the server side
Meteor.publish('test', function(){
let array = liveDb.select(
'select * from tasks',
[ { table: 'tasks' } ]
);
console.log('publish test mysql')
console.log(array);
return array;
});
on the client side
let mysqlData = new MysqlSubscription('test');
console.log('subscribe mysql data at client');
console.log(mysqlData);
console.log(mysqlData.length);
console.log(mysqlData.subscriptionId);
console.log(mysqlData[0]);
console.log(mysqlData[1]);
However, I can not get data on the client.And there is a strange phenomenon.From the logs, I find data of mysqlData.However, mysqlData.length is 0, mysqlData[0] and mysqlData[1] are undefined.
Who can help me?
You might want to call mysqlData.reactive() and check if it is ready by using mysqlData.ready() first.
In case the subscription does not work (it does not update the data automatically), you should make sure that you config your MySQL server correctly following the installation instruction.
For me, I could not make it work when I used capital letter on the database name, you should check it also.