We Keep Coding

How to save a document with a dynamic id into Cloud Firestore? Always changing

I am using Cloud Firestore as my database
This is my form codes on my webpage that creates a new document into my Cloud Firestore collection called "esequiz". So how do I code it in such a way that it always plus 1 to the number of documents there are in the database? And also set a limit to having the amount of documents inside the database
form.addEventListener('submit', (e) => {
e.preventDefault();
db.collection('esequiz').add({
question: form.question.value,
right: form.right.value,
wrong: form.wrong.value
});
form.question.value = '';
form.right.value = '';
form.wrong.value = '';
});
It currently works but it will show up as an auto generated ID. How do I make it carry on from the numbers, like as my current documents? When i save I would like it to read the current last document id, OR simply count the number of documents, then just + 1
Insight from Andrei Cusnir, counting documents in Cloud Firestore is not supported.
Now I am trying Andrei's approach 2, to query documents in descending order, then using .limit to retrieve the first one only.
UPDATED
form.addEventListener('submit', (e) => {
e.preventDefault();
let query = db.collection('esequiz');
let getvalue = query.orderBy('id', 'desc').limit(1).get();
let newvalue = getvalue + 1;
db.collection('esequiz').doc(newvalue).set({
question: form.question.value,
right: form.right.value,
wrong: form.wrong.value
});
form.question.value = '';
form.right.value = '';
form.wrong.value = '';
});
No more error, but instead, the code below returns [object Promise]
let getvalue = query.orderBy('id', 'desc').limit(1).get();
So when my form saves, it saves as [object Promise]1, which I don't know why it is like this. Can someone advise me on how to return the document id value instead of [object Promise]
I think it is because I did specify to pull the document id as the value, how do I do so?
UPDATED: FINAL SOLUTION
Played around with the codes from Andrei, and here are the final codes that works. Much thanks to Andrei!
let query = db.collection('esequiz');
//let getvalue = query.orderBy('id', 'desc').limit(1).get();
//let newvalue = getvalue + 1;
query.orderBy('id', 'desc').limit(1).get().then(querySnapshot => {
querySnapshot.forEach(documentSnapshot => {
var newID = documentSnapshot.id;
console.log(`Found document at ${documentSnapshot.ref.path}`);
console.log(`Document's ID: ${documentSnapshot.id}`);
var newvalue = parseInt(newID, 10) + 1;
var ToString = ""+ newvalue;
db.collection('esequiz').doc(ToString).set({
id: newvalue,
question: form.question.value,
right: form.right.value,
wrong: form.wrong.value
});
});
});

If I understood correctly you are adding data to the Cloud Firestore and each new document will have as name an incremental number.
If you query all the documents and then count how many are of them, then you are going to end up with many document reads as the database increases. Don't forget that Cloud Firestore is charging per document Read and Write, therefore if you have 100 documents and you want to add new document with ID: 101, then with the approach of first reading all of them and then counting them will cost you 100 Reads and then 1 Write. The next time it will cost you 101 Reads and 1 Write. And it will go on as your database increases.
The way I see is from two different approaches:
Approach 1:
You can have a single document that will hold all the information of the database and what the next name should be.
e.g.
The structure of the database:
esequiz:
0:
last_document: 2
1:
question: "What is 3+3?
right: "6"
wrong: "0"
2:
question: "What is 2+3?
right: "5"
wrong: "0"
So the process will go as follows:
Read document "/esequiz/0" Counts as 1 READ
Create new document with ID: last_document + 1 Counts as 1 WRITE
Update the document that holds the information: last_document = 3; Counts as 1 WRITE
This approach cost you 1 READ and 2 WRITES to the database.
Approach 2:
You can load only the last document from the database and get it's ID.
e.g.
The structure of the database (Same as before, but without the additional doc):
esequiz:
1:
question: "What is 3+3?
right: "6"
wrong: "0"
2:
question: "What is 2+3?
right: "5"
wrong: "0"
So the process will go as follows:
Read the last document using the approach described in Order and limit data with Cloud Firestore documentation. So you can use direction=firestore.Query.DESCENDING with combination of limit(1) which will give you the last document. Counts as 1 READ
Now you know the ID of the loaded document so you can create new document with ID: that will use the loaded value and increase it by 1. Counts as 1 WRITE
This approach cost you 1 READ and 1 WRITE in total to the database.
I hope that this information was helpful and it resolves your issue. Currently counting documents in Cloud Firestore is not supported.
UPDATE
In order for the sorting to work, you will also have to include the id as a filed of the document that so you can be able to order based on it. I have tested the following example and it is working for me:
Structure of database:
esequiz:
1:
id: 1
question: "What is 3+3?
right: "6"
wrong: "0"
2:
id:2
question: "What is 2+3?
right: "5"
wrong: "0"
As you can see the ID is set the same as the document's ID.
Now you can query all the documents and order based on that filed. At the same time you can only retrieve the last document from the query:
const {Firestore} = require('#google-cloud/firestore');
const firestore = new Firestore();
async function getLastDocument(){
let query = firestore.collection('esequiz');
query.orderBy('id', 'desc').limit(1).get().then(querySnapshot => {
querySnapshot.forEach(documentSnapshot => {
console.log(`Found document at ${documentSnapshot.ref.path}`);
console.log(`Document's ID: ${documentSnapshot.id}`);
});
});
}
OUTPUT:
Found document at esequiz/2
Document's ID: 2
Then you can take the ID and increase it by 1 to generate the name for your new document!
UPDATE 2
So, the initial question is about "How to store data in the Cloud Firestore with documents having incremental ID", at the moment you are facing issues of setting up Firestore with you project. Unfortunately, the new raised questions should be discussed in another Stackoverflow post as they have nothing to do with the logic of having incremental IDs for the document and it is better to keep one issue per question, to give better community support for members that are looking for a solution about particular issues. Therefore, I will try to help you, in this post, to execute a simple Node.js script and resolve the initial issue, which is storing to Cloud Firestore documents with incremental IDs. Everything else, on how to setup this in your project and how to have this function in your page, should be addressed in additional question, where you also will need to provide as much information as possible about the Framework you are using, the project setup etc.
So, lets make a simple app.js work with the logic described above:
Since you have Cloud Firestore already working, this means that you already have Google Cloud Platform project (where the Firestore relies) and the proper APIs already enabled. Otherwise it wouldn't be working.
Your guide in this tutorial is the Cloud Firestore: Node.js Client documentation. It will help you to understand all the methods you can use with the Firestore Node.js API. You can find helpful links for adding, reading, querying documents and many more operations. (I will post entire working code later in this steps. I just shared the link so you know where to look for additional features)
Go to Google Cloud Console Dashboard page. You should login with your Google account where your project with the Firestore database is setup.
On top right corner you should see 4 buttons and your profile picture. The first button is the Activate Cloud Shell. This will open a terminal on the bottom of the page with linux OS and Google Cloud SDK already install. There you can interact with your resources within GCP projects and test your code locally before using it in your projects.
After clicking that button, you will notice that the terminal will open in the bottom of your page.
To make sure that you are properly authenticated we will set up the project and authenticate the account again, even if it is already done by default. So first execute $ gcloud auth login
On the prompted question type Y and hit enter
Click on the generated link and authenticate your account on the prompted window
Copy the generated string back to the terminal and hit enter. Now you should be properly authenticated.
Then setup the project that contains Cloud Firestore database with the following command: $ gcloud config set project PROJECT_ID. Now you are ready to build a simple app.js script and execute it.
Create a new app.js file: nano app.js
Inside paste my code example that can be found in this GitHub link. It contains fully working example and many comments explaining each part therefore it is better that it is shared through GitHub link and not pasted here. Without doing any modifications, this code will execute exactly what you are trying to do. I have tested it my self and it is working.
Execute the script as: node app.js
This will give you the following error:
Error: Cannot find module '#google-cloud/firestore'
Since we are importing the library #google-cloud/firestore but haven't installed it yet.
Install #google-cloud/firestore library as follows: $ npm i #google-cloud/firestore. Described in DOC.
Execute the script again: $ node app.js.
You should see e.g. Document with ID: 3 is written.
If you execute again, you should see e.g. Document with ID: 4 is written.
All those changes should appear in your Cloud Firestore database as well. As you can see it is loading the ID of the last document, it is creating a new ID and then it creates a new document with the given arguments, while using the new generated ID as document name. This is exactly what the initial issue was about.
So I have shared with you the full code that works and does exactly what you are trying to do. Unfortunately, the other newly raised issues, should be addressed in another Stackoverflow post, as they have nothing to do with the initial issue, which is "How to create documents with incremental ID". I recommend you to follow the steps and have a working example and then try to implement the logic to your project. However, if you are still facing any issues with how to setup Firestore in your project then you can ask another question. After that you can combine both solutions and you will have working app!
Good luck!

I don't think the way you are trying to get the length of the collection is right and I am entirely not sure what is the best way to get that either. Because the method you are trying to implement will cost you a lot more as you are trying to read all the records of the collection.
But there can be alternatives to get the number you require.
Start storing the ID in the record and make the query with limit 1 and a descending sort on ID.
Store the latest number in another collection and increment that every time you create a new record, And fetch the same whenever needed.
These methods might fail if concurrent requests are being made without transactions.

Webhooks in Bitcoin Transactions - How to Apply Business Logics?

I want to start accepting Bitcoin on my website.
In order to do that, I wrote the following piece of code, but I truly struggle to understand how I can implement proper business logic after that the transaction is completed.
Here is the code:
<html>
<head>
<title>Pay with Bitcoin</title>
<script>
//Gets the URL of the Webpage and gets the price value of this transaction in USD.
//For simplicity Here the Value is passed in the URL.
//However in production you wanna use POST instead of GET.
const myUrl = window.location.href;
const url = new URL(myUrl);
const usdPrice = url.searchParams.get("price");
//This is the function where all the magin happens
const showQR = () => {
//URL of the api which will provide us with current BTC exchange rate
const apiUrl = "https://blockchain.info/ticker";
const hr = new XMLHttpRequest();
hr.open('GET', apiUrl, true);
hr.onreadystatechange = function(){
//Make sure the API sent a valid response
if(hr.readyState == 4){
let ticker = JSON.parse(hr.responseText);
//Get last BTC/USD exchange value from the API , then convert Price from USD to BTC
let BTCprice = ticker.USD.last;
let btcToPay = usdPrice / BTCprice;
//Make sure you have just 8 decimal points in your BTC price!!
btcToPay = btcToPay.toFixed(8);
//Use google API (or other...) to create the QR code. Pass on your btc public address and
//the amount (btc price) dynamically created. Message and label parameters can be dynamic too.
let qrurl = "https://chart.googleapis.com/chart?chs=250x250&cht=qr&chl=bitcoin:1BAnkZn1qW42uRTyG2sCRN9F5kgtfb5Bci?amount="+btcToPay+"%26label=CarRental%26message=BookingID123456";
//Populate the 'btc' DIV with QR code and other info...
document.getElementById('btc').innerHTML = "<img src=" +qrurl+"><br> <span class = 'greenMoney'>" + usdPrice + " usd / " + btcToPay + " BTC </span>";
}
}
hr.send();
};
</script>
</head>
<body onload = "showQR()">
<h1>Pay with BitCoin</h1>
<div id = "btc">
</div>
</body>
</html>
This code does the following:
Gets current USD/BTC exchange rate using the blockchain API.
takes the price in USD for the URL and converts it into BTC
generates a QR code using google API.
Embeds the price, label and message into the QR code
Renders the QR code in a DIV
I ve also set up a web hook service which will be listening to new transactions happening in the specified wallet address. Then a callback to my server is made, by mean of a POST request.
The problem is: the label and message parameters passed to the QR code will not be written in the blockchain.
They are just a handy reference for the customer to remind him what that specific transaction paid for.
As a result the callback to my server is practically useless.
In fact, the callback doesn't return any Booking Id or any other piece of information which could help me to understand who paid for what. Needless to say, in this scenario no business logic is possible: I can't update the order status on my DB, I can't send a confirmation email to the right customer.
How can I embed relevant information (e.g. Booking ID) into the BTC payment, ideally through the QR code?
If this is possible, how can I retrieve this information later on when my server receives the callback informing me that a new payment was made to my BTC wallet?

In short, you can't.
When accepting payments, you are supposed to give each invoice a new BTC address. This way, when you receive notification of an incoming transaction, you can check the receiving address to see which invoice is being paid, and compare the received amount against the expected amount.
Note
Technically, you could embed stuff like a order ID into an OP_RETURN. However, most wallets don't support transactions like that, and any users who want to pay you from an exchange account would be unable to comply.

#Raghav Sood thank you for your input which routed me to the right direction.
Using NodeJS/Express/MongoDB in the backend, I managed to implement a solution which I would like to share here.
Before starting, I wanna make a big disclaimer: this solution is not the only one, it is not the best one, it is not the fastest and probably it is not the most elegant.
Anyway, this solution has the advantage of not relying on packaged third parties solutions. This is in line with the spirit of the whole "no intermediation" philosophy of the bitcoin community. Most imortantly, your XPub always stay in your server and is NOT shared with any external service, which is probably the wisest approach.
Having said that, here is how one can show dynamic unique BTC addresses to customers:
First of all , I put in place a counter which keeps track of how many btc addresses were created for customers from a my HD wallet.
This is important to make sure than you never present the same address twice to customers, which is good for privacy of all parties and also for the sake of implementing business logic in your app.
In order to do this, I store a "counter value" into my DB. Everytime someone visits the BTC payment page, this value is retrived from mongo using a "dealCount" function and is assigned to a "serialPay" variable, which is equal to the value gotten from Mongo + 1. In the backend, the code would be something like this:
`function dealCount(){`
return new Promise(function(resolve, reject){
Deal.find({_id: "ID_OF_OBJ_WHERE_YOU_STORE_COUNTER"}, function(err, data){
if(err){
console.log(err);
}
resolve(data[0].serialDeal + 1);
})
})
};
The new value obtained (which later on will be saved again into Mongo in order to keep track of addresses created) is used to generate the new BTC public address for the customer at hand. If you keep reading you will see how.
To create new public addresses dynamically, one needs the xPub Key of his or her HD Wallet. If one is coding in NodeJS there are a couple of libraries (which can be imported into the server) that will enable this operation rather easily: bitcoinjs-lib and/or bitcore-lib. Personally I opted for Bitcore-lib, because there are less dependencies to deal with and I found the supporting material easier to digest.
Codewise, address generation goes as follows:
const bitcore = require('bitcore-lib');
app.post("/pay.html", urlencodedParser, function(req, res){
let serialPay = dealCount();
serialPay.then(function(serialPay){
const pub = new bitcore.HDPublicKey('INSERT_HERE_YOUR_XPUB_KEY');
let derivedHdPk = pub.derive('m/0/'+serialPay);
let derivedPk = derivedHdPk.publicKey;
let myDynAddress = new bitcore.Address(derivedPk);
res.render('pay', {myDynAddress: myDynAddress});
});
});
Then, using EJS as a templating engine, I could easily make the receiving bitcoin address dynamic in the front-end (/pay.ejs):
let myDynAddress = "<%=myDynAddress%>";
let qrurl = "https://chart.googleapis.com/chart?chs=250x250&cht=qr&chl=bitcoin:"+myDynAddress+"?amount="+btcToPay+"%26label=CarRental";
This will generate the QR Code Dynamically. In the original question, one can see how to render that into the webpage. In the meantime one should also put in place a function to store the updated "serialPay" counter back to the DB.
At this point one should only start monitoring incoming (non-confirmed) payments to the dynamic BTC address generated. A simple way to do it, is using the blockchain.info websocket API. When the payment arrives, things go forward as suggested by #Raghav Sood: one checks the incoming transaction making sure the customer paid the right amount to the right address.
Now you know who paid for what and all sorts of business logics can be triggered.

Improve performance for repetitive Mongo database access task

I'm building a chat bot (using MeteorJS/NodeJS) which interacts with about 2,000 active users everyday. I know the exact number of people who chat with the bot everyday because I store the users active information in a MongoDB collection called ActiveReports.
This is a scenario in my app: if a user A chat with the bot 100 times (= 100 messages) in a day, these steps will be executed:
- receive message from users
- check if this user is marked as 'active' today ? // high cost
- if yes => don't do anything
- if no => mark this user as 'active' for today
As you can see, step 2 is executed for every messages. This step is technically equivalent to accessing the ActiveReports collection, find the one with timestamp = today, user = user A. Since the ActiveReports collection has a lot of documents (about 100,000 documents), this is a fairly heavy task. This negatively affects the app's performance.
NOTE 1: This is the ActiveReports collection schema:
SimpleSchema({
// _id must be set `type` as String and `optional` as true
// to avoid ObjectId(_id) after insert in to database
_id: {
type: String,
optional: true,
},
date: {
type: Date, // Note: date is always the timestamp of the start of the current day, so 1AM timestamp and 9PM timestamp will be changed to 0AM timestamp (before the insert)
},
userId: {
type: String,
},
});
And this is how I indexed this collection:
ActiveReports._ensureIndex({ date: 1, userId: 1 }, { unique: true });
NOTE 2: A user is active in a day means he interacts with the bot at least 1 time (e.g send a message to the bot) that day.
Any ideas how I can improve this ? Please tell me if you need further information. Thank you.

Add a field last_active_date to a User schema and update it every time you get a message. If the date matches today, you are done. If it's not, you need to update the field and add a record to ActiveReports collection.
Actually, it seems to me that you are trying to use Mongo here in a way you would use a relational database. I mean, that there is no need in ActiveReports if you just want to mark a user as active.
If you are trying to build some sort of report for showing app usage per user per day, you can do it in the background. You can have a job that will run once a day (actually, if you have users in different time zones and you want to tolerate their time, you may want to run it few times a day). This job will query the User collection and add records to ActiveReports for each user it finds where last_active_date is current_date.

If you're building a stateless server application the minimum you need to do is pull the user's record to check active.
You might consider having have a daemon task process the ActiveReports and update user dates in the background. That way you only process those records once and the user info is ready to go. Also, that process can have state so it can be more optimal that updating every user for every record.

Restrict number of users in a session in vline

Can I restrict the number of users in a session? Is there any option in vline.session? Please guide if this can be done by writing custom javascript.
EDIT:
Referring to https://vline.com/developer/docs/vline.js/vline.MediaSession#examples, a two party call controller is explained. I want to ask is there any way to restrict number of users in a session? There is no such option present in session's docs. Is it supported as a part of the API?
If this can be done using custom javascript, how?
As a part of my effort, I have tried to implement vline-django examples, but could not find a section in documentation that addresses this issue.
EDIT 2: The code that is working for me.
var vlineClient = (function(){
var client, session,
authToken = {{ user|vline_auth_token|safe }},
serviceId = {% vline_service_id %},
profile = {{ user|vline_user_profile|safe }};
// Create vLine client
window.vlineClient = client = vline.Client.create({"serviceId": serviceId, "ui": true});
// Add login event handler
client.on('login', onLogin);
// Do login
client.login(serviceId, profile, authToken);
function onLogin(event) {
session = event.target;
// Find and init call buttons
var callButtons = document.getElementsByClassName('callbutton');
for (var i=0; i < callButtons.length; ++i) {
initCallButton(callButtons[i]);
}
}
// add event handlers for call button
function initCallButton(button) {
var userId = button.getAttribute('data-userid');
// fetch person object associated with username
session.getPerson(userId).done(function(person) {
// update button state with presence
function onPresenceChange() {
button.setAttribute('data-presence', person.getPresenceState());
}
// set current presence
onPresenceChange();
// handle presence changes
person.on('change:presenceState', onPresenceChange);
// start a call when button is clicked
button.addEventListener('click', function() {
person.startMedia();
});
});
}
return client;
})();
How do I move ahead?
Reference: https://vline.com/developer/docs/vline.js/

if i understand correctly the OP is trying to make a multi-user chat room - this is also what i wanted to do with vline and because i wanted a/v chat as well the number of participants should obviously be capped - it appears that the term 'session' is causing the confusion here so i will refrain from using it
i worked around this by creating a fixed number of users in a db and handling authentication
myself before actually associating a visitor with one of the prepared users - so some javascript logs in each visitor as one of those existing 'anonymous' users and sets only a logged_in? flag in the db so that the next visitor will log in as the next vacant user slot and when all slots are occupied the visitor gets a "chat room full - try again later" response
probably not the most elegant solution - for example the visitor chosen usernames are stored client-side and must be re-assigned to one of the user-definable vline session vars so it can be passed along with each message and the logged_in? db flag needs to be reset when the user exits
note that this was almost a year ago so im a bit foggy on exactly what i did but my app (rails) in up on github if youre interested to fork it - also i should add that although this sort of thing wasnt strictly supported by the vline API at the time there were at least some hints that some analogous feature was being prepared for so there may be some API support for this now - i did notice since then that they have released a "chat room demo" app on github and i would expect that their implementation is more concise than mine so you may want to look at that first - my app tho does have a mostly complete UI with gravatars and collaboration is welcomed

Multi-Page Order Form with sessions

For my web dev class we have to create a login page, verify it against encrypted records (Id, password) that we have to enter, then step through an order form (while being able to step forward and backward throughout).. so sessions and all that.. I have no idea where to even start aside from coding the html which I've already done.. Any pushes in the right direction would be helpful.. my instructor is abrasive and refuses to help most people without degrading them first.

This is kinda like a longer question.
First at login form you need to check with MYSQL / SQL / DB / etc if the username and password matches.
It's basically like this:
SELECT * from users WHERE username = 'username' AND pass = 'sha1(password)'
Or use the encryption method which you use (md5,sha1,any other for password)
Then you check out if it's returning a row. IF it's return 1 row,then everything is correct.
Then you put all this data to session. I don't know how much you need,but you can put the whole sql result to data. IT doesn't matter here as you said it's a dev class work.
So basically at every of your php you have to start with
session_start();
Then when you verified the user you put the sql result into SESSION like this:
$_SESSION['userdata'] = $sql_row_array;
With this data you can read the current loggedin user's informations. So it's like:
Get username: $_SESSION['userdata']['username']
So you can use this to identify whom bought / ordered the products and insert it into the database.