Below is my cloud function for taking Firestore export to a GCP bucket. I am trying to pass the CollectionIDs name through 'collectionsToBackup' const which I am creating by appending the current_date to the name as my subcollection names are of the format: sub_collection_name_yyyymmdd. The function deploys however when its executed, it does not take the export of my subcollection: most_valuable_items_yyyymmdd for that day. Could someone help me please identify what could be wrong with my function?
Code:
const firestore = require('#google-cloud/firestore');
const client = new firestore.v1.FirestoreAdminClient();
const bucket = 'gs://BUCKET_NAME'
const date_ob = new Date();
const date = ("0" + date_ob.getDate()).slice(-2);
const month = ("0" + (date_ob.getMonth() + 1)).slice(-2);
const year = date_ob.getFullYear();
const new_date= (year + month + date);
const collectionsToBackup= ("'" + "most_valuable_items" + "_" + new_date + "'");
exports.scheduledFirestoreBackup = (event, context) => {
const databaseName = client.databasePath(
// process.env.GCLOUD_PROJECT,
"f-2",
'(default)'
);
return client
.exportDocuments({
name: databaseName,
outputUriPrefix: bucket,
collectionIds: [collectionsToBackup]
})
.then(responses => {
const response = responses[0];
console.log(`Operation Name: ${response['name']}`);
return response;
})
.catch(err => {
console.error(err);
});
};
Your problem seems related to the querying and return of information with Firestore. As explained in this other similar case from the Community here, you will need to consider the limit of the queries in Firestore.
This limit is related to the possibility and functions that would allow you to query the subcollections, so they are sent to your bucket. Querying in Firestore is a shallow process, where, unfortunately, you can't return the whole collection and its own subcollections at once. It will be needed for you to read every subcollection, so you have its data as well as the parent collection.
So, to summarize, it's not about your code being wrong, but about changing the way you are reading your data - both collections and subcollections - so you load the data and then, send it over to the bucket. This answer here clarifies more on possibilities.
Let me know if the information clarified your doubts!
Related
I've following code snippet to get the "Controller" (The owner of the domain) but I need to get the "Registrant" of provided ENS name
const Web3 = require("web3")
const web3 = new Web3("https://cloudflare-eth.com");
var ens = web3.eth.ens;
var names = ['jtimberlake.eth', 'usman.eth'];
(async () => {
for (let domainName of names) {
// console.log('checking: ' + domainName);
const addr = await getDomain(domainName);
console.log(addr);
}
})();
async function getDomain(word) {
try {
const addr = await ens.getAddress(`${word}`)
// console.log(addr);
return addr;
} catch (err) {
console.error(err);
return;
}
}
Can you please guide how I can get the "Registrant" of provided ENS name e.g. jtimberlake.eth
Web3 is a steaming pile. It doesn't do it with its methods. The registrant used to be called the deed owner, and the controller the owner. Now it is registrant and controller. That's why the method name makes no sense at all now in Web3.js - it never got updated, and never was useful for this in the first place.
The good news is there is a simple way. You can derive the token ID of the ENS domain from its name with the getRegistrant function below. https://docs.ens.domains/dapp-developer-guide/ens-as-nft
The name variable in the docs is superfluous and does nothing. You will need to instantiate ethersjs (npm install ethers) to get the ethers methods to work. You have to use this crazy number of functions because the token ID of an ENS domain/NFT is a uint256. JavaScript hates those natively.
The web3 method to find the controller also still works well if you ever need that. I suggest putting it in another function.
const getRegistrant = (domainName) => {
const BigNumber = ethers.BigNumber
const utils = ethers.utils
const labelHash = utils.keccak256(utils.toUtf8Bytes(domainName))
const derivedTokenId = BigNumber.from(labelHash).toString()
//You need to instantiate the ENSRegistrarContract with its ABI and address. e.g. const ENSRegistrarContract = new web3.eth.Contract(ABI, ADDRESS)
ENSRegistrarContract.methods.ownerOf(derivedTokenId).call()
.then(function(registrant) {
console.log(domainName + "is owned by: " + registrant)
return registrant
})
}
const getController = (domainName) => {
//getOwner fetches the controller of a domain confusingly.
web3.eth.ens.getOwner(domainName).then(function(controller) {
console.log(domainName + "is controlled by: " + controller)
return controller
})
}
Hey im trying to figuring out if i have a firebase function and inside 4 get methods like this
export default functions
.runWith(runtimeOpts)
.firestore.document("ParentComment/{commentID}")
.onCreate(async (snap, context) => {
try {
const db = admin.firestore();
const { videoID, author, text,videoowner } = snap.data() || {};
const ref = db.doc(`videos/${videoID}`);
const videouid = await db.doc(`videos/${videoID}`).get();
const userSnap = await db.doc(`meinprofilsettings/${author}`).get();
const { commentID } = context.params;
const recoveringuser1 = await db.doc(`Recovering/${videoowner}`).get();
const recoveringuser2 = await db.doc(`Recovering/${author}`).get();
…
do i have to pay for each get method?
or does it count as 1 read?
The pricing elements for Firestore in play here are:
Document Reads: as the name implies, you will be charged for each document that your code causes to be read from the database on the server. Since you are reading 4 individual documents, there are 4 charged document reads
Network Bandwidth: you may also be charged for the bandwidth that is consumed for transferring the data. If the Cloud Function and Firestore are in the same region, this charge is waved.
For more on this, I recommend reading the Firestore pricing page in the documentation.
I'm developing an AWS Lambda in TypeScript that uses Axios to get data from an API and that data will be filtered and be put into a dynamoDb.
The code looks as follows:
export {};
const axios = require("axios");
const AWS = require('aws-sdk');
exports.handler = async (event: any) => {
const shuttleDB = new AWS.DynamoDB.DocumentClient();
const startDate = "2021-08-16";
const endDate = "2021-08-16";
const startTime = "16:00:00";
const endTime = "17:00:00";
const response = await axios.post('URL', {
data:{
"von": startDate+"T"+startTime,
"bis": endDate+"T"+endTime
}}, {
headers: {
'x-rs-api-key': KEY
}
}
);
const params = response.data.data;
const putPromise = params.map(async(elem: object) => {
delete elem.feat1;
delete elem.feat2;
delete elem.feat3;
delete elem.feat4;
delete elem.feat5;
const paramsDynamoDB = {
TableName: String(process.env.TABLE_NAME),
Item: elem
}
shuttleDB.put(paramsDynamoDB).promise();
});
await Promise.all(putPromise);
};
This all works kind of fine. If the test button gets pushed the first time, everything seems fine and is working. E.g. I received all the console.logs during developing but the data is not put into the db.
With the second try it is the same output but the data is successfully put into the Db.
Any ideas regarding this issue? How can I solve this problem and have the data put into the Db after the first try?
Thanks in advance!
you need to return the promise from the db call -
return shuttleDB.put(paramsDynamoDB).promise();
also, Promise.all will complete early if any call fails (compared to Promise.allSettled), so it may be worth logging out any errors that may be happening too.
Better still, take a look at transactWrite - https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html#transactWrite-property to ensure all or nothing gets written
I'm trying to pull an array of data from a MongoDB database, and while the code is rusty (and I do want some corrections on it if it could be done better or is missing something or is wrong), it should be taking the array, finding the "user" and "description" objects, and then putting them into a discord.js message.
I've tried referencing the objects individually, making them strings, parsing the data, but I still cant find out how to do it. Heres the code I've been using.
module.exports.run = async (bot, message, args) => {
const MongoClient = require('mongodb').MongoClient;
const url = 'mongodb+srv://something:something#something/test?retryWrites=true&w=majority';
const assert = require('assert');
try {
function remindersChecker() {
let mongoClientPromise = MongoClient.connect(url, function (err, client) {
const db = client.db("reminders");
let date = new Date().getTime();
let now = Math.floor(Date.now() / 1000);
db.collection("reminders").find().toArray(function (err, result) {
let data = JSON.toObject();
console.log(data);
let user = data.author
let description = data.description
console.log(user);
user.send("I have a reminder for you! " + description)
})
})
}
remindersChecker()
} catch(err) {
catchError()
}
}}
module.exports.help = {
name: "check"
}
(The command is temporary and will be used on a setTimeout later, hence the function rather than just plain old code.)
Thanks! And I hope I can get help soon.
probably some more information would be great to better understand the problem.
from what i can see here, you are receiving an object from your database and converting it into an array here:
db.collection("reminders").find().toArray(function (err, result) {...
now that array is actually that result obtained from the callback and you are not using it at all, you probably have to iterate on that.
plus i remember that I used to write
...find({})..
to search in the database as for SELECT*FROM in SQL. maybe that can help too.
hope this helps.
I am working on an Android Application and am using firebase as the back end for it. I am trying to get a notification system working that relies on listening to changes in the database. Having problems though as I am getting the following error. Wondered if anyone would be able to help, any extra code can be supplied.
Firebase Function
'use-strict'
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.sendNotification = functions.database.ref("Notifications/{user_id}/{notification_id}").onWrite((data, context)=>{
const user_id = context.params.user_id;
const notification_id = context.params.notification_id;
console.log("User ID : " + user_id + " | Notification ID : " + notification_id);
return admin.database().ref("Notifications/" + user_id + "/" + notification_id).get().then(queryResult => {
const job_id = queryResult.data().Job;
const from_data = admin.database().ref("Jobs/" + job_id).get();
const to_data = admin.database().ref("Users/" + user_id).get();
return Promise.all([from_data, to_data]).then(result => {
const job_name = result[0].data().advertName;
const to_name = result[1].data().fullName;
console.log("JOB : " + job_name + " | TO : " + to_name);
return undefined;
});
});
});
Function Error
It looks like you're mixing up the Realtime Database and Firestore APIs. Firebase Realtime Database APIs doens't provide a get() method to fetch data. For that, you use once() on a Reference object.
The error is pretty explicit: there is no get function in the a Firebase Realtime Database.
You seem to be trying to use Cloud Firestore, which is available under admin.firestore(). So something like:
const from_data = admin.firestore().doc("Jobs/" + job_id).get();
const to_data = admin.firestore().doc("Users/" + user_id).get();
For full samples see the NODE.JS tab in the Getting started and other Firestore documentation.