We Keep Coding

Retriving split data from firebase

// how I get the data
db.collection('Pins').get().then(snapshot => {
snapshot.forEach(pinInfo => {
pinsToMap(pinInfo)
});
});
// trying to set the data
function pinsToMap(pinInfo){
let pinName;
let pinCoOrdsLat;
let pinCoOrdsLong;
let pinToMapInfo;
pinName = doc.data().name
pinCoOrds = doc.data().coOrds
pinToMapInfo = doc.data().Info
Pins.child(Pins.coOrds).set({
coOrds: {
0:this = pinCoOrdsLat,
1:this = pinCoOrdsLong,
}
});
}
I am storing data in my database based off a map pin, I am now trying to use the stored data to create a pin on the map of the same place, how do I query out the coOrds in to pinCoOrdsLat / pinCoOrdsLong as this way doesn't seem to be working

If I correctly understand you question, the following should do the trick:
db.collection('Pins').get().then(snapshot => {
snapshot.forEach(pinInfo => {
pinsToMap(pinInfo)
});
});
// trying to set the data
function pinsToMap(pinInfo) { // IMPORTANT! => pinInfo is a DocumentSnapshot
const pinName = pinInfo.data().name
const pinCoOrds = pinInfo.data().coOrds
const pinToMapInfo = pinInfo.data().Info
//pinCoOrds is a JavaScript Array with two elements
const pinCoOrdsLat = pinCoOrds[0];
const pinCoOrdsLong = pinCoOrds[1];
//Use pinCoOrdsLat and pinCoOrdsLong the way you want, e.g. calling a leaflet method
}
You'll find here the doc for a DocumentSnapshot

What's the most efficient way to store data to multiple refs in firebase?

Let's see the next situation:
If we create an user, we have to create a new client, a new user, and a new, inital project for the user.
db = {
users: {},
clients: {},
projects: {}
};
const usersRef = firebase.database().ref("/users");
const clientsRef = firebase.database().ref("/clients");
const projectsRef = firebase.database().ref("/projects");
To keep the code clean, and separated, we can create three functions:
const newUserToDb = name => {
const newUser = usersRef.push();
newUser.set({name});
};
const newClientToDb = name => {
const newClient = clientsRef.push();
newClient.set({name});
};
const newProjectToDb = name => {
const newProject = projectsRef.push();
newProject.set({name});
};
const createUserToDb = (userName, clientName, projectName) => {
newUserToDb(userName);
newClientToDb(clientName);
newProjectToDb(projectName);
};
To make all the changes in one place, but make the code less separated:
const createUserToDb = (userName, clientName, projectName) => {
const userId = usersRef.push().key;
const clientId = clientsRef.push().key;
const projectId = projectsRef.push().key;
const updates = {};
updates[`/users/${userId}`] = userName;
updates[`/clients/${clientId}`] = clientName;
updates[`/projects/${projectId}`] = projectName;
firebase.database().ref().update(updates);
};
Is there any important difference between the two solutions above? Which is more efficient?

The important difference to the above approach is atomicity. In the first scenario, the individual collection or documents update will succeed or fail without affecting other updates. In the second scenario, all the updates will succeed else none will.
I don't think efficiency is the right term to be used for comparing the above scenarios, its more of the business/use case which will define which one you need to use

The first way seems more separated and explicit which would probably be easier for other developers to understand.

firebase snapshot is not working with node js GCF

I have a firebase database structure in this format
the database is follows this format Ordergroup/groupid/item
I want to get the buyerid for each item once a new group is created in the ordergroup node. So I try this
exports.sendorderemailtoseller = functions.database.ref('/Ordergroup/{pushId}').onCreate((snapshot, context) => {
const parentRef = snapshot.ref.parent;
const ref = snapshot.ref;
const original = snapshot.val();
const buyerid = original.buyerid;
})
I then notice that original only returns the first child and the buyerid comes out as undefined. How can I get a snapshot of all the child in the groupid excluding Ordersummary?

Your variable 'original' is actually getting the whole tree under node 1522509953304, so you will need to iterate over its children to get each buyerid, like below:
exports.sendorderemailtoseller = functions.database.ref('/Ordergroup/{pushId}').onCreate((snapshot, context) => {
const buyerids = [];
snapshot.forEach((item) => {
buyerids.push(item.val().buyerid);
});
console.log(buyerids);
});

Continue on Null Value of Result (Nodejs, Puppeteer)

I'm just starting to play around with Puppeteer (Headless Chrome) and Nodejs. I'm scraping some test sites, and things work great when all the values are present, but if the value is missing I get an error like:
Cannot read property 'src' of null (so in the code below, the first two passes might have all values, but the third pass, there is no picture, so it just errors out).
Before I was using if(!picture) continue; but I think it's not working now because of the for loop.
Any help would be greatly appreciated, thanks!
for (let i = 1; i <= 3; i++) {
//...Getting to correct page and scraping it three times
const result = await page.evaluate(() => {
let title = document.querySelector('h1').innerText;
let article = document.querySelector('.c-entry-content').innerText;
let picture = document.querySelector('.c-picture img').src;
if (!document.querySelector('.c-picture img').src) {
let picture = 'No Link'; } //throws error
let source = "The Verge";
let categories = "Tech";
if (!picture)
continue; //throws error
return {
title,
article,
picture,
source,
categories
}
});
}

let picture = document.querySelector('.c-picture img').src;
if (!document.querySelector('.c-picture img').src) {
let picture = 'No Link'; } //throws error
If there is no picture, then document.querySelector() returns null, which does not have a src property. You need to check that your query found an element before trying to read the src property.
Moving the null-check to the top of the function has the added benefit of saving unnecessary calculations when you are just going to bail out anyway.
async function scrape3() {
// ...
for (let i = 1; i <= 3; i++) {
//...Getting to correct page and scraping it three times
const result = await page.evaluate(() => {
const pictureElement = document.querySelector('.c-picture img');
if (!pictureElement) return null;
const picture = pictureElement.src;
const title = document.querySelector('h1').innerText;
const article = document.querySelector('.c-entry-content').innerText;
const source = "The Verge";
const categories = "Tech";
return {
title,
article,
picture,
source,
categories
}
});
if (!result) continue;
// ... do stuff with result
}
Answering comment question: "Is there a way just to skip anything blank, and return the rest?"
Yes. You just need to check the existence of each element that could be missing before trying to read a property off of it. In this case we can omit the early return since you're always interested in all the results.
async function scrape3() {
// ...
for (let i = 1; i <= 3; i++) {
const result = await page.evaluate(() => {
const img = document.querySelector('.c-picture img');
const h1 = document.querySelector('h1');
const content = document.querySelector('.c-entry-content');
const picture = img ? img.src : '';
const title = h1 ? h1.innerText : '';
const article = content ? content.innerText : '';
const source = "The Verge";
const categories = "Tech";
return {
title,
article,
picture,
source,
categories
}
});
// ...
}
}
Further thoughts
Since I'm still on this question, let me take this one step further, and refactor it a bit with some higher level techniques you might be interested in. Not sure if this is exactly what you are after, but it should give you some ideas about writing more maintainable code.
// Generic reusable helper to return an object property
// if object exists and has property, else a default value
//
// This is a curried function accepting one argument at a
// time and capturing each parameter in a closure.
//
const maybeGetProp = default => key => object =>
(object && object.hasOwnProperty(key)) ? object.key : default
// Pass in empty string as the default value
//
const getPropOrEmptyString = maybeGetProp('')
// Apply the second parameter, the property name, making 2
// slightly different functions which have a default value
// and a property name pre-loaded. Both functions only need
// an object passed in to return either the property if it
// exists or an empty string.
//
const maybeText = getPropOrEmptyString('innerText')
const maybeSrc = getPropOrEmptyString('src')
async function scrape3() {
// ...
// The _ parameter name is acknowledging that we expect a
// an argument passed in but saying we plan to ignore it.
//
const evaluate = _ => page.evaluate(() => {
// Attempt to retrieve the desired elements
//
const img = document.querySelector('.c-picture img');
const h1 = document.querySelector('h1')
const content = document.querySelector('.c-entry-content')
// Return the results, with empty string in
// place of any missing properties.
//
return {
title: maybeText(h1),
article: maybeText(article),
picture: maybeSrc(img),
source: 'The Verge',
categories: 'Tech'
}
}))
// Start with an empty array of length 3
//
const evaluations = Array(3).fill()
// Then map over that array ignoring the undefined
// input and return a promise for a page evaluation
//
.map(evaluate)
// All 3 scrapes are occuring concurrently. We'll
// wait for all of them to finish.
//
const results = await Promise.all(evaluations)
// Now we have an array of results, so we can
// continue using array methods to iterate over them
// or otherwise manipulate or transform them
//
results
.filter(result => result.title && result.picture)
.forEach(result => {
//
// Do something with each result
//
})
}

Try-catch worked for me:
try {
if (await page.$eval('element')!==null) {
const name = await page.$eval('element')
}
}catch(error){
name = ''
}

how to clone the mongoose query object in javascript

I am facing the problem of clone of the mongoose query object .Javascript the copy the one object into another object by call-by-ref but in my project there is scenario i need to copy one object into another object by call-by-value.
var query=domain.User.find({
deleted: false,
role: role
})
var query1=query;
I have the scenario change in the query object is not reflected in query1. I google and try so many way to clone the object but it does't work.The query object is used in another function for pagination and query1 object is used for count query.
1.I used to Object.clone(query1) error Object.clone is not function
2.I used Object.assign(query1) but it does't works fine.
3.I used other so many ways can anybody help me to sort this problem

Alternative solution using merge method:
const query = domain.User.find({
deleted: false,
role: role
}).skip(10).limit(10)
const countQuery = query.model.find().merge(query).skip(0).limit(0)
const [users, count] = await Promise.all([query, countQuery.count()])

you are trying to clone a cursor, but it is not the right approach, you probably just need to create another
like this:
var buildQuery = function() {
return domain.User.find({
deleted: false,
role: role
});
};
var query = buildQuery();
var query1 = buildQuery();

This is work for me:
const qc = sourceQuery.toConstructor();
const clonedQuery = new qc();
This code work in pagination function where sourceQuery passed as parameter and i dont known what models used. Also it work with aggregations and complex queries.
public async paging(
query: mongoose.DocumentQuery<mongoose.Document[], mongoose.Document>,
params,
transformer: any = null
) {
let page = Number(params.page);
if (!page) page = 1;
let page_size = Number(params.count);
if (!page_size) page_size = 100;
const qc = query.toConstructor();
const cq = new qc();
return cq.countDocuments().exec()
.then(async (total) => {
const s = params.sort;
if (s) {
query.sort(s);
}
query.limit(page_size);
query.skip(page_size * (page - 1));
let results = await query.exec();
if (transformer) {
results = await Promise.all(results.map((i) => transformer(i)));
}
const r = new DtoCollection();
r.pages = Math.ceil(total / page_size);
r.total = total;
(r.results as any) = results;
return r;
});
}

Sergii Stotskyi's answer works just fine and is very elegant, except that count is deprecated.
countDocuments or estimatedDocumentCount should be used instead.
However, this causes the error the limit must be positive. We can walk around this by set limit to a large integer.
const query = domain.User.find({
deleted: false,
role: role
}).skip(10).limit(10)
const countQuery = query.model.find().merge(query).skip(0).limit(Number.MAX_SAFE_INTEGER)
const [users, count] = await Promise.all([query, countQuery.countDocuments()])

Since mongoose v6 you can use Query.prototype.clone
E.g. for your code snippet:
const query = domain.User.find({
deleted: false,
role: role
})
const query1 = query.clone();