How can I use firebase properly in Nodejs for storage? I used firebase in my react app but due to some reason I switch to the backend for storage can anyone can help me with how can I do that properly? I gives me some errors when I try to do the same as I did with my react app I want it to function the same as my react app did I also have a firebase setup in my Backend.
How I did with React App
const photoHandler = async () => {
if (!photo.cancelled) {
const response = await fetch(photo);
console.log(response)
const blob = await response.blob();
const filename = photo.substring(photo.lastIndexOf('/') + 1);
const ref = firebase.storage().ref().child(filename);
const snapshot = await ref.put(blob);
const url = await snapshot.ref.getDownloadURL();
console.log(url)
}
}
How I tried to do with Nodejs:
const photoHandler = async (photo) => {
if (!photo.cancelled) {
const response = await fetch(photo);
console.log(response)
const blob = await response.blob();
const filename = photo.substring(photo.lastIndexOf('/') + 1);
const ref = firebase.storage().ref().child(filename);
const snapshot = await ref.put(blob);
const url = await snapshot.ref.getDownloadURL();
console.log(url)
}
}
In Nodejs I am passing photos in my route in which I am calling the photo handler function
like this photoHandler(photo); and that photo is a URL
Related
I expect when I call an async function to resolve promise at the end, not before.
const urls = await uploadImages({ imageChanges, questions });
// ...next step
// I will use urls
But after calling await uploadImages() it continues to run until const data = await fetch(image.src);
And then ...next step starts. How can I make it wait for imageChanges.forEach loop finish ? Should I create another nested function inside ?
const uploadImages = async ({ imageChanges, questions }) => {
if (!imageChanges.length) return null;
const storage = firebase.storage();
let urls;
try {
//** convert each new image's src from blob to downloadUrl. */
imageChanges.forEach(async image => {
const questionId = questions.findIndex(q => q.id === image.questionId);
const imagePath = `${questionId}.jpg`;
const storageRef = storage.ref(imagePath);
// **
const data = await fetch(image.src);
const blob = await data.blob();
const uploadTaskSnapshot = await storageRef.put(blob);
const downloadURL = await uploadTaskSnapshot.ref.getDownloadURL();
urls.push(downloadURL)
});
return urls;
} catch (error) {
console.log(error.message);
}
};
forEach with async doesn't work as expected. Read this answer for more info.
Try like this
const uploadImages = async ({ imageChanges, questions }) => {
if (!imageChanges.length) return null;
const storage = firebase.storage();
try {
const imageChangesUrlPromise = imageChanges.map(async () => {
const questionId = questions.findIndex(q => q.id === image.questionId);
const imagePath = `${questionId}.jpg`;
const storageRef = storage.ref(imagePath);
const data = await fetch(image.src);
const blob = await data.blob();
const uploadTaskSnapshot = await storageRef.put(blob);
const downloadURL = await uploadTaskSnapshot.ref.getDownloadURL();
return downloadURL;
})
return await Promise.all(imageChangesUrlPromise);
} catch (error) {
console.log(error.message);
}
};
and then
const urls = await uploadImages({ imageChanges, questions });
...
JavaScript does this because forEach is not promise-aware. It cannot support async and await. You cannot use await in forEach.
If you use await in a map, map will always return an array of promises. This is because asynchronous functions always return promises.
By littile modification to your code, this should work,
const uploadImages = async ({ imageChanges, questions }) => {
if (!imageChanges.length) return null;
const storage = firebase.storage();
let urls;
try {
//** convert each new image's src from blob to downloadUrl. */
await Promise.all(imageChanges.map(async image => {
const questionId = questions.findIndex(q => q.id === image.questionId);
const imagePath = `${questionId}.jpg`;
const storageRef = storage.ref(imagePath);
// **
const data = await fetch(image.src);
const blob = await data.blob();
const uploadTaskSnapshot = await storageRef.put(blob);
const downloadURL = await uploadTaskSnapshot.ref.getDownloadURL();
urls.push(downloadURL)
}));
return urls;
} catch (error) {
console.log(error.message);
}
};
const urls = await uploadImages({ imageChanges, questions });
I am trying to get the message ID of a newly created message like this.
What would I use instead of message.channel.fetch(id) to fetch a message id? I can't seem to find it:
async function messageSend() {
let messageId = await message.channel.send('Test');
let { id } = messageId;
const emojiCheck = '✅';
const emojiNo = '❌';
const nebzlaID = '569860318608490496';
message.channel.messages.fetch(id).react(emojiCheck);
message.channel.messages.fetch(id).react(emojiNo);
const suggestionDM = new Discord.MessageEmbed()
.setColor('#fffff')
.setTitle('There is a new suggestion in abc!').setDescription(`
User: ${suggestUser}
\n\nSuggestion: ${suggestion}
`);
// ...
channel.messages.fetch() returns a promise, it means you need to resolve it first to get the message you can react to. You can use await:
async function messageSend() {
const { id } = await message.channel.send('Test');
const emojiCheck = '✅';
const emojiNo = '❌';
const nebzlaID = '569860318608490496';
const messageToReact = await message.channel.messages.fetch(id);
messageToReact.react(emojiCheck);
messageToReact.react(emojiNo);
// ...
Although it's not necessary to fetch it again, you could simply react to the returned message:
async function messageSend() {
const sentMessage = await message.channel.send('Test');
const emojiCheck = '✅';
const emojiNo = '❌';
const nebzlaID = '569860318608490496';
sendMessage.react(emojiCheck);
sendMessage.react(emojiNo);
// ...
I'm uploading images to firebase storage and I don't have problems with that, but the reference that I have, return the names of the images in an array called imagesBlob. I want to return the download URL of every image after upload and put it into that imagesBlob array.
How can I do that??
Here is my function:
const uploadImagesStorage = async (imageArray) => {
const imagesBlob = [];
await Promise.all(
imageArray.map(async (image) => {
const response = await fetch(image);
const blob = await response.blob();
const ref = firebase
.storage()
.ref('provider-images')
.child(uuidv4());
await ref.put(blob).then((result) => {
imagesBlob.push(result.metadata.name);
});
}),
);
getImageUrls(imagesBlob);
};
I'm trying to create a node app that requires a URL from the user, the URL is then passed to scrape.js and using puppeteer, scrapes certain fields, and then passes the data back to app.js in a json format (so that I can then upset it into a doc). But what I receive is the entire ServerResponse and not the data in a json format as I'm intending.
I was hoping someone with more experience could shed some light. Here is what I have so far:
// app.js
const scrape = require('./scrape');
const router = express.Router();
router.get( '/', ( req, res ) => {
const url = req.body.url;
const item = new Promise((resolve, reject) => {
scrape
.scrapeData()
.then((data) => res.json(data))
.catch(err => reject('Scraping failed...'))
})
});
// scrape.js
const puppeteer = require('puppeteer');
const scrapeData = async () => {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.setViewport({ width: 360, height: 640 });
await page.goto(url);
let scrapedData = await page.evaluate(() => {
let scrapedDetails = [];
let elements = document.querySelectorAll('#a-page');
elements.forEach(element => {
let detailsJson = {};
try {
detailsJson.title = element.querySelector('h1#title').innerText;
detailsJson.desc = element.querySelector('#description_box').innerText;
} catch (exception) {}
scrapedDetails.push(detailsJson);
});
return scrapedDetails;
}));
// console.dir(scrapeData) - logs the data successfully.
};
module.exports.scrapeData = scrapeData
You have a naming problem. scrape.js is exporting the scrapeData function. Inside that function, you declared a scrapedData variable, which is not the same thing.
Where you put a:
console.dir(scrapeData) - logs the data successfully.
Add
return scrapeData;
That should solve your issue.
I'm sorry for the long question, I'm a newbie at node but I have made a CRUD API before with authentication and everything, I just need to understand how to integrate puppeteer to the API, so let me begin:
This is my project structure:
puppeteer-api
controllers - puppeteer.controller.js
routes - puppeteer.routes.js
index.js
This is my index.js file:
const puppeteer = require('puppeteer');
const express = require('express');
const booking = require('./routes/puppeteer.routes')
const app = express();
app.use('/booking', booking);
let port = 8080;
app.listen(port, () => {
console.log('Server is running on https://localhost:8080/');
});
puppeteer.routes.js:
const express = require('express');
const router = express.Router();
const puppeteer_controller = require('../controllers/puppeteer.controller');
router.get('/', puppeteer_controller.get_booking);
module.exports = router;
puppeteer.controller.js:
const puppeteer = require('puppeteer');
exports.get_booking = (req, res, next) => {
res.json = (async function main() {
try {
const browser = await puppeteer.launch({ headless: true});
const page = await browser.newPage();
await page.goto('https://www.booking.com/searchresults.es-ar.html?label=gen173nr-1DCAEoggI46AdIM1gEaAyIAQGYASy4ARfIAQzYAQPoAQGIAgGoAgM&lang=es-ar&sid=bc11c3e819d105b3c501d0c7a501c718&sb=1&src=index&src_elem=sb&error_url=https%3A%2F%2Fwww.booking.com%2Findex.es-ar.html%3Flabel%3Dgen173nr-1DCAEoggI46AdIM1gEaAyIAQGYASy4ARfIAQzYAQPoAQGIAgGoAgM%3Bsid%3Dbc11c3e819d105b3c501d0c7a501c718%3Bsb_price_type%3Dtotal%26%3B&ss=El+Bols%C3%B3n%2C+R%C3%ADo+Negro%2C+Argentina&is_ski_area=&checkin_year=&checkin_month=&checkout_year=&checkout_month=&no_rooms=1&group_adults=2&group_children=0&b_h4u_keep_filters=&from_sf=1&ss_raw=el+bols&ac_position=0&ac_langcode=es&ac_click_type=b&dest_id=-985282&dest_type=city&place_id_lat=-41.964452&place_id_lon=-71.532732&search_pageview_id=06d48fb6823e00e9&search_selected=true&search_pageview_id=06d48fb6823e00e9&ac_suggestion_list_length=5&ac_suggestion_theme_list_length=0');
await page.waitForSelector('.sr_item');
page.on('console', consoleObj => console.log(consoleObj.text()));
console.log('Retrieving hotels data');
const hoteles = page.evaluate(() => {
let hoteles = [];
let x = document.getElementsByClassName('sr_item');
hoteles.push(x);
let navigation = document.getElementsByClassName('sr_pagination_item');
for (const nav of navigation) {
nav.click();
hoteles.push(document.getElementsByClassName('sr_item'));
}
console.log('Finished looping through');
return hoteles;
});
} catch(e) {
console.log('error', e);
}
})();
};
So, what I want is to be able to send a GET request from my app and get a response from my API with a list of hotels from booking, it's just a personal project, the thing is, using Postman I'm sending the GET request but I get no response at all, so I'm wondering what I'm doing wrong and what direction to follow, if anyone would be able to point me in the right direction I would be so grateful.
The block ({})() runs your code instantly instead of on a request.
res.json is a function, you should not reassign this.
Instead, move the function somewhere else and call it like below,
async function scraper() {
try {
const browser = await puppeteer.launch({ headless: true});
const page = await browser.newPage();
await page.goto('https://www.booking.com/searchresults.es-ar.html?label=gen173nr-1DCAEoggI46AdIM1gEaAyIAQGYASy4ARfIAQzYAQPoAQGIAgGoAgM&lang=es-ar&sid=bc11c3e819d105b3c501d0c7a501c718&sb=1&src=index&src_elem=sb&error_url=https%3A%2F%2Fwww.booking.com%2Findex.es-ar.html%3Flabel%3Dgen173nr-1DCAEoggI46AdIM1gEaAyIAQGYASy4ARfIAQzYAQPoAQGIAgGoAgM%3Bsid%3Dbc11c3e819d105b3c501d0c7a501c718%3Bsb_price_type%3Dtotal%26%3B&ss=El+Bols%C3%B3n%2C+R%C3%ADo+Negro%2C+Argentina&is_ski_area=&checkin_year=&checkin_month=&checkout_year=&checkout_month=&no_rooms=1&group_adults=2&group_children=0&b_h4u_keep_filters=&from_sf=1&ss_raw=el+bols&ac_position=0&ac_langcode=es&ac_click_type=b&dest_id=-985282&dest_type=city&place_id_lat=-41.964452&place_id_lon=-71.532732&search_pageview_id=06d48fb6823e00e9&search_selected=true&search_pageview_id=06d48fb6823e00e9&ac_suggestion_list_length=5&ac_suggestion_theme_list_length=0');
await page.waitForSelector('.sr_item');
page.on('console', consoleObj => console.log(consoleObj.text()));
console.log('Retrieving hotels data');
const hoteles = page.evaluate(() => {
let hoteles = [];
let x = document.getElementsByClassName('sr_item');
hoteles.push(x);
let navigation = document.getElementsByClassName('sr_pagination_item');
for (const nav of navigation) {
nav.click();
hoteles.push(document.getElementsByClassName('sr_item'));
}
console.log('Finished looping through');
return hoteles;
});
} catch(e) {
console.log('error', e);
}
}
// Call the scraper
exports.get_booking = async (req, res, next) => {
const scraperData = await scraper();
res.json(scraperData)
}
It will make the controller into a promise and return the JSON data.