I have a shopping cart app I developed using React, I have been working on a feature which scrapes data from a grocery store using puppeteer. The problem I am having is that im not sure of how to go about and share the data from my puppeter.js file and implement the data, and display it in my react app.
Any help is appreciated.
You need to get the data and save it to the local folder in a json format. Then you can use it in your react app. For example
const puppeteer = require("puppeteer")
const fs = require("fs")
async function scrapeData () {
const browser = await puppeteer.launch()
const [page] = await browser.pages()
/*
I dont know how your data is stored or what data you want to get so i cant
implement
getting the data here. Assume that i got it and stored in a variable named
'theData'
*/
fs.writeFile("theData.json", JSON.stringify(theData), () => {
console.log("The data saved to the file named theData.json")
})
}
scrapeData()
Then you can use fs.readFile() method to read and use it in your react app.
Related
I'm trying to learn a bit of firebase.
I've made a small site where i can add and delete data from firestore as well as some log in and out functions.
I want to move forward by instead of displaying the results in the console showing them on the screen.
below is a snippet of my code, everything works okay i have imported everything correctly and it all shows in the console without errors but I cannot figure out how to get this to the HTML file in order to display it in the browser.
index.js file
//collection ref
const colRef = collection(db, 'books')
//queries
const q = query(colRef, orderBy('createdAt'))
booksDisplay = document.getElementsByClassName('books')
//real time collection data
onSnapshot(q, (snapshot)=>{
let books = []
snapshot.docs.forEach((doc)=>{
books.push({...doc.data(), id: doc.id})
})
})
booksDisplay.innerHTML='<p>'+books+'</p>'
index.html
<h2>Display Books</h2>
<div class="books"></div>
Greetings to everyone,
I have just started to learn NodeJS and have been experimenting on API routes in NextJS as its easy to setup and see what's actually going on, So I know how to make a basic get request but I'm interested in something a little complex.
So I'm trying to build an API route that can be filleted by page number, so for example api/pages/1 would return page 1 and so on.
so this is my file in /api/game.js
export default async function handler(req,res) {
const response = await fetch('https://exampleapi.com/pages/?sort=&page=1&per_page=50')
const jsonData = await response.json();
res.status(200).json(jsonData);
}
Now this works obviously but I want to know how I can make dynamic routes for all the pages. Since I'm using an external API I'm not sure how many pages exist at any moment.
So far I have created another folder and called the file [gamepage].js, I'm not sure how I would manipulate the fetch call in the game.js file in here.
export default async function handler(req, res) {
const { pno } = req.query
console.log(pro)
const response = await fetch(`https://exampleapi.com/pages/?sort=&page=${pno}&per_page=50`)
const jsonData = await response.json();
res.status(200).json(jsonData);
}
Sorry if the question is too beginner level, I'm just getting started with backend JS.
I got it to work, I was just missing the gamepage from req.query
const { pno } = req.query.gamepage
this did it.
I am currently developing a voice agent to be used in a smart speaker where users will ask about some items that are being stored in a data stream. The ultimate goal is that users ask about items' names in the stream and google actions through voice will tell them the details about those items as presented in another column in the stream.
To do this, I linked a spreadsheet to Axios to stream the content of the spreadsheet as data to be read in a webhook in google actions. The link to the data stream is HERE.
Honestly, I am new to developing apps for google actions and new to javascript overall so I might be doing silly mistakes.
In the graphical interface for google actions, I am setting a type for the items I want the user to ask about.
Then, I set an intent to recognize the item as a data type and be able to send this to the webhook.
The cloud function in the webhook is as follows:
const { conversation } = require('#assistant/conversation');
const functions = require('firebase-functions');
require('firebase-functions/lib/logger/compat'); // console.log compact
const axios = require('axios');
const app = conversation({debug: true});
app.handle('getItem', async conv => {
const data = await getItem();
const itemParam = app.types.Item;
// conv.add("This test to see if we are accessing the webhook for ${itemParam}");
data.map(item => {
if (item.Name === itemParam)
agent.add('These are the datails for ${itemParam}. It is located in zone
${item.Zone}, at level ${item.Level}');
});
});
async function getItem() {
const res = await axios.get('https://sheetdb.io/api/v1/n3ol4hwmfsmqd');
console.log(res.data);
return res.data; // To use in your Action's response
}
exports.ActionsOnGoogleFulfillment = functions.https.onRequest(app);
What the webhook is doing is getting the stream with the getItem function and then mapping the data to find the Name in the stream to match the item parameter (ItemParam) as identified by the user.
However, one of the main problems I have is that when trying to access the item from the user, I am using app.types.Item, but this does not work as when testing I get an error saying: "error": "Cannot read property 'Item' of undefined". I think what is happening is that I am not using the correct way to call the Item in the conversation app.
Also, I am not sure exactly how the linking to the database will work. In other works, I am not sure if
data.map(item => {
if (item.Name === itemParam)
agent.add('These are the datails for ${itemParam}. It is located in zone
${item.Zone}, at level ${item.Level}');
will work.
I have tried multiple things to solve but I am really struggling so any help with this would be really appreciated. Also, I know that I rushed to explain things, so please let me know if you need me to explain better or clarify anything.
Thank you
There are three points I am seeing that won't work.
First, app.types.Item is not the way to get this parameter. You should instead use conv.intent.params['Item'].resolved to get the user's spoken name.
Second, you are trying to use agent.add to include text, but there is no agent in your environment. You should instead be using conv.add.
Third, the text you are sending is not properly escaped between backticks ``. It is the backtick that allows you to use template literals.
Altogether your code can be rewritten as:
const { conversation } = require('#assistant/conversation');
const functions = require('firebase-functions');
require('firebase-functions/lib/logger/compat'); // console.log compact
const axios = require('axios');
const app = conversation({debug: true});
app.handle('getItem', async conv => {
const data = await getItem();
const itemParam = conv.intent.params['Item'].resolved;
data.map(item => {
if (item.Name === itemParam)
conv.add(`These are the datails for ${itemParam}. It is located in zone
${item.Zone}, at level ${item.Level}`);
});
});
async function getItem() {
const res = await axios.get('https://sheetdb.io/api/v1/n3ol4hwmfsmqd');
console.log(res.data);
return res.data; // To use in your Action's response
}
exports.ActionsOnGoogleFulfillment = functions.https.onRequest(app);
This question already has an answer here:
Next js nested dynamic routes: data fetching via getstaticprops with shared data between routes
(1 answer)
Closed last year.
im fetching a huge json data from an external API inside my getstaticprops. this data will then be divided into parts to be send to other static pages (hundreds of pages) as props.
// page1.tsx
const page1 = ({ page1Data }) => {
...
}
const getStaticProps: GetStaticProps = async () => {
const res = await fetch('https://hugedata')
const jsonData = await res.json()
const { page1Data, page2Data, page300Data, ...allData } = jsonData
return { props: { page1Data } }
}
i only know how to send the staticprops data to the component inside the file like this.
is there a way send these data to other static pages/routes ? (e.g. to page2.tsx)
It's not really possible to send data between pages. But you can see the methods described here.
I suggest you really check router and inspect data inside it, you can find a lot of info about data on the page.
P.S. As I see, you are trying to get a lot of data and you need to use pagination and split it to the parts, you can use this lib
there is way to see the content of the FileSystem and also delete content from there?
i have data inside the expo FileSystem and i need :
look inside FileSystem because i dont know how to find this file because i work with the expo client sdk with the qr code scan and i dont understand how can i find this file there .
i want to know how to delete contents from the file system .
how can i see the sqlite database as a real table ?
this is my examle :
saveDbData = async () => {
const { data, isLoaded } = this.state;
for (let i = 0; i < data.length; i++) {
const mediaData = data[i];
const dbResult = await insertPlace(
mediaData.artist,
mediaData.image,
mediaData.title,
mediaData.url
);
console.log("SQL", `${FileSystem.documentDirectory}/SQLite/places.db`);
}
const fetchawesome = await fetchPlaces();
console.log(fetchawesome)
};
Since there are three questions from your part, I'll try to answer them one by one and provide a possible solution.
look inside FileSystem because i dont know how to find this file
because i work with the expo client sdk with the qr code scan and i
dont understand how can i find this file there .
You can access the file system using the FileSystem package from expo. Here's how you can do it. Import the package first.
import * as FileSystem from "expo-file-system";
Then you can find the URI of the file like this.
const { uri } = await FileSystem.getInfoAsync(
`${FileSystem.documentDirectory}SQLite/${"yourDB.db"}`
);
I want to know how to delete contents from the file system.
You can delete a file if you have access to its location or URI. This is how you can delete it. It returns a promise.
FileSystem.deleteAsync(fileUri)
how can I see the SQLite database as a real table?
This is a tricky part because since you're using your android phone to run the application via expo, you won't have direct access to your db. One thing you can do is to move the db to someplace else where you have access to and using the Sharing API from expo to save or send the db to yourself via email or any other way.
import * as Sharing from "expo-sharing";
let documenturi = `${FileSystem.documentDirectory}/yourDB.db`;
await FileSystem.copyAsync({
from: uri,
to: documenturi,
});
Sharing.shareAsync(documenturi);
You can make this as a function which fire's on a Button press on you can even put this in useEffect or lifecycle methods to fire up when the screen loads.