Using the ipfs-http-client I can successfully add data using the below method. The console.log(added) returns an object with the path, cid, and size keys.
However, the console.log(exists) line returns Object [AsyncGenerator] {}.
I would like to be able to check if a data string exists. Is this possible?
import { create as ipfsHttpClient } from 'ipfs-http-client'
const ipfsClient = ipfsHttpClient('https://ipfs.infura.io:5001/api/v0')
const handleData = async (data) => {
const added = await ipfsClient.add(data)
console.log(added)
const exists = await ipfsClient.get(data)
console.log(exists)
}
handleData('hello world')
The get method returns AsyncIterable<Uint8Array> object, which may be what you're printing out. To get each bytes, you will have to loop over it:
const cid = 'QmQ2r6iMNpky5f1m4cnm3Yqw8VSvjuKpTcK1X7dBR1LkJF'
for await (const buf of ipfs.get(cid)) {
// do something with buf
console.log(buf);
}
If all you care about is whether the data exists, you can just call next() method on the iterator and check for null or error.
const exists = (await ipfs.get(cid)).next() !== null
Related
I am trying to make barbershop web app where costumer can see list of free appointments and when they reserve free appointment I want to delete that field from firebase.
I have a collection which represents one barber.
This is how it looks in firebase.
As you see radno_vrijeme is object or map in firebase which contains 6 arrays, and in each array there is list of free working hours.
In my function I am able to do everthing except last line where I need to update firebase collection.
const finishReservation = async () => {
try {
const freeTimeRef = collection(db, `${barber}`);
const q = query(freeTimeRef);
const querySnap = await getDoc(q);
querySnap.forEach(async (doc) => {
const radnoVrijeme = doc.data().radno_vrijeme;
// Find the index of the hour you want to delete
const index = radnoVrijeme["Mon"].indexOf(hour);
// Remove the hour from the array
radnoVrijeme["Mon"].splice(index, 1);
// Update the document in the collection
console.log(radnoVrijeme);
const radnoVrijemeMap = new Map(Object.entries(radnoVrijeme));
await freeTimeRef.update({ radno_vrijeme: radnoVrijemeMap });
});
} catch (error) {
console.log(error);
}
};
I tried to pass it as JSON stringified object, but it didn't work. I always get this error :
"FirebaseError: Expected type 'ya', but it was: a custom Ia object"
When you are trying to fetch multiple documents using a collection reference or query, then you must use getDocs():
const finishReservation = async () => {
try {
const freeTimeRef = collection(db, `${barber}`);
const q = query(freeTimeRef);
const querySnap = await getDocs(q);
const updates = [];
querySnap.forEach((d) => {
const radnoVrijeme = d.data().radno_vrijeme;
const index = radnoVrijeme["Mon"].indexOf(hour);
radnoVrijeme["Mon"].splice(index, 1);
const radnoVrijemeMap = new Map(Object.entries(radnoVrijeme));
updates.push(updateDoc(d.ref, { radno_vrijeme: radnoVrijemeMap }))
});
await Promise.all(updates);
console.log("Documents updated")
} catch (error) {
console.log(error);
}
};
getDoc() is used to fetch a single document using a document reference.
Right now, I coded a function to go like this
async function checkPlayerScam(ign) {
const UUID = await getUUID(ign);
if(MATCHING){
playerIsScammer = true
}
else {
playerIsScammer = false
}
}
The MATCHING is just a placeholder at the moment. I want to check their UUID, and make sure it isn't in this list: https://raw.githubusercontent.com/skyblockz/pricecheckbot/master/scammer.json
Any idea how? It needs to be relatively fast
EDIT: It'd also be cool if I could get the reason from the list, but that's not as necessary
https://lodash.com/docs/#find
Use lodash _.find to
const uuid = '000c97aaf948417a9a74d6858c01aaae'; // uuid you want to find
const scammer = _.find(scammersList, o => o.uuid === uuid);
if (scammer) { // if scammer found
console.log(scammer);
console.log(scammer.reason)
}
For anyone wondering, this is how I solved it:
async function checkPlayerScam(ign) {
const UUID = await getUUID(ign);
const response = await fetch(`https://raw.githubusercontent.com/skyblockz/pricecheckbot/master/scammer.json`);
const result = await responsejson();
if (result[UUID] = null) {
playerIsScammer == False
}
else{
playerIsScammer == True
}
}
This function will fetch the data, then check if the uuid 1d0c0ef4295047b39f0fa899c485bd00 exists. Assuming that you already fetched the data somewhere else and stored it, all you need to do is check if a given uuid exists by adding the following line where you please:
!!data[uuidToCheck]
uuidToCheck should be the uuid string that you are looking for.
This line will return true if the uuid exists and false otherwise.
In terms of the spacetime complexity, this function runs in constant time [O(1)] and O(N) space. This is the fastest time you can get it to run.
data[uuidToCheck].reason will return the reason.
async function playerIsScammer(uuidToCheck) {
uuidToCheck = '1d0c0ef4295047b39f0fa899c485bd00';
const response = await fetch('https://raw.githubusercontent.com/skyblockz/pricecheckbot/master/scammer.json');
if (response.ok){
let data = await response.json();
if(!!data[uuidToCheck])
return data[uuidToCheck].reason;
return false
}
}
I am trying to pass certain values inside a function and based on that values i have to query the data in google cloud firestore but i am getting an undefined when i am trying to pass the values.
here is y tried way
const {profileCampaignSelector} =require('./profileCampaignSelector');
const checkinReminder = async() =>{
const end = momentTz()
.subtract(2, 'days')
.endOf('day')
.format();
const profileSnapshot = await rootCollections.zyx
.where('timestamp','<=',momentTz(end).valueOf()).limit(5).get();
profileSnapshot.forEach(doc =>{
console.log(doc.data())
profileCampaignSelector('joined','user',doc.data());
});
};
module.exports ={checkinReminder};
and here i am trying to get the values which i have passed
const profileCampaignSelector = async (stage, type, profile) => {
console.log(stage)
}
module.exports = { profileCampaignSelector };
execution file
const profileCampaignSelector =require('./profile');
profileCampaignSelector();
i have solved this way but don't know why i am getting undefined
I have a route in my NodeJs app which takes a post request and then retrieves some data with await. The first await in my function works fine but then it complains when i call another function with await. I marked the function which causes the issue with >>>>
SyntaxError: await is only valid in async function
Has this error to do with the forEach Loop and if so how can i fix that
farmRoutes.post('/bulkemail/:farmid', async(req, res) => {
try{
const farmid = req.params.farmid
// Check if we have an Emails Array
if (typeof req.body.emails != 'undefined' && req.body.emails instanceof Array ){
// Assign Emails Array from Body to emails
const emails = req.body.emails
// get the current Emails Array from Farm Doc
let oldEmail = await couch.getSubDoc('contacts', farmid ,'emails')
// Loop thru all emails in the new Array
emails.forEach((email) => {
console.log(email)
// Check if the email is curently already in the Farm Doc
var data = _.find(oldEmail, function(emailItem){ return emailItem.address == email; });
var index =_.indexOf(oldEmail,data);
if (index > -1) {
// Email already Exists will not be created
console.log("Email already in the List")
} else {
// if not in current Farm create new Email object and add to the oldEmail Array
>>>>>var newEmail = await contact.simp_email(simp_email)
// Upsert the updated Email Arra to farm Doc
var success = await cb.insertArrayItem(req.bucket, farmid, "emails", newEmail )
console.log(success)
}
})
} else {
console.log("we have no new emails")
}
Every function that awaits, must be an async function. Including the one passed to forEach.
emails.forEach(async (email) => {
Alternatively, you could avoid creating an iteration function by using a simple for of loop. This is usually preferred to using the forEach since it's a bit simpler and doesn't create any new functions.
for (const email of emails) {
//...
var newEmail = await contact.simp_email(simp_email)
//...
}
I'm just starting to play around with Puppeteer (Headless Chrome) and Nodejs. I'm scraping some test sites, and things work great when all the values are present, but if the value is missing I get an error like:
Cannot read property 'src' of null (so in the code below, the first two passes might have all values, but the third pass, there is no picture, so it just errors out).
Before I was using if(!picture) continue; but I think it's not working now because of the for loop.
Any help would be greatly appreciated, thanks!
for (let i = 1; i <= 3; i++) {
//...Getting to correct page and scraping it three times
const result = await page.evaluate(() => {
let title = document.querySelector('h1').innerText;
let article = document.querySelector('.c-entry-content').innerText;
let picture = document.querySelector('.c-picture img').src;
if (!document.querySelector('.c-picture img').src) {
let picture = 'No Link'; } //throws error
let source = "The Verge";
let categories = "Tech";
if (!picture)
continue; //throws error
return {
title,
article,
picture,
source,
categories
}
});
}
let picture = document.querySelector('.c-picture img').src;
if (!document.querySelector('.c-picture img').src) {
let picture = 'No Link'; } //throws error
If there is no picture, then document.querySelector() returns null, which does not have a src property. You need to check that your query found an element before trying to read the src property.
Moving the null-check to the top of the function has the added benefit of saving unnecessary calculations when you are just going to bail out anyway.
async function scrape3() {
// ...
for (let i = 1; i <= 3; i++) {
//...Getting to correct page and scraping it three times
const result = await page.evaluate(() => {
const pictureElement = document.querySelector('.c-picture img');
if (!pictureElement) return null;
const picture = pictureElement.src;
const title = document.querySelector('h1').innerText;
const article = document.querySelector('.c-entry-content').innerText;
const source = "The Verge";
const categories = "Tech";
return {
title,
article,
picture,
source,
categories
}
});
if (!result) continue;
// ... do stuff with result
}
Answering comment question: "Is there a way just to skip anything blank, and return the rest?"
Yes. You just need to check the existence of each element that could be missing before trying to read a property off of it. In this case we can omit the early return since you're always interested in all the results.
async function scrape3() {
// ...
for (let i = 1; i <= 3; i++) {
const result = await page.evaluate(() => {
const img = document.querySelector('.c-picture img');
const h1 = document.querySelector('h1');
const content = document.querySelector('.c-entry-content');
const picture = img ? img.src : '';
const title = h1 ? h1.innerText : '';
const article = content ? content.innerText : '';
const source = "The Verge";
const categories = "Tech";
return {
title,
article,
picture,
source,
categories
}
});
// ...
}
}
Further thoughts
Since I'm still on this question, let me take this one step further, and refactor it a bit with some higher level techniques you might be interested in. Not sure if this is exactly what you are after, but it should give you some ideas about writing more maintainable code.
// Generic reusable helper to return an object property
// if object exists and has property, else a default value
//
// This is a curried function accepting one argument at a
// time and capturing each parameter in a closure.
//
const maybeGetProp = default => key => object =>
(object && object.hasOwnProperty(key)) ? object.key : default
// Pass in empty string as the default value
//
const getPropOrEmptyString = maybeGetProp('')
// Apply the second parameter, the property name, making 2
// slightly different functions which have a default value
// and a property name pre-loaded. Both functions only need
// an object passed in to return either the property if it
// exists or an empty string.
//
const maybeText = getPropOrEmptyString('innerText')
const maybeSrc = getPropOrEmptyString('src')
async function scrape3() {
// ...
// The _ parameter name is acknowledging that we expect a
// an argument passed in but saying we plan to ignore it.
//
const evaluate = _ => page.evaluate(() => {
// Attempt to retrieve the desired elements
//
const img = document.querySelector('.c-picture img');
const h1 = document.querySelector('h1')
const content = document.querySelector('.c-entry-content')
// Return the results, with empty string in
// place of any missing properties.
//
return {
title: maybeText(h1),
article: maybeText(article),
picture: maybeSrc(img),
source: 'The Verge',
categories: 'Tech'
}
}))
// Start with an empty array of length 3
//
const evaluations = Array(3).fill()
// Then map over that array ignoring the undefined
// input and return a promise for a page evaluation
//
.map(evaluate)
// All 3 scrapes are occuring concurrently. We'll
// wait for all of them to finish.
//
const results = await Promise.all(evaluations)
// Now we have an array of results, so we can
// continue using array methods to iterate over them
// or otherwise manipulate or transform them
//
results
.filter(result => result.title && result.picture)
.forEach(result => {
//
// Do something with each result
//
})
}
Try-catch worked for me:
try {
if (await page.$eval('element')!==null) {
const name = await page.$eval('element')
}
}catch(error){
name = ''
}