We Keep Coding

How to implement server side search filter in redux tool kit using query builder RTK?

I want to apply server side search filter by text using redux toolkit.
I have two query builder methods in place. One for fetching all items and second for fetching only filtered data.
Query builder for fetching all items is
getAllBlogs: builder.query<BlogType[], void>({
queryFn: async () => {
const collectionRef = collection(Firestore, BLOG_COLLECTION)
const q = query(collectionRef, limit(1000))
const resp = await getDocs(q)
return {
data: resp.docs.map((doc) => doc.data() as BlogType),
}
},
providesTags: (result) => {
const tags: { type: 'Blogs'; id: string }[] = [
{ type: 'Blogs', id: 'LIST' },
]
if (result) {
result.forEach(({ id }) => {
tags.push({
type: 'Blogs',
id,
})
})
}
return tags
},
}),
This works fine and I'm getting the whole list through useGetAllBlogsQuery data.
Query builder for fetching filtered data is here: (Partially completed)
getBlogsByTitle: builder.query<BlogType[], string>({
queryFn: async (title) => {
const collectionRef = collection(Firestore, BLOG_COLLECTION)
const q = query(
collectionRef,
where('searchIndex', 'array-contains', title),
limit(1000),
)
const resp = await getDocs(q)
return {
data: resp.docs.map((doc) => doc.data() as BlogType), // Correct data
}
},
// I'm trying to only push the resultant items in state. This is not working
providesTags: (result) => {
const tags: { type: 'Blogs'; id: string }[] = []
if (result) {
result.forEach(({ id }) => {
tags.push({
type: 'Blogs',
id,
})
})
}
return tags
},
}),
I have react component looks like this where I'm calling these queries.
const Blogs: NextPage = () => {
const { data: blogs } = blogsApi.useGetAllBlogsQuery()
const [getBlogsByTitle] = blogsApi.useLazyGetBlogsByTitleQuery()
const debounced = useDebouncedCallback(async (value) => {
const { data } = await getBlogsByTitle(value)
console.log(data) // Correct data
}, 500)
return (
<div>
<InputText
onChange={(e) => debounced(e.target.value)}
/>
</div>
)}
The above code has two functionalities.
Fetch all the items on initial load.
Filter when debounced function is being called.
What I want is when getBlogsByTitle is called it will auto update the same state blogs in redux and we don't have to do much.
We are getting correct response in getBlogsByTitle but this query is not updating state with only its filtered response.
I'm new to redux-toolkit. Can someone help me out here where am I doing wrong ?

Flattening JSON API response in Node

JSON Object returned from API Response:
{3 items
"total_reviews":574
"stars_stat":{5 items
"1":"2%"
"2":"1%"
"3":"3%"
"4":"8%"
"5":"85%"
}
"result":[10 items
0:{9 items
"id":"R3BWKTLO7CM3Y9"
"asin":{...}2 items
"review_data":"Reviewed in the United States on April 15, 2022"
"date":{2 items
"date":"April 15, 2022"
"unix":1649988000
}
"name":"Sally N."
"rating":5
"title":"Great alcohol !"
"review":"Great products"
"verified_purchase":true
}
1:{...}9 items
2:{...}9 items
3:{...}9 items
4:{...}9 items
5:{...}9 items
6:{...}9 items
7:{...}9 items
8:{...}9 items
9:{...}9 items
]
I've tried various approaches but keep getting stuck implementing the approach or realizing it's wrong so I just commented out all the stuff I tried. I first tried to just select for a specific information in the response object and store it in a variable, then passing the info to a record object that matches the CSV headers I need, so I can write it to a file, however, couldn't make it work iteratively in a loop for the different key/value pairs I need. Then I read about an inbuilt json object function that lets you loop through a json object based on the keys object.keys, values object.values, or both object.entries, but I have nested key/values and some of the information I need is nested. So I think the solution is to figure out how to flatten the JSON object into just the key-values I need, and that is what I'm seeking help with. Ultimately trying to write this info to a csv file.
I need the following fields: customer name, product name, SKU, review content, star rating, and time and date created
My Code I've tried:
const axios = require("axios");
const converter = require("json-2-csv");
const fs = require("fs");
const moment = require('moment');
const createCsvWriter = require('csv-writer').createObjectCsvWriter;
const csvWriter = createCsvWriter({
path: './ProductReviews.csv',
header: ['asin', 'name', 'review', 'rating'].map((item) => ({ id: item, title: item }))
})
const options = {
method: 'GET',
url: 'https://amazon23.p.rapidapi.com/reviews',
params: {asin: 'B00DUGMVC6', sort_by: 'recent', page: '1', country: 'US'},
headers: {
'X-RapidAPI-Key': 'redacted',
'X-RapidAPI-Host': 'redacted'
}
};
axios.request(options).then(function (response) {
console.log(response.data);
// for (let i = 0; i < 10; i++ ) {
// let name = response.data.result[i].name;
// let content = response.data.result[i].review;
// let rating = response.data.result[i].rating;
// let date = response.data.result[i].date.date
// let momentvar = moment(date, 'MMM DD, YYYY').format();
// const records = [
// {
// customername: name, productname: 'Isopropyl Alcohol 99.9%', sku: 'B00DUGMVC6', reviewcontent: content, starrating: rating, timeanddate: date
// }
// ]
// for (let [key, value] of Object.entries(response.data)) {
// console.log(key, value);
// }
// try {
// csvWriter.writeRecords(response.data.result[0]);
// } catch (error) {
// console.log(error);
// }
// converter.json2csv(response.data, (err, csv) => {
// if (err) {
// throw err;
// }
// fs.writeFileSync('todos.csv', csv);
// })
// csvWriter.writeRecords(records) // returns a promise
// .then(() => {
// console.log('...Done');
// });
// }
// var response = JSON.parse(response);
// console.log(response);
// csvWriter.writeRecords(response.data)
// converter.json2csv(response.data, (err, csv) => {
// if (err) {
// throw err;
// }
// console.log(csv);
// })
}).catch(function (error) {
console.error(error);
});

Just skimming css-writer documentation, it looks like the package wants two js objects: one describing the header, and another array of records who's keys match the ids given in the header object.
From what I can make of the OP's input object, the transform is straight-forward. The only unclear bit is what should be written for the asin key, which is described in the question as an object, but not elaborated further.
Here's a demo of the transform to csv-writer params. In it, we just stringify the asin object...
const data = {
"total_reviews":574,
"stars_stat":{
"1":"2%",
"2":"1%",
"3":"3%",
"4":"8%",
"5":"85%"
},
"result":[
{
"id":"R3BWKTLO7CM3Y9",
"asin":{
"original":"B00DUGMVC6",
"variant":"B00DUGMVC6"
},
"review_data":"Reviewed in the United States on April 15, 2022",
"date":{
"date":"April 15, 2022",
"unix":1649988000
},
"name":"Sally N.",
"rating":5,
"title":"Great alcohol !",
"review":"Great products",
"verified_purchase":true
},
// more like this
]
}
const result = data.result;
// OP should replace the previous line with const result = response.data.result;
const header = ['asin', 'name', 'review', 'rating'].map((item) => ({ id: item, title: item }))
const records = result.map(({ asin: { original, variant } , name, date: { date }, review, rating }) => {
date = new Date(date).toISOString();
return { original, variant, name, review, rating, date };
});
console.log(header, records);
// // OP should replace the previous line with:
// const csvWriter = createCsvWriter({
// path: './ProductReviews.csv',
// header: header
// })
// csvWriter.writeRecords(records)
edit
Calling and writing several pages might look something like the following. Notes: (1) from reading, I learned that asin is like a product id, probably invariant over all reviews, (2) reasonably low volume (see if it performs, then consider streaming if it doesn't), (3) guessing that no records returned for a page means we're done.
async function getReviewPage(asin, page, csvWriter) {
const options = {
method: 'GET',
url: 'https://amazon23.p.rapidapi.com/reviews',
params: {asin, sort_by: 'recent', page: page+'', country: 'US'},
headers: {
'X-RapidAPI-Key': 'redacted',
'X-RapidAPI-Host': 'redacted'
}
};
const response = await axios.request(options);
const result = response.data.result;
const records = result.map(({ asin: { original, variant } , name, date: { date }, review, rating }) => {
date = new Date(date).toISOString();
return { original, variant, name, review, rating, date };
});
if (records.length) await csvWriter.writeRecords(records)
return records.length === 0;
}
async function getAllReviews(asin) {
const header = ['asin', 'name', 'review', 'rating'].map((item) => ({ id: item, title: item }))
const csvWriter = createCsvWriter({
path: './ProductReviews.csv',
header: header
});
let done = false;
let page = 1;
while (!done) {
done = await getReviewPage(asin, page++, csvWriter);
}
return page-1
}
// test it with
getAllReviews('myASIN').then(pagesWritten => console.log(`wrote ${pagesWritten} pages`);
// run it one of two ways:
async function getAllReviewsForAllProducts(asinArray) {
// in parallel, though you might get throttled by the service...
return await Promise.all(asinArray.map(getAllReviews));
// OR sequentially
for (let asin of asinArray) {
await getAllReviews(asin)
}
}

I am having trouble calling a product from the database. (MongoDb - Nodejs)

When I typed phone in the search box, I found and fetched all categories with the word phone in the database. Then I wanted to find the products by matching the _id number of this category with the category id number of the product. but I cannot collect the products I find in a single array. that's why I can't print them all on the screen. Since two different arrays are created in the arrays, it prints the products in the first arrays, but does not pass to the second arrays.
array in array
As you can see from the picture, I cannot print it because the 3rd product is in the other array.
function escapeRegex(text) {
return text.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, "\\$&");
};
let model = [];
const searchRegex = new RegExp(escapeRegex(req.query.search), 'gi');
SubSubCategory.find({ "name": searchRegex})
.then(subsubcategoriesProduct => {
subsubcategoriesProduct.forEach(p => {
Product.find({ categories: p._id })
.then(finalProduct => {
model.push(finalProduct);
res.render('shop/products', {
title: 'Tüm Ürünler',
products: model,
path: '/products',
searchRegex: searchRegex
});
...

If there are 50 products in subsubcategoriesProduct, you are consequently launching 50 new Mongo queries at once inside the forEach. Each of these Product.find operations is asynchronous and will complete some time later, triggering 50 res.render. You can't do that, you can only have one res.render.
Dealing with this kind of things using the traditional .then() syntax is complicated and easily leads to a callback hell (then inside then inside then). Using await instead of .then() makes things way easier.
Also, instead of making 50 queries (one for each _id) you should make one query with an array of _id.
const subcategories = await SubSubCategory.find({ name: searchRegex}, '_id')
.lean() // return only JSON, not full Mongoose objects
.exec(); // Returns a Promise so we can use await on it
const ids = subcategories.map(s => s._id);
const model = await Product.find({ categories: { $in : ids } }).lean().exec();
res.render('shop/products', {
title: 'Tüm Ürünler',
products: model,
path: '/products',
searchRegex: searchRegex
});

I solved my problem this way.
function escapeRegex(text) {
return text.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, "\\$&");
};
exports.getSearch = async (req, res, next) => {
try{
const subsubcategories = await SubSubCategory.find();
const subcategories = await SubCategory.find();
const categories = await Category.find();
if(req.query.search){
var searchRegex = new RegExp(escapeRegex(req.query.search), 'gi');
}
const subsubcategoriesProduct = await SubSubCategory.find({ "name": searchRegex}, '_id')
const ids = subsubcategoriesProduct.map(s => s._id);
const finalProduct = await Product.find({ categories: {$in: ids} });
res.render('shop/products', {
title: 'Tüm Ürünler',
products: finalProduct,
path: '/products',
categories: categories,
subcategories: subcategories,
subsubcategories: subsubcategories,
searchRegex: searchRegex,
inputs:{
takeSecondHand: '',
takeMinPrice: '',
takeMaxPrice: ''
}
});
}
catch(err){
next(err);
}
}

Get all records from collection with all refrences in Firestore

Hi I'm currently blocked because I can't get all records from a collection with references values.
I would like to get all records from collection events (it works) but when I wanna merge the category information associated with categoryId my code doesn't work anymore.
Events collection
Categories collection
export const getEventsRequest = async () => {
const output = [];
const data = await firebase.firestore().collection('events').get();
data.forEach(async (doc) => {
const {
name,
address,
city,
duration,
level,
startDate,
maxPeople,
categoryId,
} = doc.data();
const { name: categoryName, color } = (
await firebase.firestore().collection('categories').doc(categoryId).get()
).data();
output.push({
name,
address,
city,
duration,
level,
startDate,
maxPeople,
category: { name: categoryName, color },
});
});
return output;
};
Example testing in a React Native project
const [events, setEvents] = useState([]);
const [isEventsLoading, setIsEventsLoading] = useState(false);
const getEvents = async () => {
setEvents([]);
setIsEventsLoading(true);
try {
const evts = await getEventsRequest();
setEvents(evts);
setIsEventsLoading(false);
} catch (e) {
console.error(e);
}
};
useEffect(() => {
getEvents();
}, []);
console.log('events', events);
Output
events Array []
Expected
events Array [
{
name : "blabla",
address: "blabla",
city: "blabla",
duration: 60,
level: "hard",
startDate: "13/04/2021",
maxPeople: 7,
category: {
name: "Football",
color: "#fff"
},
},
// ...
]
I don't know if there is a simpler method to retrieve this kind of data (for example there is populate method on mongo DB).
Thank you in advance for your answers.

When you use CollectionReference#get, it returns a Promise containing a QuerySnapshot object. The forEach method on this class is not Promise/async-compatible which is why your code stops working as you expect.
What you can do, is use QuerySnapshot#docs to get an array of the documents in the collection, then create a Promise-returning function that processes each document and then use it with Promise.all to return the array of processed documents.
In it's simplest form, it would look like this:
async function getDocuments() {
const querySnapshot = await firebase.firestore()
.collection("someCollection")
.get();
const promiseArray = querySnapshot.docs
.map(async (doc) => {
/* do some async work */
return doc.data();
});
return Promise.all(promiseArray);
}
Applying it to your code gives:
export const getEventsRequest = async () => {
const querySnapshot = await firebase.firestore()
.collection('events')
.get();
const dataPromiseArray = querySnapshot.docs
.map(async (doc) => {
const {
name,
address,
city,
duration,
level,
startDate,
maxPeople,
categoryId,
} = doc.data();
const { name: categoryName, color } = (
await firebase.firestore().collection('categories').doc(categoryId).get()
).data();
return {
name,
address,
city,
duration,
level,
startDate,
maxPeople,
category: { name: categoryName, color },
};
});
// wait for each promise to complete, returning the output data array
return Promise.all(dataPromiseArray);
};

how can I fetch a lot of users by api??with using async await

I'm trying to get all users from api and I need to find the user which most get paid.
so for example
let users=['tom','jenny','smith','Joe']
async function getUsers() {
let response = await fetch(`http://something.something?q=${users}`);
let data = await response.json();
return data;
}
getUsers().then(data =>console.log(data))
so my plan is users[0],users[1] something like a make function which I add index number via loop.
and get all users and find out who get the most paid.
so my question is how can do fetch users step by step.

You can use Array reduce to get the user who less payed and combine with Promise.all using async / await to fetch all user data
'use strict';
/* mock data for testing purposes
const usersData = [{
name: 'john doe',
payed: 10,
}, {
name: 'john doe01',
payed: 5,
}, {
name: 'john doe02',
payed: 8,
}, {
name: 'john doe03',
payed: 20,
}, {
name: 'john doe04',
payed: 40,
}, {
name: 'john doe05',
payed: 37,
}];
*/
async function getUsers() {
const usersResponse = await Promise.all(
usersName.map(userName => fetch(`http://something.something?q=${userName}`))
);
return usersResponse.map(userResponse => userResponse.json());
}
async function init() {
try {
const usersName = [
'tom',
'jenny',
'smith',
'Joe',
];
const usersData = await getUsers(usersName);
const userWhoLessPayed = usersData.reduce((prevUser, currentUser) => {
if (prevUser.payed > currentUser.payed) {
return currentUser;
}
return prevUser;
});
console.log(userWhoLessPayed);
} catch (e) {
console.error(e);
}
}
init();

Maybe try something like this? Do you need to use async?
Basically, the strategy is this:
Return all users. Unless the endpoint has a way to filter (via
parameters - in some way where you currently cannot ascertain the
highest paid artist)
Once returned, take that list of users returned from the endpoint
and filter. Below, we're filtering a returned array of objects,
looking for the highest value of a fictional key called 'pay'.
$.when( getAllArtists() ).then(function (allArtistResults) {
$.each(allArtistResults, function(key, value){
// let's just pretend we get an array of objects back
// you could just filter it here?
// props on below: https://stackoverflow.com/questions/4020796/finding-the-max-value-of-an-attribute-in-an-array-of-objects
Math.max.apply(Math, value.map(function(artists) { return artists.pay; }))
}
}
function getAllArtists() {
return $.ajax({
url: 'your-endpoint-here',
type: 'GET'
});
}