We Keep Coding

React - axios fetching empty array [duplicate]

I am currently working on social media mern stack react app. I am using node js and express as my backend services , also using mongoose to store my data and axios and redux thunk which connect the backend to the front end. Till now I had no issue recieving and sending data to the server. Right now I am trying to create search post get request ,base on a keyword the user entered. The issue with it, that when I am sending the keyword to the server instead of recieving the string it gets undefined value, like redux thunk not sending anything. I will be very thankful if someone could help me with that. I am watching the code over and over again and can't find out the reason for that.
My post controller class(I copied only the relevant function):
import express from "express";
const app = express();
import Post from "../model/PostModel.js";
import ErrorHandlng from "../utilities/ErrorHandling.js";
import bodyParser from "body-parser";
import catchAsync from "../utilities/CatchAsync.js";
import User from "../model/UserModel.js";
app.use(express.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
export const getPosts = catchAsync(async (req, res, next) => {
const data = req.body.keyword;
const page = parseInt(req.query.page || "0");
const PAGE_SIZE = 20;
const query = new RegExp(data, "i");
const total = await Post.countDocuments({});
const posts = await Post.find({ $or: [{ title: query }, { content: query }] })
.limit(PAGE_SIZE)
.skip(PAGE_SIZE * page);
if (!posts) {
return next(new ErrorHandlng("No posts were found", 400));
}
res.status(200).json({
status: "success",
data: {
totalPages: Math.ceil(total / PAGE_SIZE),
posts,
},
});
});
My api class(front end,copied only the calling for that specific get request):
import axios from "axios";
const baseURL = "http://localhost:8000";
axios.defaults.withCredentials = true;
const API = axios.create({
baseURL,
credentials: "include",
headers: {
Accept: "application/json",
"Content-Type": "application/json",
},
});
export const getPostsByKeyword = (keyword, page) =>
API.get(`/post/getPostsByKey?page=${page}`, keyword);
Post slice class:
export const fetchPostsByKeyWord = createAsyncThunk(
"post/getKeyword",
async ({ keyword, page }, { fulfillWithValue, rejectWithValue }) => {
try {
const response = await api.getPostsByKeyword(keyword, page);
if (response.statusCode === "400") {
throw new Error("There are no available posts");
}
const fetchData = await response.data.data.posts;
const totalPages = await response.data.data.totalPages;
return fulfillWithValue({ fetchData, totalPages });
} catch (err) {
console.log(err.response.message);
}
}
);
const initialState = { status: "undefined" };
const PostSlice = createSlice({
name: "post",
initialState,
reducers: {},
extraReducers: {},
});
export const postActions = PostSlice.actions;
export default PostSlice;
Calling the backend:
dispatch(fetchPostsByKeyWord({ keyword, page }))
.unwrap()
.then((originalPromiseResults) => {
console.log("thte " + " " + originalPromiseResults.totalPages);
console.log("The data is" + originalPromiseResults.fetchData);
setTotalPages(originalPromiseResults.totalPages);
})
.catch((err) => {
console.log(err.message);
});
As you can see I have not copied the whole code, I copied only the parts that are relevants for the question.

Browsers cannot currently send GET requests with a request body. XMLHttpRequest (which Axios uses) will ignore it and fetch() will trigger an error.
See also HTTP GET with request body for extra discussion on why trying this might be a bad idea.
You should instead pass everything required in the query string, preferably via the params option so it is correctly encoded...
export const getPostsByKeyword = (keyword, page) =>
API.get("/post/getPostsByKey", { params: { page, keyword } });
and grab the data via req.query server-side.
const { page, keyword } = req.query;
With vanilla JS, you can use URLSearchParams to construct the query string...
const params = new URLSearchParams({ page, keyword });
// XHR
const xhr = new XMLHttpRequest();
xhr.open("GET", `/post/getPostsByKey?${params}`);
// Fetch
fetch(`/post/getPostsByKey?${params}`); // GET is the default method
Your Axios instance creation could also be a lot simpler...
Axios is usually quite good at setting the correct content-type header, you don't have to
Your Express app isn't doing any content-negotiation so you don't need to set the accept header
Unless you're actually using cookies (which it doesn't look like), you don't need credential support
const API = axios.create({ baseURL });

How to validate GitHub webhook with Deno?

I'm trying to make a GitHub webhook server with Deno, but I cannot find any possible way to do the validation.
This is my current attempt using webhooks-methods.js:
import { Application } from "https://deno.land/x/oak/mod.ts";
import { verify } from "https://cdn.skypack.dev/#octokit/webhooks-methods?dts";
const app = new Application();
app.use(async (ctx, next) => {
try {
await next();
} catch (_err) {
ctx.response.status = 500;
}
});
const secret = "...";
app.use(async (ctx) => {
const signature = ctx.request.headers.get("X-Hub-Signature-256");
if (signature) {
const payload = await ctx.request.body({ type: "text" }).value;
const result = await verify(secret, payload, signature);
console.log(result);
}
ctx.response.status = 200;
});
The verify function is returning false every time.

Your example is very close. The GitHub webhook documentation details the signature header schema. The value is a digest algorithm prefix followed by the signature, in the format of ${ALGO}=${SIGNATURE}:
X-Hub-Signature-256: sha256=d57c68ca6f92289e6987922ff26938930f6e66a2d161ef06abdf1859230aa23c
So, you need to extract the signature from the value (omitting the prefix):
const signatureHeader = request.headers.get("X-Hub-Signature-256");
const signature = signatureHeader.slice("sha256=".length);
Update: Starting in release version 3.0.1 of octokit/webhooks-methods.js, it is no longer necessary to manually extract the signature from the header — that task is handled by the verify function. The code in the answer has been updated to reflect this change.
Here's a complete, working example that you can simply copy + paste into a project or playground on Deno Deploy:
gh-webhook-logger.ts:
import { assert } from "https://deno.land/std#0.177.0/testing/asserts.ts";
import {
Application,
NativeRequest,
Router,
} from "https://deno.land/x/oak#v11.1.0/mod.ts";
import type { ServerRequest } from "https://deno.land/x/oak#v11.1.0/types.d.ts";
import { verify } from "https://esm.sh/#octokit/webhooks-methods#3.0.2?pin=v106";
// In actual usage, use a private secret:
// const SECRET = Deno.env.get("SIGNING_SECRET");
// But for the purposes of this demo, the exposed secret is:
const SECRET = "Let me know if you found this to be helpful!";
type GitHubWebhookVerificationStatus = {
id: string;
verified: boolean;
};
// Because this uses a native Request,
// it can be used in other contexts besides Oak (e.g. `std/http/serve`):
async function verifyGitHubWebhook(
request: Request,
): Promise<GitHubWebhookVerificationStatus> {
const id = request.headers.get("X-GitHub-Delivery");
// This should be more strict in reality
assert(id, "Not a GH webhhok");
const signatureHeader = request.headers.get("X-Hub-Signature-256");
let verified = false;
if (signatureHeader) {
const payload = await request.clone().text();
verified = await verify(SECRET, payload, signatureHeader);
}
return { id, verified };
}
// Type predicate used to access native Request instance
// Ref: https://github.com/oakserver/oak/issues/501#issuecomment-1084046581
function isNativeRequest(r: ServerRequest): r is NativeRequest {
// deno-lint-ignore no-explicit-any
return (r as any).request instanceof Request;
}
const webhookLogger = new Router().post("/webhook", async (ctx) => {
assert(isNativeRequest(ctx.request.originalRequest));
const status = await verifyGitHubWebhook(ctx.request.originalRequest.request);
console.log(status);
ctx.response.status = 200;
});
const app = new Application()
.use(webhookLogger.routes())
.use(webhookLogger.allowedMethods());
// The port is not important in Deno Deploy
await app.listen({ port: 8080 });

How to prevent async - await freezing in javascript?

Good day I have a custom adonisjs command that pulls from an API.
async handle (args, options) {
// Status
// Open = 1979
// Get all jobs with open status.
const pullJobController = new PullJobsFromJobAdderController;
let token = await pullJobController.get_token();
if(token){
const jobs = await this._getOpenJobs('https://jobs/open-jobs', token , 1979);
}
}
async _getOpenJobs(url, accessToken, status) {
url = url + '?statusId=' + status
const headers = {
'Authorization': 'Bearer ' + accessToken
}
const options = {
method: 'GET',
url: url,
headers: headers
}
return (await rp(options).then(function (result) {
return {
status: true,
info: JSON.parse(result)
}
}).catch(function (error) {
return {
status: false
}
}));
} // _getOpenJobs()
PullJobsFromJobAdderController
async get_token()
{
// This works if directly returning the token.
// return "9ade34acxxa4265fxx4b5x6ss7fs61ez";
const settings = await this.settings();
const jobAdderObject = new this.JobAdder(settings.jobadder['client.id'], settings.jobadder['client.secret'])
const jobadderOauthObject = this.model('JobadderOauth');
const accessInfo = await jobadderOauthObject.jobdderLatestAccess();
let isAccessExpired = await this.checkAccessValidity(accessInfo.created_at);
let accessToken = accessInfo.access_token;
let apiEndpoint = accessInfo.api_endpoint;
if(isAccessExpired === true){
let refreshTokenInfo = await jobAdderObject.refrehToken(accessInfo.refresh_token)
if (refreshTokenInfo.status === true) {
let refreshTokenDetails = JSON.parse(refreshTokenInfo.info)
accessToken = refreshTokenDetails.access_token
apiEndpoint = refreshTokenDetails.api
await jobadderOauthObject.create({
code: accessInfo.code,
access_token: refreshTokenDetails.access_token,
refresh_token: refreshTokenDetails.refresh_token,
scope: 'read write offline_access',
api_endpoint: refreshTokenDetails.api
})
}
}
return accessToken;
} // get_token()
The function async get_token works as expected, it supplies me with a fresh token to be used by the adonisjs command. However it freezes after running the command.
But if I return the string token directly. The custom command handle() works as expected and terminates after running.
Scenario 1: (Directly returning the token string from PullJobsFromJobAdderController)
I run my custom command "adonis pull:jobs" and it runs as expected displaying in the terminal the result of the pulled data from the api.
Terminal is ready to accept another command.
Scenario 2: (Comment out the directly returned string token from PullJobsFromJobAdderController)
I run my custom command "adonis pull:jobs" and it runs as expected
displaying in the terminal the result of the pulled data from the
api.
Terminal is not accepting commands until I press ctrl+c and terminate the current job/command.
Perhaps I am missing something regarding async await calls.
Can someone point / help me to the right direction?
TIA

I got it, for anyone else having this kind of problem with adonis commands:
wrap the task inside your handle in a try... catch block then always have Database.close() and process.exit() in finally.

fetch url with cookies

so im scraping a web and i need to use a specific cookies but i dont know how to exactly use "fetch"
const url="https://www.example.com";
let response = await fetch(url),
html = await response.text();
let $ = cheerio.load(html)
var example= $('.exampleclass').text();
Now i can scrape the web but in case i would have to use a specific cookies i dont know how to put in on the fetch.
In python was something like that
response = requests.get(url, headers=headers, cookies=cookies)
Thank you!

You can add the cookies on the headers on node-fetch, I've made a helper function that you can use for your purposes:
const cookieMaker = object => {
const cookie = [];
for (const [key, value] of Object.entries(object)) {
cookie.push(`${key}=${value}`);
}
return cookie.join('; ');
};
const fetchText = async (url, cookie) => {
const r = await fetch(url, {
headers: {
Cookie: cookieMaker(cookie),
},
});
return await r.text();
};
fetchText('http://someurl.com', { token: 'abc', myValue: 'def' });

Uploading PDF to external API results in blank pages inside PDF

I'm new to Node.js and I need to upload some PDFs to an external API (Zip Forms).
Right now I have the code below but the PDF pages are blank when they arrive at the destination. I tried saving the PDF locally, using the same binary data that I'm sending to the API, and the PDFs are correctly saved.
I am also using setTimeout method here because I cannot find a method that waits for the PDF to read, before sending it to the API.
Also tried binary instead of latin-1 in readFileSync method, but it doesn't change anything.
Code:
const aws = require('aws-sdk');
const https = require('https');
const request = require('request');
const { createWriteStream, readFileSync, writeFileSync } = require('fs');
const s3 = new aws.S3(); // Pass in opts to S3 if necessary
// Look up order and related info.
var order = await Order.findOne({ id })
.populate('agent');
if (createZiplogixTransaction) {
ziplogixTransactionId = await sails.helpers.ziplogix.createZiplogixTransaction.with({
ziplogixContextId: ziplogixContextId,
transactionName: order.propertyStreetAddress + ', ' + order.propertyCity,
// FUTURE: if the transaction helper is updated, include actual order information
// e.g. Primary seller name, property street address, etc.
});
}
if (!order) {
throw 'noSuchOrder';
}
// Permissions
if (this.req.me && this.req.me.accountType !== 'agent' && !ziplogixContextId) {
throw 'forbidden';
}
let savedPdfs = await PdfOrderExternalId.find({ orderId: id });
await PdfOrderExternalId.destroy({
where: { orderId: id }
});
for (const pdf of pdfs) {
let url = await s3.getSignedUrl('getObject', {
Bucket: 'disclosure-pdfs',
Key: pdf.uploadFd,
Expires: 60 * 5
});
let file = createWriteStream(`/tmp/${pdf.slug}.pdf`);
await https.get(url, async (response) => {
await response.pipe(file);
// Need to wait for file to write on disk :|. Doesn't work with await or Promise (Why? IDK)
setTimeout(async () => {
let postData = await readFileSync(`/tmp/${pdf.slug}.pdf`, 'latin1');
let queryString = `Name=${pdf.displayName}&Description=${pdf.displayName}`;
savedPdfs.forEach(item => {
if (item.pdfTemplate === pdf.pdfTemplate) {
queryString += `Id=${item.externalId}`;
}
});
request({
method: 'POST',
url: `${sails.config.custom.ziplogixApiBaseUrl}/transactions/${ziplogixTransactionId}/documents/file?${queryString}`,
headers: {
'X-Auth-ContextID': ziplogixContextId,
'X-Auth-SharedKey': sails.config.custom.ziplogixSharedKey,
'Content-Type': ['application/pdf', 'application/pdf']
},
body: postData
}, async (error, response, body) => {
// code here ...
});
}, 1000);
});
}
await exits.success(Date.now());
Any ideas what I'm doing wrong?
Thank you