No matter what I try, I cannot get my data to print as it comes in. I tried this tutorial:
https://jakearchibald.com/2016/streams-ftw/
This SO post: How to handle streaming data using fetch?
(and a several other SO posts I can't recall)
And, tried reading the docs: https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams
test.py
import time
import sys
for i in range(1,10):
print('test', i)
time.sleep(.5)
sys.stdout.flush()
server.js
firmwareRouter.get('/run_test', (req, res) => {
res.writeHead(200, { 'Content-Type': 'text/event-stream', 'Cache-control': 'no-cache' });
const pathToExample = '/server/path/to/test.py';
const { spawn } = require('child_process');
const pythonProcess = spawn('python', [pathToExample]);
pythonProcess.stdout.on('data', (data) => {
// console.log(data.toString());
res.write(data);
});
pythonProcess.on('close', (code) => {
res.end('Process has ended');
});
});
Firmware.js (Version 1)
export default function Firmware(props) {
const [data, setData] = useState('');
async function runTest() {
try {
const url = 'api/firmware/run_test'
const request = {
method: 'GET',
headers: {
'Content-Type': 'text/event-stream'
},
}
const res = await fetch(url, request);
const reader = res.body.getReader();
const chunks = [];
const decoder = new TextDecoder('utf-8');
let done, value;
while (!done) {
({ value, done } = await reader.read());
value = decoder.decode(value);
if (done) {
return chunks;
}
console.log(value);
chunks.push(value);
setData(value);
}
} catch (err) {
console.log('frontend:Firmware', err);
}
}
return (
<Fragment>
{data}
<button onClick={() => runTest()}>Run Test </button>
</Fragment >
)
}
Firmware.js (Version 2)
export default function Firmware(props) {
const [data, setData] = useState('');
async function runTest() {
const url = 'api/firmware/run_test'
const request = {
method: 'GET',
headers: {
'Content-Type': 'text/event-stream'
},
}
fetch(url, request).then((res) => {
let reader = res.body.getReader();
let decoder = new TextDecoder();
reader.read().then(function processResult(result) {
if (result.done) return;
console.log(decoder.decode(result.value, { stream: true }))
return reader.read().then(processResult);
})
})
}
// return ()
No matter what, it only prints to the browser once the process has finished. I need it to print as the python script prints. What am I missing here?
Okay... So, one "solution" is to build it. After I built the app (npm run build), it worked.
Note: I am serving my app from express/node.
It seems to be a webpack or middleware issue between the backend and frontend and using a Proxy. I am going to try to use CORS, turn off React's proxy and go from there.
Related
Im implementing a notification system for my app and its working well but it is not real time, How can I implement SSE using my setup below
Client Side
const [fetchnotification, setfetchnotification] = useState('');
const getadminnotif = async()=>{
const api = axios.create({
baseURL: http://localhost:3000/books,
withCredentials: true,
headers: { 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json' }
});
const query = await api.get(`/adminNotifications?page=${notifpage}`)
setfetchnotification(query.data.findnotifs)
}
useEffect(async()=>{
getadminnotif()
})
return(
// here i mapped the fetchnotification array to get the notifications
)
Server Side
here im using express router to get the api calls from client
const {getadminNotifs} = require('../controllers/bookControllers');
router.get('/adminNotifications', getadminNotifs )
getadminNotifs
const getadminNotifs = asyncHandler(async (req, res) => {
const page = parseInt(req.query.page) || 1;
const pageSize = 4;
const skip = (page - 1) * pageSize;
const findallnotifs = await AdminNotif.find().sort({'dateAdded':-1}).exec()
const findnotifs = await AdminNotif.find().sort({'dateAdded':-1}).skip(skip).limit(pageSize).exec()
const notiflength = findallnotifs.length
var unreadcounts = findallnotifs.filter(({ status }) => status === 'unread').length
const pages = Math.ceil(notiflength / pageSize)
res.status(200).json({ findnotifs, notiflength, pages, unreadcounts })
})
what ive tried so far is this but its not working
I changed the getadminotif function to call an eventsource and then on the client side,instead of using router, i fetched the api calls in app.js
const getadminnotif = async()=>{
const eventSource = new EventSource(`http://localhost:3000/realtime`,{
headers: {
Accept: "text/event-stream",
},
});
eventSource.onmessage = (e) => setfetchnotification(e.data.findnotifs);
eventSource.onerror = err => {
console.log('EventSource error: ', err);
};
return () => {
eventSource.close();
};
}
app.get("/realtime", function (req, res) {
res.writeHead(200, {
Connection: "keep-alive",
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
});
setInterval(() => {
res.write(
`data: ${getadminNotifs}`
);
res.write("\n\n");
}, 5000);
});
I am trying to create a process that uploads an image, previews it once, and then uploads it to Imgur if the image is OK.
The code is as follows.
const [img, setImg] = useState([])
const previewImg = ({ target: { files } }) => {
if (img.length > 5) return
const reader = new FileReader()
reader.onload = ({ target: { result } }) => {
setImg((img) => [...img, { id: generateID(), src: result }])
}
reader.readAsDataURL(files[0])
}
const uploadImugr = async (e) => {
e.preventDefault();
const base64 = img[0].src.toString().replace(/data:.*\/.*;base64,/, '');
const res = await fetch('/api/upload/', {
method: 'POST',
body: base64,
});
console.log(await res.json());
}
return (
<>
<input type="file" onChange={previewImg} />
{img.length > 0 && img.map((item) => {
return <img key={item.id} src={item.src} />}
}
<button onClick={uploadImgur}>Upload Imgur</button>
</>
)
The following is the API route for next.js.
Imgur API
const uploadImugrAPI = async (req: NextApiRequest, res: NextApiResponse) => {
const formData = new FormData();
　formData.append('image', req.body);
const resImgur = await fetch("https://api.imgur.com/3/upload", {
method: 'POST',
headers: {
Authorization: 'Client-ID MY-CLIEND-ID',
},
body: formData,
})
res.status(200).json(resImgur.json());
};
export default uploadImugrAPI;
When the above API is executed, the following error message will be displayed.
POST http://localhost:3000/api/upload 500 (Internal Server Error)
Uncaught (in promise) SyntaxError: Unexpected token I in JSON at position 0
I'm new to Next.js and external APIs, so I'm not sure what keywords to search on Google for to solve this problem.
Please help me.
Thank you.
Add
When I tried with Postman, I was able to upload images to Imugr by passing a binary file.
Therefore, I changed the code as follows to pass a binary file instead of base64 and tried it.
const [imgArray, setImgArray] = useState([])
+ const [srcArray, setSrcArray] = useState([])
const uploadImg = ({ target: { files } }) => {
if (imgArray.length > 5) return
+ setImgArray((imgArray) => [...imgArray, files[0]])
const reader = new FileReader()
reader.onload = ({ target: { result } }) => {
const uploadImgSrc = result.toString()
setSrcArray((srcArray) => [
...srcArray,
{ id: generateID(), src: uploadImgSrc.toString() },
])
formRef.current.inputImg.value = ''
}
reader.readAsDataURL(files[0])
}
const uploadImugr = async (e) => {
e.preventDefault();
+ const formData = new FormData();
+ formData.append("image", imgArray[0])
const res = await fetch('/api/upload/', {
method: 'POST',
body: formData,
});
console.log(await res.json());
}
The result was that the following error was displayed in the console.
POST http://localhost:3000/api/upload 500 (Internal Server Error)
Request failed with status code 500
After 2 days of frustration, I've patched together a solution based on several answers I stumbled upon. Convert the file to base64 client side and send that as json to the API.
//client.tsx
async function submit(e: React.FormEvent<HTMLFormElement>) {
e.preventDefault();
if (!file) return;
let base64Img = await getBase64(file);
if (typeof base64Img == 'string') {
base64Img = base64Img.replace(/^data:.+base64,/, '')
}
const result = await fetch('/api/upload', {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({image: base64Img}),
})
const response = await result.json() // response.data is an object containing the image URL
}
function getBase64(file: File): Promise<string | ArrayBuffer | null> {
return new Promise((resolve, reject) => {
const reader = new FileReader()
reader.readAsDataURL(file)
reader.onload = () => resolve(reader.result)
reader.onerror = error => reject(error)
})
}
//upload.ts
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
const fd = new FormData();
fd.append('image', req.body.image)
fd.append('type', 'base64')
const response = await fetch('https://api.imgur.com/3/image', {
method: "POST",
headers: {
Authorization: "Client-ID process.env.IMGUR_ID",
},
body: fd,
redirect: 'follow',
})
const data = await response.json();
return res.json(data)
}
Also I found using https://api.imgur.com/3/image instead of https://api.imgur.com/3/upload better as the errors were more helpful.
I have multiple API calls with fairly lengthy, yet similar, response/error handling for each call.
What is the best non-repetitive ways to make multiple independent api calls that update state using fetch?
Copying and pasting 40+ instances of fetch doesn't seem right.
I want to avoid doing this ....
fetch(url,options)
.then((response) => {
// ...
return response.json
})
.then((data) => {
setState(data)
//...
})
.catch((err) => {
//Error logic here
})
Here's what I've done so far:
I made (found and modified) a useFetch hook...
useFetch.ts
//Only calls fetch() when .load() is called.
const useFetch = (path : string, HttpMethod : string, dependencies : any = [] , body : {} | undefined = undefined) => {
const history = useHistory()
const [response, setResponse] = useState<any>({});
const [error, setError] = useState<string>("");
const [isLoading, setIsLoading] = useState<boolean>(false);
const [controller, setController] = useState(2)
const [isReady, setIsReady] = useState<any>(false)
const load = ():void => {
setError("")
//This prevents useEffect from triggering on declaration.
if (isReady) {
//Math.random() is just to get useEffect to trigger.
setController(Math.random())
}
}
const token = localStorage.getItem("token");
let requestOptions:any = {
method: HttpMethod,
headers: {
"Content-Type": "application/json",
"Access-Control-Allow-Origin": "* always",
Authorization: "Token " + token,
},
};
if (body !== undefined) {
requestOptions["body"] = {
body: JSON.stringify(body)
}
}
const URI = BASE_URI + path
useEffect(() => {
const fetchData = async () => {
if (controller !== 2) {
setIsLoading(true);
try {
const res = await fetch(URI, requestOptions);
const json = await res.json();
if (json?.action == "ENFORCE_BILLING" ) {
history.push(BILLING_CREDENTIALS_PATH, { enforceBillingPopUp: true });
}
if (json?.action == "ENFORCE_SMS_CONFIRMATION") {
// Should we log user out, as well?
history.push(CONFIRMATION_CODE_PATH)
}
if (res.ok) {
setResponse(json);
setIsLoading(false)
} else {
setError(json)
setIsLoading(false)
}
} catch (err) {
setError(err);
// Error logic here...
}
}
}
};
fetchData()
setIsReady(true)
}, [controller, ...dependencies]);
return { response, setResponse ,error, isLoading, load, isReady };
};
Component.tsx
//Inside react functional component...
// Prepares to fetch data from back-end
const data1 = useFetch(PATH1, "GET");
const data2 = useFetch(PATH2, "GET");
const data3 = useFetch(PATH3, "GET");
useEffect(() => {
// Initial on load data fetch
// .load() fetches data
data1.load();
data2.load();
data3.load();
}, [activeReservations.isReady]);
// Sort data depending on sort selection
...
Is useFetch considered bad practice? What are the advantages of using Redux, instead?
Any help would be greatly appreciated. Thanks.
I am fairly new to Jest and am struggling to understand correct way of testing code that uses nested dependencies like MongoDb.
Here is my file hierarchy
src/getOrder/index.js <- code I want to test
src/singletons/index.js <- a singleton that will be created and used by getOrder/index.js
My getOrder/index.js looks something like this
const { getSuper, catchError } = require('../singletons');
module.exports = async function (context, req)
{
let response = {}
if (req.body.guid)
{
try
{
response = await getOrder(context, req);
}
catch (err)
{
response = catchError(context, err)
}
}
else
{
response.data = 'Missing Payload'
response.status = 400;
}
context.res =
{
status: response.status,
headers: { 'Content-Type': 'application/json' },
body: response
}
}
async function getOrder(context, req)
{
//get API singleton
let sd = await getSuper()
//get order
let res = await sd.get(`/orders/${req.body.guid}`);
//return
return { 'status': res.status, 'data': res.data };
}
And then my singletons/index.js looks like this
const axios = require('axios');
const https = require('https');
const MongoClient = require('mongodb').MongoClient;
const DateTime = require('luxon').DateTime
const dbOptions =
{
useUnifiedTopology: false,
useNewUrlParser: true
};
//singleton variables
//mongo db connection
let db;
//super connection
let sd = {};
async function getDb()
{
//establish db connection if one isn't present
if (!db)
{
const client = new MongoClient(process.env.dbUri, dbOptions);
db = await (await client.connect()).db(process.env.dbName)
}
return db;
}
async function getSuper()
{
if (sd.exp && sd.exp > DateTime.local().toString().substr(0, 19))
{
return sd.instance;
}
else
{
//get db connection
let db = await getDb()
//get token
let token = await db.collection('secrets').findOne({ 'name': process.env.SDAccessToken })
//set exp time
sd.exp = token.exp;
if (!sd.instance)
{
//creat axios instance
sd.instance = axios.create({
baseURL: process.env.SDApiUrl,
httpsAgent: new https.Agent({ keepAlive: true }),
headers: { 'Content-Type': 'application/json' }
})
//set token
sd.instance.defaults.headers.common['Authorization'] = `Bearer ${token.value}`
}
else
{
//update token
sd.instance.defaults.headers.common['Authorization'] = `Bearer ${token.value}`
}
return sd.instance;
}
}
function catchError(context, err)
{
let response = {}
if (err.response && err.response.data)
{
response.status = err.response.status
response.data = err.response.data
context.log(response)
}
else
{
context.log(err)
response.status = 500
response.data = err
}
return response;
}
module.exports =
{
getDb,
getSuper,
catchError
}
Notice how when the sd singleton is initialized it also uses the db singleton (it calls getDb)
So I am unsure how to mock either of these from my test file. I AM trying to use #shelf/jest-mongodb to mock my database, however I am quite unsure how to turn into a singleton under a mock getDb function that will live somewhere outside of the test (so it can be reused in other tests, etc)
Essentially I figured out that as long as you have the mocking package I was using (#shelf/jest-mongodb) configured in the jest.config.js file and the jest-mongodb-config.js file then it will mock the db functionality where it is initialized. Whether it's directly in your tests, or 5 nest submodules deep in your project folder
I am working on solutions using which i can send desktop push notification to subscribed clients.
I have created basic solution in where whenever user click on button i ask user for whether they want to allow notifications for my app or not!
I am getting an error of "Registration failed - permission denied" whenever i click on button for first time.
So that i am not able to get required endpoints to save at backend
Here is my code
index.html
<html>
<head>
<title>PUSH NOT</title>
<script src="index.js"></script>
</head>
<body>
<button onclick="main()">Ask Permission</button>
</body>
</html>
index.js
const check = () => {
if (!("serviceWorker" in navigator)) {
throw new Error("No Service Worker support!");
} else {
console.log("service worker supported")
}
if (!("PushManager" in window)) {
throw new Error("No Push API Support!");
} else {
console.log("PushManager worker supported")
}
};
const registerServiceWorker = async () => {
const swRegistration = await navigator.serviceWorker.register("/service.js?"+Math.random());
return swRegistration;
};
const requestNotificationPermission = async () => {
const permission = await window.Notification.requestPermission();
// value of permission can be 'granted', 'default', 'denied'
// granted: user has accepted the request
// default: user has dismissed the notification permission popup by clicking on x
// denied: user has denied the request.
if (permission !== "granted") {
throw new Error("Permission not granted for Notification");
}
};
const main = async () => {
check();
const swRegistration = await registerServiceWorker();
const permission = await requestNotificationPermission();
};
// main(); we will not call main in the beginning.
service.js
// urlB64ToUint8Array is a magic function that will encode the base64 public key
// to Array buffer which is needed by the subscription option
const urlB64ToUint8Array = base64String => {
const padding = "=".repeat((4 - (base64String.length % 4)) % 4);
const base64 = (base64String + padding)
.replace(/\-/g, "+")
.replace(/_/g, "/");
const rawData = atob(base64);
const outputArray = new Uint8Array(rawData.length);
for (let i = 0; i < rawData.length; ++i) {
outputArray[i] = rawData.charCodeAt(i);
}
return outputArray;
};
const saveSubscription = async subscription => {
console.log("Save Sub")
const SERVER_URL = "http://localhost:4000/save-subscription";
const response = await fetch(SERVER_URL, {
method: "post",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify(subscription)
});
return response.json();
};
self.addEventListener("activate", async () => {
try {
const applicationServerKey = urlB64ToUint8Array(
"BFPtpIVOcn2y25il322-bHQIqXXm-OACBtFLdo0EnzGfs-jIGXgAzjY6vNapPb4MM1Z1WuTBUo0wcIpQznLhVGM"
);
const options = { applicationServerKey, userVisibleOnly: true };
const subscription = await self.registration.pushManager.subscribe(options);
console.log(JSON.stringify(subscription))
const response = await saveSubscription(subscription);
} catch (err) {
console.log(err.code)
console.log(err.message)
console.log(err.name)
console.log('Error', err)
}
});
self.addEventListener("push", function(event) {
if (event.data) {
console.log("Push event!! ", event.data.text());
} else {
console.log("Push event but no data");
}
});
Also i have created a bit of backend as well
const express = require("express");
const cors = require("cors");
const bodyParser = require("body-parser");
const webpush = require('web-push')
const app = express();
app.use(cors());
app.use(bodyParser.json());
const port = 4000;
app.get("/", (req, res) => res.send("Hello World!"));
const dummyDb = { subscription: null }; //dummy in memory store
const saveToDatabase = async subscription => {
// Since this is a demo app, I am going to save this in a dummy in memory store. Do not do this in your apps.
// Here you should be writing your db logic to save it.
dummyDb.subscription = subscription;
};
// The new /save-subscription endpoint
app.post("/save-subscription", async (req, res) => {
const subscription = req.body;
await saveToDatabase(subscription); //Method to save the subscription to Database
res.json({ message: "success" });
});
const vapidKeys = {
publicKey:
'BFPtpIVOcn2y25il322-bHQIqXXm-OACBtFLdo0EnzGfs-jIGXgAzjY6vNapPb4MM1Z1WuTBUo0wcIpQznLhVGM',
privateKey: 'mHSKS-uwqAiaiOgt4NMbzYUb7bseXydmKObi4v4bN6U',
}
webpush.setVapidDetails(
'mailto:janakprajapati90#email.com',
vapidKeys.publicKey,
vapidKeys.privateKey
)
const sendNotification = (subscription, dataToSend='') => {
webpush.sendNotification(subscription, dataToSend)
}
app.get('/send-notification', (req, res) => {
const subscription = {endpoint:"https://fcm.googleapis.com/fcm/send/dLjyDYvI8yo:APA91bErM4sn_wRIW6xCievhRZeJcIxTiH4r_oa58JG9PHUaHwX7hQlhMqp32xEKUrMFJpBTi14DeOlECrTsYduvHTTnb8lHVUv3DkS1FOT41hMK6zwMvlRvgWU_QDDS_GBYIMRbzjhg",expirationTime:null,keys:{"p256dh":"BE6kUQ4WTx6v8H-wtChgKAxh3hTiZhpfi4DqACBgNRoJHt44XymOWFkQTvRPnS_S9kmcOoDSgOVD4Wo8qDQzsS0",auth:"CfO4rOsisyA6axdxeFgI_g"}} //get subscription from your databse here.
const message = 'Hello World'
sendNotification(subscription, message)
res.json({ message: 'message sent' })
})
app.listen(port, () => console.log(`Example app listening on port ${port}!`));
Please help me
Try the following code:
index.js
const check = () => {
if (!("serviceWorker" in navigator)) {
throw new Error("No Service Worker support!");
} else {
console.log("service worker supported")
}
if (!("PushManager" in window)) {
throw new Error("No Push API Support!");
} else {
console.log("PushManager worker supported")
}
};
const saveSubscription = async subscription => {
console.log("Save Sub")
const SERVER_URL = "http://localhost:4000/save-subscription";
const response = await fetch(SERVER_URL, {
method: "post",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify(subscription)
});
return response.json();
};
const urlB64ToUint8Array = base64String => {
const padding = "=".repeat((4 - (base64String.length % 4)) % 4);
const base64 = (base64String + padding)
.replace(/\-/g, "+")
.replace(/_/g, "/");
const rawData = atob(base64);
const outputArray = new Uint8Array(rawData.length);
for (let i = 0; i < rawData.length; ++i) {
outputArray[i] = rawData.charCodeAt(i);
}
return outputArray;
};
const registerServiceWorker = async () => {
return navigator.serviceWorker.register("service.js?"+Math.random()).then((swRegistration) => {
console.log(swRegistration);
return swRegistration;
});
};
const requestNotificationPermission = async (swRegistration) => {
return window.Notification.requestPermission().then(() => {
const applicationServerKey = urlB64ToUint8Array(
"BFPtpIVOcn2y25il322-bHQIqXXm-OACBtFLdo0EnzGfs-jIGXgAzjY6vNapPb4MM1Z1WuTBUo0wcIpQznLhVGM"
);
const options = { applicationServerKey, userVisibleOnly: true };
return swRegistration.pushManager.subscribe(options).then((pushSubscription) => {
console.log(pushSubscription);
return pushSubscription;
});
});
};
const main = async () => {
check();
const swRegistration = await registerServiceWorker();
const subscription = await requestNotificationPermission(swRegistration);
// saveSubscription(subscription);
};
service.js
self.addEventListener("push", function(event) {
if (event.data) {
console.log("Push event!! ", event.data.text());
} else {
console.log("Push event but no data");
}
});
I can think of three reasons that the permission is denied
1) your site is not on https (including localhost that is not on https), the default behaviour from chrome as far as i know is to block notifications on http sites. If that's the case, click on the info icon near the url, then click on site settings, then change notifications to ask
2) if you are on Safari, then safari is using the deprecated interface of the Request permission, that is to say the value is not returned through the promise but through a callback so instead of
Notification.requestPermission().then(res => console.log(res))
it is
Notification.requestPermission(res => console.log(res))
3) Your browser settings are blocking the notifications request globally, to ensure that this is not your problem run the following code in the console (on a secured https site)
Notification.requestPermission().then(res => console.log(res))
if you receive the alert box then the problem is something else, if you don't then make sure that the browser is not blocking notifications requests