We Keep Coding

How to get an image from queued request body (Workbox PWA)

I am using Workbox in my PWA. If app is offline and image posting request fails - it is automatically added to IndexedDB.
const post = (file: File) => {
const formData = new FormData()
formData.append("image", file);
axios.post('http://some.com/upload', formData, {
headers: {
'Content-Type': 'multipart/form-data'
}
}).then(...)
I want to show user all images of queued requests in offline mode. How i get requests:
import { openDB } from 'idb';
const db = await openDB('workbox-background-sync');
const queuedRequests = await db.getAllFromIndex('requests', 'queueName');
Body of request is ArrayBuffer (how request is stored in IndexedDB is showed on the picture). I don't understand how to "extract" image and pass it to
createObjectURL( imageBlob ).
If i decode body like this:
let view = new Uint8Array(body);
let bodyAsString = new TextDecoder().decode(view);
I got post body which contains image bytes after "image/png\r\n\r\n":
------WebKitFormBoundaryfc7IW9qvYqE2SdSm\r\nContent-Disposition: form-data; name="image"; filename="1.png"\r\nContent-Type: image/png\r\n\r\n�PNG\r\n\x1A\n\x00\...
I don't think that parsing payload using regexp, then encoding it is a good approach. How can this be done?

How do I send compressed data from Laravel server to frontend JS (Solved)

Solved: Added the solution I used in an answer below
I have a compressed json file in my storage folder with path storage/app/public/data.json.gz I am trying to send this data to my js frontend via a fetch request. Sending the data is working but I am having trouble decompressing it back to JSON with js to use in my js code. I've read that it might be possible for the browser to automatically decompress this but I'm not sure how to enable it. Also the reason I am trying to send over the data compressed is because it is 130 MB of data that shrinks down to 7 MB when compressed and I am hoping that sending less data will speed up the site for users.
Laravel route that sends compressed file
Route::get('/chunks/{index}', function ($index) {
$path = 'public/chunks_'.$index.'.json.gz';
if (!Storage::exists($path)) {
abort(404);
}
return Storage::response($path);
});
Currently I am using the fetch API to get the data
JS Code
let chunks = await getZipFile("/chunks/0",[]).then((data) => {
return data;
});
public static async getZipFile(url: string, params: any = {}, method = "GET", headers: any = {
"Content-Type": "application/zip",
}){
headers['X-Requested-With'] = 'XMLHttpRequest';
let options: any = {
'method': method,
'headers': headers
};
url += "?" + new URLSearchParams(params).toString();
const result = await fetch(url, options).then((response) => response);
return result;
};
any help would be appreciated, currently I can retrieve the compressed data and convert it to a string with result.text() but I have not been able to figure out how to decompress it. I tried using Zlib to decompress but got an error Can't resolve './zlib_bindings'. So looking for a solution similar to using Zlib (or something similar) to decompress or figuring out to configure the server/browser to automatically decompress.

I ended up taking Moradnejad's answer and used zip.js, here is the updated code
Laravel Route:
Instead of declaring a route to send the file I used a laravel symbolic link to get a .zip file from my public storage. (https://laravel.com/docs/9.x/filesystem#the-public-disk)
Also incase it is helpful here is the command I wrote to create the .zip file from the files in my storage.
public function handle()
{
$fileNames = [];
for($i = 0; $i < 10000/1000; $i++){
array_push($fileNames,'public/chunks_'.$i.'.json');
}
$this->zipFiles($fileNames,'./storage/app/public','./storage/app/public/chunks.zip');
return 0;
}
public function zipFiles($files,$path,$zipFileNameAndPath) {
$zip = new ZipArchive;
$zip->open($zipFileNameAndPath, ZipArchive::CREATE);
foreach ($files as $file) {
$zip->addFile($path.'/'.$file,$file);
}
$zip->close();
}
Updated JS request code, I used result.blob() to return a blob of the data.
public static zipRequest(url: string){
return this.getZipFile(url);
}
public static async getZipFile(url: string, params: any = {}, method = "GET", headers: any = {
"Content-Type": "application/zip",
}){
headers['X-Requested-With'] = 'XMLHttpRequest';
let options: any = {
'method': method,
'headers': headers
};
if ("GET" === method) {
url += "?" + new URLSearchParams(params).toString();
} else {
//add csrf token to post request
options.headers["X-CSRF-TOKEN"] = document.querySelector<HTMLElement>('meta[name="csrf-token"]')!.getAttribute('content');
options.body = JSON.stringify(params);
}
const result = await fetch(url, options).then((response) => response);
return result.blob();
};
Updated JS handle blob result, I am using zip.js to get all 10 json files from the .zip data, and then I am merging the 10 json files together.
import * as zip from "#zip.js/zip.js";
async function getAllChunks() {
let chunks = await Helper.getZipFile("storage/chunks.zip",[]).then( async (data) => {
//console.log(data);
let allChunks: any = [];
let textReader = new zip.BlobReader(data);
let zipReader = new zip.ZipReader(textReader);
let entries = await zipReader.getEntries();
for(let i = 0; i < entries.length; i++){
let entry = entries[i];
if(entry.getData){
let textWriter = new zip.TextWriter();
let jsonString = await entry.getData(textWriter);
let chunks = await JSON.parse(jsonString);
allChunks.push.apply(allChunks, chunks);
}
}
return allChunks;
});
return chunks;
}

You're mixing two ideas. HTTP requests can be compressed and decompressed at a lower level than application level. This means that it will handle compression and decompression by itself, if enabled. See here.
What you have here is a compressed file. No frontend or ajax call would decompress it automatically for you.
Solutions:
Either enable compression for HTTP requests and depend on it to handle compression automatically. So send the uncompressed in this version. This could be helpful.
Or use a frontend library, like 'zip.js', to decompress when you receive the compressed file.

Send canvas.toDataURL images to nodejs

I'm trying to send image from front-end script to my server.
Front-end script:
var img_data = canvas.toDataURL('image/jpg'); // contains screenshot image
// Insert here POST request to send image to server
And I'm trying to accept the data in the backend and store it into req.files to be able to access like this:
const get_image = (req, res) => {
const File = req.files.File.tempFilePath;
}
What way can I do to send the image to the server and get the image like in the example above?

your img_data is a base 64 string, which you can send to server directly in a post request
e.g.
await fetch('/api/path', { method: 'POST', headers: { "content-type": "application/json"}, body: JSON.stringify({ file: img_data }) });
On your backend, you can convert this string to binary, and save to file.
var fs = require('fs');
app.post('/api/path', async (req, res) => {
const img = req.body.file;
var regex = /^data:.+\/(.+);base64,(.*)$/;
var matches = string.match(regex);
var ext = matches[1];
var data = matches[2];
var buffer = Buffer.from(data, 'base64'); //file buffer
.... //do whatever you want with the buffer
fs.writeFileSync('imagename.' + ext, buffer); //if you do not need to save to file, you can skip this step.
....// return res to client
})

You have to convert it to a Blob first, and then append it to a Form. The form would be the body of the request that you send to server.
canvas.toBlob(function(blob){
var form = new FormData(),
request = new XMLHttpRequest();
form.append("image", blob, "filename.png");
request.open("POST", "/upload", true);
request.send(form);
}, "image/png");

Flask-restful api post gives back empty body via javascript

Having a local database running via python, I'm trying to do some api requests to it via a website. First tried both GET's and POST's as python unittest, which worked fine.
Then using javascript; GET function working perfect but my POST function, whatever I do, sends over an empty body to the python function (variable data in python code) or in other words a dict with nothing in it, while I'm passing data through it.
relevant python snippet:
conn = sq3.connect("temp.db", check_same_thread=False)
class Login(Resource):
def post(self):
data = flask.request.form.to_dict()
lst = conn.execute(f"""SELECT AccountID, Role FROM Account
WHERE Email = \"{data['email']}\"
AND PassW = \"{data['passw_hashed']}\"
""").fetchall()
return {"LoginSucces": [{"AccountId": e[0], "Role": e[1]} for e in lst]}
app = flask.Flask(__name__)
cors = CORS(app, resources={r"/*": {"origins": '*'}})
api = Api(app)
api.add_resource(Login, "/login")
app.run(port=8080)
Javascript:
function req_login(){
let email_p = document.getElementById("login_email").value
let passw = document.getElementById("login_passw").value
let data = JSON.stringify({email: email_p,passw_hashed: passw.hashCode()});
const request = new XMLHttpRequest();
request.open("POST", IP+"/login");
request.setRequestHeader("Accept", "application/json");
request.setRequestHeader('Content-Type', 'application/json');
request.send(data);
request.onload = (e) => {
let jsret = JSON.parse(request.response);
let topkey = Object.keys(jsret);
let data_arr = jsret[topkey];
alert(data_arr['AccountId']);
}
}
Tried with manual data in javascript as well to see if reading out the data was the problem, without succes with the following bit of code:
const data = `{email: "tst#gmail.com", passw: "testtest123"}`;
Where does it go wrong, what can I try/change. If you need any more info, send in comment

After quite a bit of debugging, I found the solution myself. Hope it helps someone:
replace data = flask.request.get_json()
with data = flask.request.json

parse HttpClientRequest response

i have another novice (and probably stupid) question. i am using HttpClientRequest and making a post call. also i have the response.
var url = <my url>
var request = new HttpClientRequest(url);
request.header["Content-Type"] = "application/x-www-form-urlencoded";
request.method = "POST";
try
{
request.execute();
var rawResponse = request.response.body.toString();
}
the response from server is in the following format:
{"token":"abc","expires_in":9292,"refresh":"deeDfTTgendj"}
i just need to extract "expires_in" and "refresh" fields from the response

Since that is valid JSON, you can parse it:
var rawResponse = request.response.body.toString(),
objectLiteral = JSON.parse(rawResponse);
var expires_in = objectLiteral['expires_in'],
refresh = objectLiteral['refresh'];
var rawResponse = '{"token":"abc","expires_in":9292,"refresh":"deeDfTTgendj"}';
objectLiteral = JSON.parse(rawResponse);
var expires_in = objectLiteral['expires_in'],
refresh = objectLiteral['refresh'];
console.log(expires_in, refresh);
Note: check out browser support for JSON.parse()