I am using firebase to storage my 5 jpegs.
I need to download them and pass them to function to retrieve EXIF data from them.
To accomplish that tash i have made bellow script which is:
checking how many files are in specific folder in database
getting download url's for every file
looping through every url and executing function wchich is downloading files from those url's
The problem is: first four images are downloaded properly. The last one is allways not downloaded completelly.
Please see bellow code. Note fucntion called getHTML1 is redundant do getHTML, and gives the same result.
I just tried different approach.
TLDR:
How to make fetch wait for last file being completelly downloaded?
import {
getStorage,
ref,
listAll,
getDownloadURL
} from "firebase/storage";
const storage = getStorage();
const listRef = ref(storage, 'images');
const firebaseDownloadHandler = async() => {
function getHTML1(url) {
return new Promise(function(resolve, reject) {
var xhr = new XMLHttpRequest();
xhr.open('get', url);
xhr.responseType = 'blob';
xhr.onload = function() {
var status = xhr.status;
if (status === 200) {
let blob = new Blob([xhr.response], {
type: 'image/jpeg'
})
console.log(xhr.response);
resolve(blob);
} else {
reject(status);
}
};
xhr.send();
});
}
function getHTML(url) {
return new Promise(async function(resolve, reject) {
await fetch(url, {
method: 'GET',
type: 'image/jpeg'
})
.then(res => res.blob())
.then(blob => {
resolve(blob)
});
})
}
const res = await listAll(listRef);
const requests = res.items.map(itemRef => getDownloadURL(itemRef))
const urls = await Promise.all(requests);
const processArray = async() => {
console.log(urls);
const finalResult = [];
return new Promise(async function(resolve, reject) {
for (let i = 0; i < urls.length; i++) {
const result = await getHTML(urls[i + 1]);
finalResult.push(result);
if (finalResult.length === urls.length) {
resolve(finalResult);
}
}
})
};
const downloaded = await processArray();
return await downloaded;
}
export default firebaseDownloadHandler;
result:
20:06:11.499
Array(5) [ "https://firebasestorage.googleapis.com/v0/b/geolocf.appspot.…025.jpg?alt=media&token=8ecbfc6d-07ed-4205-9599-a6e36dd444ed", "https://firebasestorage.googleapis.com/v0/b/geolocf.appspot.…028.jpg?alt=media&token=06a422fa-64f2-482f-9f63-c39aaf1d9354", "https://firebasestorage.googleapis.com/v0/b/geolocf.appspot.…855.jpg?alt=media&token=6ae03b2c-bd82-49fc-bcb6-0de0683e7d50", "https://firebasestorage.googleapis.com/v0/b/geolocf.appspot.…402.jpg?alt=media&token=a22ef4dd-7f79-40aa-90df-57ad73b10248", "https://firebasestorage.googleapis.com/v0/b/geolocf.appspot.…646.jpg?alt=media&token=e04958d0-ed2f-44f6-9931-18644ffbe8b8" ]
firebaseDownloadHandler.js:45
20:06:14.882
Array(5) [ Blob, Blob, Blob, Blob, Blob ]
0: Blob { size: 6428869, type: "image/jpeg" }
1: Blob { size: 7402504, type: "image/jpeg" }
2: Blob { size: 2858717, type: "image/jpeg" }
3: Blob { size: 3045876, type: "image/jpeg" }
4: Blob { size: 2278, type: "text/html; charset=utf-8" }
length: 5
<prototype>: Array []
I'm not sure if this is the cause of the problem, but you're overusing custom promises.
As far as I can see, this function:
function getHTML(url) {
return new Promise(async function(resolve, reject) {
await fetch(url, {
method: 'GET',
type: 'image/jpeg'
})
.then(res => res.blob())
.then(blob => {
resolve(blob)
});
})
}
Can be shorted to this without any change in functionality:
function getHTML(url) {
return fetch(url, {
method: 'GET',
type: 'image/jpeg'
})
.then(res => res.blob())
}
Yeah. Error is not caused by fetch itself.
I wrongly passed urls to fetching function.
Should be
const result = await getHTML(urls[i]);
Instead of
const result = await getHTML(urls[i + 1]);
Related
I have a problem with fetch and Ionic 6. It is not sending image data to the server.
The code is as follows:
let formData = new FormData();
formData.append('photo', pathImage);
fetch(this.MAIN_URL + '/uploadAvatar?token=' + token, { method: 'POST', body: formData})
.then(response => response.json())
.then(data => {
alert(JSON.stringify(data));
})
.catch(error => {
alert(JSON.stringify(error));
})
Thing is, the image data is not being sent to the server. The variable pathImage comes from Ionic's Crop plugin, and it is like this:
file:///storage/emulated/0/Android/data/io.ionic.starter/cache/(random_number).jpg
I think it is not getting the data from pathImage, but how can I change that? I tried turning pathImage into a blob but that didn't work either.
After doing some research, here is the answer to my question:
let formData = new FormData();
const file = await Filesystem.readFile({ path: pathImage.split('?')[0] });
let blob = new Blob([this.fixBinary(atob(file.data))], { type: 'image/jpeg' });
formData.append('photo', blob);
fetch(this.MAIN_URL + '/uploadAvatar?token=' + token, { method: 'POST', body: formData})
.then(response => response.json())
.then(data => {
alert(JSON.stringify(data));
})
.catch(error => {
alert(JSON.stringify(error));
})
Where fixBinary is a function that fixes the binary data coming from the jpg file:
fixBinary (bin: any) {
var length = bin.length;
var buf = new ArrayBuffer(length);
var arr = new Uint8Array(buf);
for (var i = 0; i < length; i++) {
arr[i] = bin.charCodeAt(i);
}
return buf;
}
Now, if I pick and crop a file, the end result is uploaded to the server just fine.
I want to save the pdf to Cloudant. With the code below, I get an error opening the Attachment in Cloudant. "An error was encountered when processing this file"
I can put fake string data in the "._attachments[name].data" field and it will save.
The Cloudant docs say the data content needs to be in base64 and that is what I am attempting.
Cloudant says "The content must be provided by using BASE64 representation"
function saveFile() {
var doc = {};
var blob = null;
//fileName is from the input field model data
var url = fileName;
fetch(url)
.then((r) => r.blob())
.then((b) => {
blob = b;
return getBase64(blob);
})
.then((blob) => {
console.log(blob);
let name = url._rawValue.name;
doc._id = "testing::" + new Date().getTime();
doc.type = "testing attachment";
doc._attachments = {};
doc._attachments[name] = {};
doc._attachments[name].content_type = "application/pdf";
doc._attachments[name].data = blob.split(",")[1];
console.log("doc: ", doc);
})
.then(() => {
api({
method: "POST",
url: "/webdata",
auth: {
username: process.env.CLOUDANT_USERNAME,
password: process.env.CLOUDANT_PASSWORD,
},
data: doc,
})
.then((response) => {
console.log("result: ", response);
alert("Test has been submitted!");
})
.catch((e) => {
console.log("e: ", e);
alert(e);
});
console.log("finished send test");
});
}
function getBase64(file) {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.readAsDataURL(file);
reader.onload = () => resolve(reader.result);
reader.onerror = (error) => reject(error);
});
}
any ideas?
Thanks
CouchDB, and by extension Cloudant, has a means of handling a "multi-part" request where the JSON document and the attachments are sent in the same request. See https://docs.couchdb.org/en/3.2.2/api/document/common.html#put--db-docid
They are modelled in CouchDB's Nano project here: https://www.npmjs.com/package/nano#multipart-functions
const fs = require('fs');
fs.readFile('rabbit.png', (err, data) => {
if (!err) {
await alice.multipart.insert({ foo: 'bar' }, [{name: 'rabbit.png', data: data, content_type: 'image/png'}], 'mydoc')
}
});
Alternatively, you could write the document first and add the attachment in a supplementary request. Using the current Cloudant SDKs:
write document https://cloud.ibm.com/apidocs/cloudant?code=node#putdocument
write attachment https://cloud.ibm.com/apidocs/cloudant?code=node#putattachment
const doc = {
a: 1,
b: 2
}
const res = await service.putDocument({
db: 'events',
docId: 'mydocid',
document: doc
})
const stream = fs.createReadStream('./mypdf.pdf')
await service.putAttachment({
db: 'events',
docId: 'mydocid',
rev: res.result.rev, // we need the _rev of the doc we've just created
attachmentName: 'mypdf',
attachment: stream,
contentType: 'application/pdf'
})
I found out I was doing too much to the PDF file. No need to make to blob then convert to base64.
Only convert to base64.
async function sendFiles() {
try {
const url = fileName;
const doc = {};
doc._attachments = {};
doc._id = "testing::" + new Date().getTime();
doc.type = "testing attachment";
for (let item of url._value) {
const blob2 = await getBase64(item);
let name = item.name;
doc._attachments[name] = {};
doc._attachments[name].content_type = item.type;
doc._attachments[name].data = blob2.split(",")[1];
}
const response = await api({
method: "POST",
url: "/webdata",
data: doc,
});
} catch (e) {
console.log(e);
throw e; // throw error so caller can see the error
}
console.log("finished send test");
fileName.value = null;
}
function getBase64(file) {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.readAsDataURL(file);
reader.onload = () => resolve(reader.result);
reader.onerror = (error) => reject(error);
});
}
This works for me.
I am currently attempting to get an image upload component built in Zapier but am having issues getting it to work. I have to be able to download an image, and then POST it to a new endpoint without storing it locally. Currently, the closest I've been able to do is get an IncomingMessage to POST, but I know that's not right.
Does anyone have any advice?
let FormData = require('form-data');
let http = require('https');
const makeDownloadStream = (url) => {
new Promise((resolve, reject) => {
http.request(url, resolve).on('error', reject).end();
});
}
const makeUploadStream = (z, bundle, options) => {
var imageRequest = options;
const promise = z.request(imageRequest);
return promise.then((response) => {
return response.data;
});
}
const addAttachment = async (z, bundle) => {
/*var request = {
'url': bundle.inputData.attachment
};
const promiseAt = z.request(request);
return promiseAt.then((stream) => {*/
const form = new FormData();
var data = `{"type": "records", "attributes": {"form_id": ${bundle.inputData.form_id}}}`
const stream = await makeDownloadStream(bundle.inputData.attachment);
form.append(`field_${bundle.inputData.field_id}`, stream);
form.append('data', data);
var request = {
'url': bundle.inputData.url,
'method': 'PUT',
'headers': {
'Content-Type': `multipart/form-data; boundary=${form.getBoundary()}`
},
'body': form
};
const response = await makeUploadStream(z, bundle, request);
return response;
//});
}
I figured it out myself. For anyone needing to upload an image on Zapier, here it is:
let FormData = require('form-data');
const makeDownloadStream = (z, bundle) => {
var imageRequest = {
'url': bundle.inputData.attachment,
'method': 'GET',
'raw': true
};
const promise = z.request(imageRequest);
return promise.then(async (response) => {
var buffer = await response.buffer();
return {
'content-type': response.headers.get('content-type'),
'content': buffer,
'filename': response.headers.get('content-disposition').replace('attachment; filename="', '').replace('"', '')
}
});
}
const addAttachment = async (z, bundle) => {
const form = new FormData();
const content = await makeDownloadStream(z, bundle);
form.append(`field_${bundle.inputData.field_id}`, Buffer.from(content.content.toString('binary'), 'binary'), {
filename: content.filename
});
const request = {
'url': `${bundle.inputData.url}/api/records/${bundle.inputData.record_id}`,
'method': 'PUT',
'headers': {
'Content-Type': `multipart/form-data; boundary=${form.getBoundary()}`,
'Content-Length': form.getLengthSync()
},
'body': form
};
const promise = z.request(request);
return promise.then((response) => {
return response.data;
});
}
I've tried the imgbb-uploader npm package but I only got it to work with other image URLs and not local files.
axios
.post("https://api.imgbb.com/1/upload", {
image: BASE_64_STRING,
name: file.name,
key: process.env.MY_API_KEY,
})
.then((res) => console.log(res))
.catch((err) => console.log(err));```
What about this:
// fileinput is the file/index 0 from input-type-file eg: e.target.myfileinput.files[0]
const uploadImg = ( fileinput ) => {
const formData = new FormData();
formData.append( "image", fileinput ); // has to be named 'image'!
let apiresponse = axios.post( 'https://api.imgbb.com/1/upload?key=your-api-key', formData )
.then( res => { return res.data } )
.catch( error => { return null } )
return apiresponse;
}
//From graph ql perspective
const { createReadStream, filename } = await file;
const url = "<URL_TO_IMAGE_SERVER&key=<YOUR_API_KEY>";
const stream = createReadStream();
const form = new FormData();
form.append("image", stream, filename);
try {
const response = await axios.post(url, form, {
headers: { ...form.getHeaders() },
});
console.log({ response });
return { Location: response.data.display_url };
} catch (error) {
return { ...error };
}
I am uploading an image from the local (ios) filesystem to Firebase using RNFetchBlob
Although my image is being deleted, my blob is remaining in the filesystem after closing it.
I tried unlinking the blob but it still remains
function createBlob(uri, mediaType) {
const Blob = RNFetchBlob.polyfill.Blob;
const typePrefix = isImage(mediaType) ? 'image' : 'video';
window.Blob = Blob;
window.XMLHttpRequest = RNFetchBlob.polyfill.XMLHttpRequest;
return new Promise((resolve, reject) => {
const uid = guid();
const mediaRef = firebase.storage().ref('users').child(`${uid}.${mediaType}`);
let uploadBlob = null;
Blob.build(uri, { type: `${typePrefix}/${mediaType}` })
.then((blob) => {
uploadBlob = blob;
return mediaRef.put(blob, { type: `${typePrefix}/${mediaType}` });
})
.then((response) => {
uploadBlob.close();
resolve(response);
})
.catch(error => {
reject(error);
});
});
}
This is how it looks in my simulator folder after closing the blob
After the createBlob I am deleting the file from local storage using RNFetchBlob.fs.unlink(path)
I don't think you're close() is working because the uploadBlob variable was created in the previous closure, but not passed on to the next .then block. I'm thinking this might work...
Blob.build(uri, { type: `${typePrefix}/${mediaType}` })
.then((blob) => {
uploadBlob = blob;
return ({
response: mediaRef.put(blob, { type: `${typePrefix}/${mediaType}` }),
blob: uploadBlob
})
.then(({response, blob}) => {
blob.close();
resolve(response);
})