I need to upload N files, with some data attached to each file, like the name of the recipient of the file.
To do that, I have something like this:
let form = new FormData();
for (let file of file_list) {
form.append('files', file);
form.append('metadata', JSON.stringify(file.metadata));
}
I send that using a simple axios POST, and on the server side, I match each file with its metadata using the index of the file in the list.
It works but is not super reliable.
Is there a reliable way to upload a list of files along with attached metadata?
I'd rather avoid converting to base64 due to the size of the files which can already be pretty high.
When appending a file you can use the third argument to set a filename.
By generating a unique name, you can relate it to your metadata.
let uniqueId = 0;
const metadata = {};
for (let file of file_list) {
uniqueId++;
const fileName = `file-${uniqueId}.bin`;
form.append('files', file, fileName);
metadata[fileName] = file.metadata;
}
form.append('metadata', JSON.stringify(metadata));
Related
I have a process where a client uploads a document. This document can be in the form of a PDF, JPG or PNG file only and it should be reuploaded once a year (it is an errors and omissions insurance policy).
I am saving this file in a container.
For deleting files from anywhere at the application, I have this function (Node):
deleteFromBlob = async function (account, accountKey, containerName, blobFolder, blobName) {
try {
const {
BlobServiceClient,
StorageSharedKeyCredential
} = require("#azure/storage-blob");
const sharedKeyCredential = new StorageSharedKeyCredential(account, accountKey);
const blobServiceClient = new BlobServiceClient(
`https://${account}.blob.core.windows.net`,
sharedKeyCredential
);
const containerClient = blobServiceClient.getContainerClient(containerName);
const blockBlobClient = containerClient.getBlockBlobClient(blobFolder + '/' + blobName);
const uploadblobResponse = await blockBlobClient.deleteIfExists()
return true
}
catch(e) {
return false
}
}
And this works perfect when I know the file name and extension I want to delete, like "2448.pdf":
let deleteFile = await utils.deleteFromBlob(account, accountKey, "agents", "/eopolicies/", userData.agentid.toString() + ".pdf" )
But the problem Im facing is that the function above is to delete a file I know exists; for example, if the agent ID is 2448 and he uploads "policy.pdf" I save it as "2448.pdf" for easy file identification.
The problem Im facing is if the agent uploaded a .PNG last year. a .DOC a year before, and a .PDF now. If that's the case, I want to delete 2448.* and keep only the latest version of the document.
So I tried changing my function to
let deleteFile = await utils.deleteFromBlob(account, accountKey, "agents", "/eopolicies/", userData.agentid.toString() + ".*" )
And of course it is not working...
I tried to find a solution and all I found is one to list the content of a folder, then loop it and delete the specific file I want; but that will not work for me since there are 37,000 EO policies on that folder.
Is there a way to delete files with a specific name, and whatever extension?
Thanks.
I've never tried using a wildcard on the extension side of the file name. However, I would iterate through the files in the directory and find the one that contains the specific string you are looking for. Get it's index, and delete from there.
I am trying to use the MS Graph API and ReactJS to download a file from SharePoint and then replace the file. I have managed the download part after using the #microsoft.graph.downloadUrl value. Here is the code that gets me the XML document from SharePoint.
export async function getDriveFileList(accessToken,siteId,driveId,fileName) {
const client = getAuthenticatedClient(accessToken);
//https://graph.microsoft.com/v1.0/sites/{site-id}/drives/{drive-id}/root:/{item-path}
const files = await client
.api('/sites/' + siteId + '/drives/' + driveId + '/root:/' + fileName)
.select('id,name,webUrl,content.downloadUrl')
.orderby('name')
.get();
//console.log(files['#microsoft.graph.downloadUrl']);
return files;
}
When attempting to upload the same file back up I get a 404 itemNotFounderror return. Because this user was able to get it to work I think I have the MS Graph API correct, although I am not sure I'm translating correctly to ReactJS syntax. Even though the error message says item not found I think MS Graph might actually be upset with how I'm sending the XML file back. The Microsoft documentation for updating an existing file state the contents of the file in a stream should be returned. Since I've loaded the XML file into the state I'm not entirely sure how to send it back. The closest match I found involved converting a PDF to a blob so I tried that.
export async function putDriveFile(accessToken,siteId,itemId,xmldoc) {
const client = getAuthenticatedClient(accessToken);
// /sites/{site-id}/drive/items/{item-id}/content
let url = '/sites/' + siteId + '/drive/items/' + itemId + '/content';
var convertedFile = null;
try{
convertedFile = new Blob(
[xmldoc],
{type: 'text/xml'});
}
catch(err) {
console.log(err);
}
const file = await client
.api(url)
.put(convertedFile);
console.log(file);
return file;
}
I'm pretty sure it's the way I'm sending the file back but the Graph API has some bugs so I can't entirely be sure. I was convinced I was getting the correct ID of the drive item but I've seen where the site ID syntax can be different with the Graph API so maybe it is the item ID.
The correct syntax for putting an (existing) file into a document library in SharePoint is actually PUT /sites/{site-id}/drive/items/{parent-id}:/{filename}:/content I also found this code below worked for taking the XML document and converting into a blob that could be uploaded
var xmlText = new XMLSerializer().serializeToString(this.state.xmlDoc);
var blob = new Blob([xmlText], { type: "text/xml"});
var file = new File([blob], this.props.location.state.fileName, {type: "text/xml",});
var graphReturn = await putDriveFile(accessToken, this.props.location.state.driveId, this.state.fileId,file);
So in a client-side HTML page, a user selects a file and uploads it to the JavaScript code. JavaScript parses the file and sends it to the server and back to everyone else who is on the site. Then every client makes a blob download link for the file. It's easy when I can send the file to server and back like this.
But now, I want to make that file available for future users of the site without saving it to a location. This is in a chat program, so I've been sending messages from users as strings to a database. I'd like to create a program to send the aforementioned File object to the shortest string possible and then recreate the file (including all metadata) at another client from this string.
What is the standard way to convert a Blob to a string and back again without losing anything? If there's multiple ways, what results in the shortest string?
I found the answer to my question, I had to modify some other answers from SO questions that only sorta applied to my question. Here's what I found:
This is on the uploading-client, in the function called when a file is uploaded:
let inp = document.getElementById("file_input");
let reader = new FileReader();
reader.onload = function(){
send_off_to_other_clients(reader.result);
}
reader.readAsBinaryString(inp.files[0]);
On the other clients:
<script>
function get_blob_from_string (string, type, name) {
let array = new Uint8Array(string.length);
for (let i = 0; i < string.length; i++){
array[i] = string.charCodeAt(i);
}
let end_file = new Blob([array], {type: type, name: name});
let a = document.createElement("a");
a.href = URL.createObjectURL(end_file);
a.download = name;
a.target = "_blank";
a.click();
}
</script>
end_file is the returned-to-blob version, and then I create an anchor tag to download it. Probably isn't "proper" but it works.
For a file upload I collect multiple files :
<input type="file" multiple onchange="uploadFiles(this)" id="fd" />
In the JS function uploadFiles() I retrieve each file and append it to a formData object :
function uploadFiles(e) {
var fileList = document.getElementById(e.id).files;
var fd = new FormData();
for (var i = 0; i < fileList.length; i++) {
var file = fileList[i];
fd.append('files[]', file, file.name);
}
}
Finally I upload it via AJAX to a server.
Each file object contains 4 key/value parameter with the key: name, LastModified, webkitRelativePath and size.
On the server side I need additional information per file (description from a user) which I want to send with the same upload process. My question is :
Is it possible to add params to the file object so I can retrieve them which each file on the server side ?
If not, how to bind additional parameters to each appended file object in the FormData object ?
Any hints are welcome. Info: I use pure JS only.
I have an HTML5/javscript app which uses
<input type="file" accept="image/*;capture=camera" onchange="gotPhoto(this)">
to capture a camera image. Because my app wants to be runnable offline, how do I save the File (https://developer.mozilla.org/en-US/docs/Web/API/File) object in local storage, such that it can be retrieved later for an ajax upload?
I'm grabbing the file object from the using ...
function gotPhoto(element) {
var file = element.files[0];
//I want to save 'file' to local storage here :-(
}
I can Stringify the object and save it, but when I restore it, it is no longer recognised as a File object, and thus can't be used to grab the file content.
I have a feeling it can't be done, but am open to suggestions.
FWIW My workaround is to read the file contents at store time and save the full contents to local storage. This works, but quickly consumes local storage since each file is a 1MB plus photograph.
You cannot serialize file API object.
Not that it helps with the specific problem, but ...
Although I haven't used this, if you look at the article it seems that there are ways (although not supported yet by most browsers) to store the offline image data to some files so as to restore them afterward when the user is online (and not to use localStorage)
Convert it to base64 and then save it.
function gotPhoto(element) {
var file = element.files[0];
var reader = new FileReader()
reader.onload = function(base64) {
localStorage["file"] = base64;
}
reader.readAsDataURL(file);
}
// Saved to localstorage
function getPhoto() {
var base64 = localStorage["file"];
var base64Parts = base64.split(",");
var fileFormat = base64Parts[0].split(";")[1];
var fileContent = base64Parts[1];
var file = new File([fileContent], "file name here", {type: fileFormat});
return file;
}
// Retreived file object
Here is a workaround that I got working with the code below. I'm aware with your edit you talked about localStorage but I wanted to share how I actually implemented that workaround. I like to put the functions on body so that even if the class is added afterwards via AJAX the "change" command will still trigger the event.
See my example here: http://jsfiddle.net/x11joex11/9g8NN/
If you run the JSFiddle example twice you will see it remembers the image.
My approach does use jQuery. This approach also demonstrates the image is actually there to prove it worked.
HTML:
<input class="classhere" type="file" name="logo" id="logo" />
<div class="imagearea"></div>
JS:
$(document).ready(function(){
//You might want to do if check to see if localstorage set for theImage here
var img = new Image();
img.src = localStorage.theImage;
$('.imagearea').html(img);
$("body").on("change",".classhere",function(){
//Equivalent of getElementById
var fileInput = $(this)[0];//returns a HTML DOM object by putting the [0] since it's really an associative array.
var file = fileInput.files[0]; //there is only '1' file since they are not multiple type.
var reader = new FileReader();
reader.onload = function(e) {
// Create a new image.
var img = new Image();
img.src = reader.result;
localStorage.theImage = reader.result; //stores the image to localStorage
$(".imagearea").html(img);
}
reader.readAsDataURL(file);//attempts to read the file in question.
});
});
This approach uses the HTML5 File System API's to read the image and put it into a new javascript img object. The key here is readAsDataURL. If you use chrome inspector you will notice the images are stored in base64 encoding.
The reader is Asynchronous, this is why it uses the callback function onload. So make sure any important code that requires the image is inside the onLoad or else you may get unexpected results.
You could use this lib:
https://github.com/carlo/jquery-base64
then do something similar to this:
//Set file
var baseFile = $.base64.encode(fileObject);
window.localStorage.setItem("file",basefile);
//get file
var outFile = window.localStorage.getItem("file");
an other solution would be using json (I prefer this method)
using: http://code.google.com/p/jquery-json/
//Set file
window.localStorage.setItem("file",$.toJSON(fileobject));
//get file
var outFile = $.evalJSON(window.localStorage.getItem("file"));
I don't think that there is a direct way to Stringify and then deserialize the string object into the object of your interest. But as a work around you can store the image paths in your local storage and load the images by retrieving the URL for the images. Advantages would be, you will never run out of storage space and you can store 1000 times more files there.. Saving an image or any other file as a string in local storage is never a wise decision..
create an object on the global scope
exp: var attmap = new Object();
after you are done with file selection, put your files in attmap variable as below,
attmap[file.name] = attachmentBody;
JSON.stringify(attmap)
Then you can send it to controller via input hidden or etc. and use it after deserializing.
(Map<String, String>)JSON.deserialize(attachments, Map<String,String>.class);
You can create your files with those values in a for loop or etc.
EncodingUtil.base64Decode(CurrentMapValue);
FYI:This solution will also cover multiple file selection
You could do something like this:
// fileObj = new File(); from file input
const buffer = Buffer.from(await new Response(fileObj).arrayBuffer());
const dataUrl = `data:${fileObj.type};base64,${buffer.toString("base64")}`;
localStorage.setItem('dataUrl', dataUrl);
then you can do:
document.getElementById('image').src = localStorage.getItem('dataUrl');