I am making a web app with VueJS that makes axios calls to a NodeJS API that uses express. I'm currently trying to send files to my NodeJS so it can save them. However, even though my browser displays all the properties of my array of files, my NodeJS reads it as empty. I've read all the questions previously asked on this topic but I haven't made any progress whatsoever. Note that I can POST any other data just fine, all my SQL Insert requests do well, mind you they don't involve sending arrays.
Example of promising solution that did not work : https://stackoverflow.com/a/66541165/12498040
Here is my client side JS, console.log() prints an array of N files object in the browser console :
console.log(data)
await axios.post('/api/fichiers', {fichiers:data});
Here is my NodeJS, with console.log(element) it prints N number of "{}" in the browser console and console.log(element.name) prints N number of "undefined" in the server terminal :
app.post('/api/fichiers', (req, res) => {
req.body.fichiers.forEach((element) => {
console.log(element.name);
});
});
Thank you for any help you could provide :)
You are getting an array of empty objects because File objects are not serializable to JSON.
If you want to send files, you need to replace the object containing a plain array you are passing to axios (which it will try to serialize to JSON) with a FormData object.
Then you need body parsing middleware in your Express application which can handle the multipart/form-data request body that a FormData object will generate. The body-parser module can't do that, but its documentation links to a number of options.
Thank you !! It's now working as intended. Here is how I made it if anyone is interested :
Client-side JS :
var formData = new FormData();
for (let i = 0; i < data.length; i++) {
formData.append("files", data[i]);
}
await axios.post('/api/fichiers', formData);
NodeJS :
const multer = require('multer')
var storage = multer.diskStorage(
{
destination: function (req, file, cb) {
cb(null, './tempDir/')
},
filename: function (req, file, cb) {
cb(null, Date.now() + '_' + file.originalname);
}
}
);
var upload = multer({ storage: storage });
app.post('/api/fichiers', upload.array('files'));
I then installed npm fs.extra to move the files to the desired directory
Related
I want upload the content of an excel file into the server in order to get its data and do some stuff...
I came up with the following code, however it seems like it is not working properly as the following error displays in the console Error: Can't set headers after they are sent.
The file is getting uploaded into the folder and the json message is being displayed... However I do not know if I am going to face any issue in the future...
Actually I just need the excel data no need for the excel being uploaded... Maybe you could give me a workaround, guys...
const router = express.Router();
const storage = multer.diskStorage({
destination(req, file, cb) {
cb(null, 'uploads/');
},
filename(req, file, cb) {
cb(
null,
`${file.fieldname}-${Date.now()}${path
.extname(file.originalname)
.toLowerCase()}`
);
},
});
const excelFilter = (req, file, cb) => {
if (
file.mimetype.includes('excel') ||
file.mimetype.includes('spreadsheetml')
) {
cb(null, true);
} else {
cb('Please upload only excel file.', false);
}
};
const upload = multer({
storage,
fileFilter: excelFilter,
});
router.post('/', upload.single('file'), (req, res) => {
var workbook = XLSX.readFile(req.file.path);
var sheet_name_list = workbook.SheetNames;
var xlData = XLSX.utils.sheet_to_json(workbook.Sheets[sheet_name_list[0]]);
res.json(xlData).sendFile(`/${req.file.path}`, { root: path.resolve() });
});
May I have a res.json and res.sendFile together in the same api endpoint in express?
No, you cannot. Each of those methods, sends a complete http response (including calling res.end() which terminates the http request) and you can only send one http response to each incoming request. The particular error you're getting has to do with the res.sendFile() trying to configure the response that it's getting ready to send and finding that the http response object has already been used for sending a response and can't be used again.
Ordinarily, if you wanted to sent two different pieces of data, you would just combine them into a single Javascript object and just call res.json() on the object that contains both pieces of data.
But, sending a binary file is not something you can easily put in a JSON package. You could construct a multipart response where one part was the JSON and one part was the file. You could JSON encode binary data (though that's inefficient). I presume there are probably some modules that would help you do that, but for most clients, that isn't what they are really expecting or equipped to handle.
The only way to a proper solution is for us to understand what client/server workflow you're trying to implement here and why you're trying to send back the same file that was just uploaded. There would normally not be a reason to do that since the client already has that data (they just uploaded it).
My task is to download json-file from website (pubchem) using only the query string (h2o for example) and JS. I know it's possible to do with parsing, but this is too much code because of number of pages i need to parse for getting destination. Is there any other options to solve the problem?
Using google didnt give me any of idea ):
You will still need to do some parsing if you really want to automate this, since only using a query parameter will get you to the main page that lists the 'articles' and you need to go in to find the URL that will give you the JSON format. But! I think you can "reverse engineer" it since the URLS for the article and its JSON format are very similar.
I checked out the website and tried to download one of the files that they have for https://pubchem.ncbi.nlm.nih.gov/compound/3076959 and it turns out to get the JSON representation this was the URL https://pubchem.ncbi.nlm.nih.gov/rest/pug_view/data/compound/748328/JSON/
As you can see they are very similar and you might be able to figure out how different topics such as compound for example construct the JSON output endpoint.
To download the JSON files using NodeJS is to use the node-fetch module or axios library to send your http requests to the JSON endpoint and from there you can save the response to a file on your machine.
Here is an example of how you can do this with axios and the NodeJS fs module in order to save the file to your machine.
const fs = require("fs");
const fetch = require("node-fetch");
async function downloadASJson(url, fileName) {
const response = await fetch(url);
const jsonContent = await response.buffer();
fs.writeFile(`${fileName}.json`, jsonContent, "utf8", function (err) {
if (err) {
console.log("An error occured while writing JSON Object to File.");
return console.log(err);
}
console.log("JSON file has been saved.");
});
}
try {
downloadASJson(
"https://pubchem.ncbi.nlm.nih.gov/rest/pug_view/data/compound/748328/JSON/",
"2-Methyl-3-(5'-bromobenzofuroyl-2')-4-dimethylaminomethyl-5-hydroxybenzofuran HCl H20"
);
} catch (err) {
console.log(error);
}
You save the following code in a file called app.js for example, and you can use node app.js to run it. Don't forget to install the dependencies.
I am trying to submit a image file using a POST Request, to server, where in the body of the front end fetch request i added body as FormData like this
let formdata = new FormData(form)
async function finalFetch(formdata){
let postReq = await fetch('/api/fileupload', {method : 'POST', body : formdata})
let result = await postReq.json()
return result.url;
}
Now on submit of this form in the backend i am handling the data like this
req.on('data', (chunk)=>{
console.log(chunk);
})
req.on('end', ()=>{
// pseudo code, this will get replaced by something else
res.write(JSON.stringify({
msg : 'File Upload done',
url : '/232'
}));
res.end();
})
and the above implementation gives me a Buffer, i have no idea how to write file in the server form this Buffer
I know there are lots of modules out there to handle the forms like multer, express, formidable, but i don't want to use any of them i am trying to understand how these packages work. I am just using core node js.
There are tons of third party packages why not use them, one of the famous one is https://www.npmjs.com/package/multer, by default node dosnt give you an option to write files.
After the buffer is fully received you should use core node module 'fs' to write buf to file.
const fs = require('fs');
fs.writeFile('file.name', buffer);
see docs fs.writeFile
Sorry, I tend to be a bad writer when I have not fully woken up, let me revise.
I am using expressjs with passportjs (local strategy) to manage my server and using connect-busboy to manage file uploading. I do not think passport will play a role in this.
Here is the server code for managing file uploads:
app.post('/upload', isLoggedIn, (req, res) => {
if(req.busboy){
req.pipe(req.busboy);
req.busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
if(mimetype.match(/^image\//)){
var root = path.join(__dirname, "../public/images/");
if(fs.existsSync(path.join(root, filename))){
var name = getUnique(path.join(root, filename));
} else {
var name = filename;
}
var ws = fs.createWriteStream(path.join(root, name), { flags: "a" });
file.pipe(ws);
}
});
}
});
As for my client page, it is used to change a JSON object which will get re-uploaded to the server as a configuration tool. When I upload a new image asynchronously I need to get the filename to update this JSON object while working on it. For uploading from the clients end I am using dropzonejs, which did not require any configuration on my part to work.
So, in summary I upload a number of images via dropzone asynchronously, busboy and fs on my server save the file, and I would like to get the filename returned to my javascript to modify the existing JSON object.
Edit solution:
Thanks to Elliot Blackburn for pointing me in the right direction.
By calling:
ws.on('close', () => {
res.send({filename: name});
});
after file.pipe(ws); to send the response back to the client. On the client side modify dropzone to handle the response like so:
dropzone.on('success', (file, res) => {
console.log(res);
});
Just send it in the normal http response. It'll depend what library you're using but most will allow you to trigger a normal req, res, next express call. From that you can access the file object, and return anything you want.
Something like:
req.send({filename: name}); // name is the filename var set earlier in the code.
Once you've finished editing the file and such, you can get the name and put it into that returned object and your client will receive that as object as the response which you can act upon.
I'm working on an express.js app which should load an API definition file (most likely the swagger file in JSON format). Currently I've created a middleware, which should parse the JSON file via fs.readFile() and JSON.parse() in order to check user's permissions on accessing some resource. So basically each time the request is performed, my middleware gets the same JSON file and parses it, which is obviously a piece of extra work. Is it possible to load this JSON file, parse and store it to some internal object in a sort of global configuration and reload it in case it was modified so as not to perform the same operation on each request?
Of course, you could create a function like this (pseudo code):
var jsonData=null;
function getConfiguration() {
if (!jsonData) {
jsonData= readFileSync(...);
}
return jsonData;
}
module.exports.getConfiguration=getConfiguration;
Or, as #AleksandrM commented, you can just "import" it using require:
This is what worked for me, very much aligned with the answer here
let config_file = path.join(__dirname, 'config.json')
let configJSON = {}
function getConfiguration() {
jsonfile.readFile(config_file, function (err, obj) {
if (err) console.error(err)
console.dir(obj)
configJSON = obj
})}
Then on app startup
app.listen(port, () => {
getConfiguration()
console.log(`App listening at http://localhost:${port}`)
})