We Keep Coding

POST image to django rest API always returns 'No file was submitted'

I am trying to get this working now for days -.-
Using a simple NodeJS express server, I want to upload an image to a Django instance through Post request, but I just can't figure out, how to prepare the request and embed the file.
Later I would like to post the image, created from a canvas on the client side,
but for testing I was trying to just upload an existing image from the nodeJS server.
app.post('/images', function(req, res) {
const filename = "Download.png"; // existing local file on server
// using formData to create a multipart/form-data content-type
let formData = new FormData();
let buffer = fs.readFileSync(filename);
formData.append("data", buffer); // appending the file a buffer, alternatively could read as utf-8 string and append as text
formData.append('name', 'var name here'); // someone told me, I need to specify a name
const config = {
headers: { 'content-type': 'multipart/form-data' }
}
axios.post("http://django:8000/images/", formData, config)
.then(response => {
console.log("success!"); // never happens :(
})
.catch(error => {
console.log(error.response.data); // no file was submitted
});
});
What am I doing wrong or did I just miss something?
EDIT
I just found a nice snippet with a slighlty other approach on the npm form-data page, on the very bottom (npmjs.com/package/form-data):
const filename = "Download.png"; // existing local file on server
let formData = new FormData();
let stream = fs.createReadStream(filename);
formData.append('data', stream)
let formHeaders = formData.getHeaders()
axios.post('http://django:8000/images/', formData, {
headers: {
...formHeaders,
},
})
.then(response => {
console.log("success!"); // never happens :(
})
.catch(error => {
console.log(error.response.data); // no file was submitted
});
sadly, this doesn't change anything :( I still receive only Bad Request: No file was submitted
I don't really have much Django code just a basic setup using the rest_framework with an image model:
class Image(models.Model):
data = models.ImageField(upload_to='images/')
def __str__(self):
return "Image Resource"
which are also registered in the admin.py,
a serializer:
from .models import Image
class ImageSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Image
fields = ('id', 'data')
using automatic URL routing.
I wrote a simple test script and put the same image on the django server, to verify that image uploads works, and it does:
import requests
url = "http://127.0.0.1:8000/images/"
file = {'data': open('Download.png', 'rb')}
response = requests.post(url, files=file)
print(response.status_code) # 201

I had a similar problem: I used the same Django endpoint to upload a file using axios 1) from the client side and 2) from the server side. From the client side it worked without any problem, but from the server side, the request body was always empty.
My solution was to use the following code:
const fileBuffer = await readFile(file.filepath)
const formData = new FormData()
formData.append('file', fileBuffer, file.originalFilename)
const response = await fetch(
urlJoin(BACKEND_URL),
{
method: 'POST',
body: formData,
headers: {
...formData.getHeaders(),
},
}
)
A few relevant references that I found useful:
This blog post, even though it seems the author manages to send form data from the server side using axios, I did not manage to reproduce it on my case.
This issue report in the axio repository, where one comment suggests to use fetch.

In your node.js express server instead of adding the image to the form data, try directly sending the stream in the API post.
const filename = "Download.png"; // existing local file on server
//let formData = new FormData();
let stream = fs.createReadStream(filename);
//formData.append('data', stream)
let formHeaders = formData.getHeaders()
axios.post('http://django:8000/images/', stream, {
headers: {
...formHeaders,
},
})
.then(response => {
console.log("success!"); // never happens :(
})
.catch(error => {
console.log(error.response.data); // no file was submitted
});

I still didn't manage to get this working with axios so I tried another package for sending files as post requests, namely unirest, which worked out of the box for me.
Also it is smaller, requires less code and does everything I needed:
const filename = "Download.png"; // existing local file on server
unirest
.post(url)
.attach('data', filename) // reads directly from local file
//.attach('data', fs.createReadStream(filename)) // creates a read stream
//.attach('data', fs.readFileSync(filename)) // 400 - The submitted data was not a file. Check the encoding type on the form. -> maybe check encoding?
.then(function (response) {
console.log(response.body) // 201
})
.catch((error) => console.log(error.response.data));
If I have some spare time in the future I may look into what was wrong with my axios implementation or someone does know a solution pls let me know :D

How to write correct app.get()? (node.js / express / boby-parser)

I create server (node.js / express / boby-parser)
and I need to get array of objects 'users'.
its part of code from server.js file:
let users = [{
name: 'user1',
}];
app.get('/users/', (req, res) => {
const filePath = path.join(pth.dir, 'build', 'index.html');
res.json(users);
res.sendFile(filePath);
});
Its my code from frontend:
const handleResponse = (response) => {
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
};
const getAll = (baseUrl) => {
const requestOptions = {
method: 'GET'
};
return fetch(baseUrl, requestOptions).then(handleResponse);
};
Something wrong with my code at server. (I just dodnt know how to use express server).
when I use getAll function I got JSON text replace my page. Can anyone help? How should I write app.get() in server.js. Or do I need write in server part one app.get() to get page or another app.get() to get JSON data?

Why are you trying to send a file in the response?:
res.sendFile(filePath);
For starters, the response content can either be JSON or a file (or any of a variety of other things of course), but not both. With data like JSON or XML it's possible to combine multiple objects into one larger object for a single response, but this won't work if the content types are entirely different as it is with a file.
Looking at your client-side code, you're not even doing anything with that file. You only read the JSON data:
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
So the simplest approach here would just be to not try to send back the file:
app.get('/users/', (req, res) => {
res.json(users);
});
Edit: Based on comments below, you seem to be struggling with the different requests the client makes to the server. The browser loading the page is one request with one response. If that page includes JavaScript that needs to fetch data, that would be a separate AJAX request with its own response containing just that data.
It’s possible to use JSON (or any data) server-side to populate a page template and return a whole page with the data. For that you’d need to use (or build) some kind of templating engine in the server-side code to populate the page before returning it.

The res.json() represents the HTTP response that an Express app sends when it gets an HTTP request. On the other hand, res.sendFile() transfers the file at the given path.
In both cases, the flow is essentially transferred to client who might have made the request.
So no, you cannot use res.sendFile and res.json together.
var options = {
headers: {
'name': 'user1',
}
};
res.sendFile(path.join(__dirname, 'build', 'index.html'), options);
Thats really the closest you can do to achieve the desired task.

Converting circular structure to JSON - 3rd party API

I'm building out a project that is making a call to a blockchain API. Unfortunately, the data I'm getting back is circular so while it works in Postman my server errors when trying to convert it to JSON. I tried using JSON.stringify but nothing changed.
Here's the controller function:
blockchainController.search = (req, res) => {
axios({
method: 'GET',
url: `https://chain.api.btc.com/v3/address/${req.body.address}/tx`
})
.then(data => {
res.json({
message: 'Transactions loaded',
data: data
})
})
.catch(err => {
console.log(err);
res.send(err);
})
};
Any ideas for a workaround or a fix? I'd like to be able to send this data to my front-end but it ain't happening.

A solution could be to use a library designed to prune circular references.
I happen to have built such library: https://github.com/Canop/JSON.prune
You can simply call it with
let json = JSON.prune(yourCircularObject);
This adds some "-pruned-" marks whenever a reference is ignored.
If you prefer a "silent" removal, you can do
let json = JSON.prune(yourCircularObject, {prunedString: undefined });

How to upload a file in to object storage using node js call

Iam trying to create a post call which basically takes a file(eg img,pdf file) and then it need to upload in to object storage on bluemix.I was able to authenticate and get the token and create the authurl.I just need to pass file which we upload along with the url.But Iam out of ideas how I can get the file uploaded from postman to be passed to that url with in the post call..Below is my code
app.post('/uploadfile',function(req,res){
getAuthToken().then(function(token){
if(!token){
console.log("error");
}
else{
var fileName = req.body.file;
console.log("data",file);
console.log(SOFTLAYER_ID_V3_AUTH_URL,"url");
var apiUrl = SOFTLAYER_ID_V3_AUTH_URL + config.projectId + '/' + containerName + fileName ;
url : apiurl,
method :'PUT',
headers :{
'X-Auth-Token': token
},function(error, response, body) {
if(!error && response.statusCode == 201) {
res.send(response.headers);
} else {
console.log(error, body);
res.send(body);
}
}
}
})
});
Can someone help here.

Since you're using Express, you should use something like:
https://www.npmjs.com/package/express-fileupload
https://github.com/mscdex/connect-busboy
https://github.com/expressjs/multer
https://github.com/andrewrk/connect-multiparty
https://github.com/mscdex/reformed
Without a body parser that handles file uploads you will not be able to get the uploaded file in the Express request handler.
Then, you need to pass the uploaded file to the request that you're making.
For that you should use this module:
https://www.npmjs.com/package/bluemix-object-storage
There is no need to reinvent the wheel when there are tested and eay to use solutions available. Especially when you're dealing with sensitive information like API keys and secrets I would not advice you to implement your own solution from scratch, unless you really know what you're doing. And if you really know what you're doing, then you don't need to seek advice for things like that.

Here is the official Object Storage SDK for Node.js:
https://github.com/ibm-bluemix-mobile-services/bluemix-objectstorage-serversdk-nodejs
Connect to Object Storage:
var credentials = {
projectId: 'project-id',
userId: 'user-id',
password: 'password',
region: ObjectStorage.Region.DALLAS
};
var objStorage = new ObjectStorage(credentials);
Create a container:
objstorage.createContainer('container-name')
.then(function(container) {
// container - the ObjectStorageContainer that was created
})
.catch(function(err) {
// AuthTokenError if there was a problem refreshing authentication token
// ServerError if any unexpected status codes were returned from the request
});
}
Create a new object or update an existing one:
container.createObject('object-name', data)
.then(function(object) {
// object - the ObjectStorageObject that was created
})
.catch(function(err) {
// TimeoutError if the request timed out
// AuthTokenError if there was a problem refreshing authentication token
// ServerError if any unexpected status codes were returned from the request
});

Can't POST data using JavaScript fetch API

I'm trying to POST data to my server using the fetch API and every time I do I end up with an empty body.
As far as I can tell from the fetch documentation, to post data you need to pass an object with a method key set to "POST" and a body key with a string or FormData object (or some others).
Here's a simple example that is just not working as I would expect:
var formData = new FormData();
formData.append("test", "woohoo");
formData.append("foo", "bar")
// prints out contents of formData to show that it has contents!
formData.forEach(function(key, value) {
console.log(`${key}: ${value}`);
});
fetch("/fetch", {
method: "POST",
body: formData
}).then(function(response) {
return response.json();
}).then(function(json) {
// Logs empty object
console.log(json);
}).catch(function(err) {
console.log(err);
});
I also have an express server on the back of this that logs the request body and echoes it in the response.
app.post("/fetch", function(req, res) {
// Logs empty object :(
console.log(req.body);
res.send(JSON.stringify(req.body));
});
I add the formData as the body of the fetch request, but nothing ever seems to go through.
Please find my test files in this gist here and help!

This is because your node server isn't understanding mulitpart forms correctly (which is what FormData) will send by default.
You can use node-muliparty on your server you can read the form easily without touching the frontend code.
Something like:
var multiparty = require('multiparty');
var app = express();
app.post("/fetch", function(req, res) {
var form = new multiparty.Form();
form.parse(req, function(err, fields, files) {
console.log(fields);
res.send(JSON.stringify(fields));
});
});
in your index.js will show you it working correctly. There might be a way to make this work with body-parser but I don't know how myself.