I have a GraphQL server, hosted on express. I want to return images to the client by sending back nodejs buffer objects. How can i config graphql server, to return bytes, instead of json? I don't wish to do this through base64, as the image are large in size.
You have to return JSON, but there's still a way. We're using GraphQL to return images stored in Blob fields in a legacy sql database. We're using sequelize, graphql-sequelize, and graphql-js.
We've defined the Blob fields to be String types in our graphql schema and so they come through in the json reply just fine.
Then we convert to a buffer before delivering, like
const imgBuffer = new Buffer.from(imgObj.data, 'ascii');
The only problem is we're now having trouble saving image data back to the database via the graphql interface. Our mutation function gives us a syntax error when it finds certain bad unicode characters in the strings, like \U0000 and whatnot (so I found your question looking for a solution to that).
There's a way, but it's complicated, and very manual, and I'm only going to give you an overview of what I've done in ApolloServer, but I think it should be enough.
First, you need to use the "Accept" header in your request to send a binary mime type, and send a matching "Content-Type" in your response. This is nessisary to be efficient, but not nessisary to work, as you'll see (with EJSON).
To serialize and deserialize respecting the headers you may need to write an express middleware, and you'll need to handle base64 encoding with a {$data: "..."} encapsulating object (as EJSON does) or just (strangely) returning null, if someone makes a request for binary data using "application/json" for their "accept" header. You'll also want to choose what binary formats that you'll support. I only use 1: "application/x-msgpack", but I hear that "application/cbor" is becoming more popular. You can use a library for EJSON, MessagePack, and CBOR to do your serialization, so this isn't as hard as it sounds.
I would then strongly recommend using the #defer on any images. See this post for more information on #defer: https://www.apollographql.com/blog/introducing-defer-in-apollo-server-f6797c4e9d6e/
I've done it. It wasn't easy, and it would be better if ApolloServer worked this way "out of the box".
It's better to send a hashed & temporary link to download it
The URL can be hashed to be non-accessible by other users.
Backend can expire the link or remove the binary file on the static server.
There might be an answer to your question by using the node module found here.
Related
I want to know how to get all the data passed on the WebSocket
I alredy tryied using FireFox to see but all the data are strange unicode text and symbols (game link is https://sploop.io) is there an way to maybe decrypt it?
I also tryied using
var data= new WebSocket("usa1.sploop.io/ws")
data.onmessage = (sa)=>{console.log(sa)}
And after some actions in the game the code logged an object that didnt have any of the data...
You're already getting all the data the WebSocket is receiving. The problem is that the data is "encoded" binary data using the game's protocol. The scripts in Sploop.io know how to decode this data (and encode new data to be sent back), but since you don't "speak" that protocol, it looks like gibberish to you.
Problem aside, you can have fun and all, but trying to cheat or so isn't nice towards other players.
I know that blob is a data type for binary data as integer is a datatype for int. As they say, It's used to store files directly in database (we move our audio file into blob, and save that blob in database).
Question 1) why to store blob for audio if I can just put the audio in storage for example path /var/www/audio.mp3 and in database I store path_name /var/www/audio.mp3?
Question 2) which is better ? how netflix stores movies? just blobs or what?
Question 3) Curious if there're any cons or prons if you could just give me ideas so that I know when to use them .
Putting the blob in the database, rather than a file, allows you to grow to multiple servers with load balancing. If you put the data in files, you would have to replicate the files between the server. Most databases have built-in replication features, this isn't as easy for regular files.
Better to use external storage/cdn for serving such kind of large content.
How Netflix and our works? They upload content on external bucket i. e. S3 and write file name in db for identification. According to user file access frequency that file cache on CDN/edge location. User will get awesome experience while content server from their nearest edge location
With blob you can store all kinds of stuff.
Do you communicate with an API via SOAP or JSON and want to store it in the database? Use a blob. Want to log what a user filled into a form when it threw an exception? Store the entire post as a blob. You can save everything as is. It's handy for logging if you have different data formats. I know an API which expects some data via SOAP and some as JSON. To log the communication I use blob because the response may be in XML, JSON, a number (http code 203 for empty but accepted) or an exception as array.
I can send a file to the server using axios and the FormData api, like that :
persist(avatar){
let data = new FormData();
data.append('avatar', avatar);
axios.post(`/api/users/${this.user.name}/avatar`, data)
.then(() => flash('Avatar uploaded!'));
}
The avatar param, passed to persist(), is a file object from a form input of type "file".
I can then grab the file on the server side.
Is it possible to do it without using FormData ? That is, to simulate the FormData work ? Basically, I'm trying to understand the extra work done by the FormData api. Maybe it's not possible using axios, and I should do that with plain XMLHttpRequest.
Of course, simply sending the file object won't work :
axios.post(`/api/users/${this.user.name}/avatar`, {avatar: avatar})
On the server side, the avatar object will be empty. I can send metadata like avatar.name, but not the whole object.
Yes, it is possible to do the encoding manually on the client. Writing a form data encoder yourself may be useful if you really want to learn every little detail about how forms work. However, for most applications, I would not recommend it. The FormData API should be used in production.
You will need to refer to RFC 7578 for how to implement the encoder.
This specification defines the multipart/form-data media type, which can be used by a wide variety of applications and transported by a wide variety of protocols as a way of returning a set of values as the result of a user filling out a form.
More resources:
Meaning of multipart/form-data
Source code for the form-data library on npm
I've been looking through the entire Socket.IO docs, but, even though they promise it is there, I can't find a simple, minimal example, of how one would send binary data between server/client.
How is it done?
It is in fact in the documentation. The current documentation for Socket.io says under Socket.emit:
[...] Emits an event to the socket identified by the string name. Any other
parameters can be included. All datastructures are supported, including Buffer [...]
So, if you can send a buffer, you can send binary data. All you have to do is to pack your data into a Buffer object.
You may want to read Socket.io Binary Support and Sending and Receiving Binary
Starting from socket.io 1.0 it is possible to send binary data. http://socket.io/blog/introducing-socket-io-1-0/
How ever the way of sending and receiving binary data is not clear in the official documentation. The only documentation is:
var socket = new WebSocket('ws://localhost');
socket.binaryType = 'arraybuffer';
socket.send(new ArrayBuffer);
I suggest you to take a look at this answer, where you can find basic example with code implementation for server and client (javascript and java too):
How to send binary data with socket.io?
The good part is that it also works on Android! (if you wish)
Cheers
All I have is <input type="file" name="myFileUpload" />.
After the user chooses a file (most likely an image), how do I obtain the actual contents of the file as a string? (If possible, please tell me anything about base 64 and url encoding/decoding.)
I was asked to obtain such a string and set it as a value of a JSON object, then such a JSON object would be posted to the server "as is", that is, application/json; charset=utf-8.
I'm not sure if the above is a common practice since I'm accustomed to just posting such data as multipart/form-data which I was told not to use.
The receiver of this gigantic JSON object is an ASP.net Web API Controller. I suppose there would be a problem with deserialization if such an object is potentially multi-megabytes large.
So again, how to obtain the image bytes as a string and what problems may I encounter if I try to post such a large JSON object especially when it's received the server-side.