I am working on a ReactJS project and in that, I have to fetch (by JS fetch API) a document (.docx/.doc) file from the server and convert it into JSON. After some research, I got to know that document files are unstructured that's why we can't convert them into JSON. So my question is.
-Is it true that document files could not be converted into JSON by JS?
I used the below code but it's not working. The same code is working with excel files.
fetch("url/container/file.docx").then(res => {
return res.arrayBuffer()
}).then(result => {
const _Uint8Array = new Uint8Array(result);
//for xlsx here i am using xlsx.utils.sheet_to_json
console.log(_Uint8Array);
})
I found the same questions on the internet but I didn't get any proper solution to my issue.
Related
I use an html table where it's content can be changed with mouse drag and drop implemented. Technically, you can move the data from any table cell to another. The table size 50 row * 10 column with each cell given a unique identifier. I want to export it to .xlsx format with C# EPPlus library, and give back the exported file to client.
So I need the pass the whole table data upon a button press and post it to either a web api or an mvc controller, create an excel file (like the original html table data) and send it back to download with browser.
So the idea is to create an array which contains each of table cell's value ( of course there should be empty cells in that array), and post that array to controller.
The problem with that approach lies in the download, if I call the api or mvc controller with regular jquery's ajax.post it did not recognize the response as a file.
C# code after ajax post:
[HttpPost]
public IHttpActionResult PostSavedReportExcel([FromBody]List<SavedReports> savedReports, [FromUri] string dateid)
{
//some excel creation code
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StreamContent(new MemoryStream(package.GetAsByteArray()))
};
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
{
FileName = dateid + "_report.xlsx"
};
ResponseMessageResult responseMessageResult = ResponseMessage(response);
return responseMessageResult;
}
Usually, for this kind of result I could use window.location = myurltocontroller to download properly , but that is only for GET requests, POST anything is not possible.
I found some answers which could help me in this topic:
JavaScript post request like a form submit
This points out I should go with creating a form, which passes the values, but I do not know how to do so in case of arrays (the table consists 50*10 = 500 values which I have to pass in the form)
I tried some only frontend solutions to the html-excel export problem, which of course does not require to build files on api side, but free jquery add-ins are deprecated, not customizeable, handle only .xls formats, etc.
I found EPPlus nuget package a highly customizeable tool, that is why I want to try this is at first place.
So the question is: how can I post an array of 500 elements, that the controller will recognize, generate the file, and make it automatically download from browser?
If you can provide some code that would be fantastic, but giving me the right direction is also helpful.
Thank you.
You can use fetch() (docs) to send the request from the JS frontend. When the browser (JS) has received the response, it can then offer its binary content as a download. Something like this:
fetch("http://your-api/convert-to-excel", // Send the POST request to the Backend
{
method:"POST",
body: JSON.stringify(
[[1,2],[3,4]] // Here you can put your matrix
)
})
.then(response => response.blob())
.then(blob => {
// Put the response BLOB into a virtual download from JS
if (navigator.appVersion.toString().indexOf('.NET') > 0) {
window.navigator.msSaveBlob(blob, "my-excel-export.xlsx");
} else {
var a = window.document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = "my-excel-export.xlsx";
a.click();
}});
So the JS part of the browser actually first downloads the file behind the scenes, and only when it's done, it's triggering the "download" from the browsers memory into a file on the HD.
This is a quite common scenario with REST APIs that require bearer token authentication.
I need to get gtfs data and show raw data on screen. I tried to use gtfs-stream module, but not sure, how to get data from it.
var gtfs_source = 'https://www.bkk.hu/gtfs/budapest_gtfs.zip'
request.get(gtfs_source).pipe(gtfs.enhanced()).on('data', (entity) => {
console.log(entity)
})
gtfs-utils has a rich set of functions, but it is unable to get data from a compressed file.
For a site that I'm making, I wanted to have an additional feature which uses this plugin on GitHub called cratedigger that uses WebGL and Three.js. It is a 3D virtual crate that contains vinyl records, and simulates "crate digging" of albums. The plugin gets the data for the vinyl titles, artists, and covers through a string in JSON format that is parsed through JSON and stored in a const variable. The original code (index.js):
const data = JSON.parse('[{"title":"So What","artist":"Miles
Davis","cover":"http://cdn-
images.deezer.com/images/cover/63bf5fe5f15f69bfeb097139fc34f3d7/400x400-
000000-80-00.jpg","year":"2001","id":"SOBYBNV14607703ACA","hasSleeve":false},
{"title":"Stolen Moments","artist":"Oliver Nelson","cover":"http://cdn-
images.deezer.com/images/cover/99235a5fbe557590ccd62a2a152e4dbe/400x400-
000000-80-00.jpg","year":"1961","id":"SOCNMPH12B0B8064AA","hasSleeve":false},
{"title":"Theme for Maxine","artist":"Woody Shaw","cover":"http://cdn-
images.deezer.com/images/cover/bb937f1e1d57f7542a64c74b13c47900/400x400-
000000-80-00.jpg","year":"1998","id":"SOMLSGW131343841A7","hasSleeve":false}]
');
You can view the source code here. The above code is in line 3.
For my site though, I want the titles, artists, covers, etc. to come from my MySQL database. So what I did is when I click a button found in my main site, it will run the sql query of my database and then convert it into a .json file:
//getdatabase.php
<?php
include 'includes/session.php';
include 'includes/header.php';
$conn = $pdo->open();
$data = $conn->query("
SELECT name as title,
artist_name as artist,
concat('http://website.com/images/', photo) as cover,
year(date_created) as year,
id,
hasSleeve
from products
where category_id = '5';
")->fetchAll(PDO::FETCH_ASSOC);
foreach($data as &$row){
$row['hasSleeve'] = filter_var($row['hasSleeve'],
FILTER_VALIDATE_BOOLEAN);
}
$json_string = json_encode($data);
$file = 'cratedigger.js/src/htdocs/data.json';
file_put_contents($file, $json_string);
$pdo->close();
header('location: cratedigger.js/lib/index.html');
?>
Afterwards, it will be redirected to the index of the cratedigger plugin. To retrieve the data in the .json file, I used fetch API in the index.js file under the src folder of this plugin. So I replaced the original code in the plugin with this:
//replaced the long line of const data=JSON.parse('[{"title":...]'); with
this:
let data = [];
fetch('data.json').then(function(resp) {
return resp.json();
})
.then(function(data) {
console.log(data); //outputs data from my database
return data;
});
console.log(data); //output is 0 or none
I use node.js to build and test this plugin, and when I test it with the code I used, the crate appears but with no records in it (a blank wooden crate). In the console, the log inside the fetch api does output the data from the json file, but the log outside the fetch outputs zero or none. I figured that it was that fetch is asynchronous, so the second console.log didn't output any data because it didn't wait for fetch to finish.
And that's my problem. I want to replace the original code that uses a string in JSON format, and replace it with data from my database. Some of the solutions that I came up with that didn't work are:
Use await and async - This is still asynchronous so it couldn't store my data in the variable.
XMLHttpRequest is mostly asynchronous too and its synchronous part is already deprecated.
Place fetch inside a function - My problem with this code is that the variable "data" used in the original code is used/called in other parts of the source files like these examples:
function fillInfoPanel(record) {
if (record.data.title) {
titleContainer.innerHTML = record.data.title;
}
}
//or this//
cratedigger.loadRecords(data, true, () => {
bindEvents();
});
So calling a function like myData(); to get the data from fetch wouldn't work as I need it to be inside the data variable. I'm not sure how I'm supposed to replace data variable used in other parts of the code with a function.
Any suggestions that might work? I don't necessarily need to use fetch to retrieve data. I'm more into HTML, PHP & CSS, and not that familiar with Javascript, so I'm stuck here. Node.JS was something that I've learned a week ago so the codes initially confused me. I've read articles and watched YouTube tutorials about javascript, json, etc. but they confused me more than help with my problem.
I am using angularjs and exporting data in excel from the table that has been uploaded.
I am using the following code:
function (e)
{
window.open('data:application/vnd.ms-excel,' + encodeURIComponent($('div[id$=exportable]').html()));
e.preventDefault();
It is allowing me to download the file but extension is missing. Can you pls guide me how to change file name and extension both for the same?
Thanks
Excel to JSON
This is basically what you are looking for, some way to turn excel to JSON data with JavaScript. Here is a link to an article about how it can be done using XLSXReader in a web app along with AngularJS.
Sample Code
$(function() {
$("#xlsxFile").change(function(event) {
var file = this.files[0],
sheets;
XLSXReader(file, true, function(xlsxData) {
sheets = xlsxData.sheets;
// Do somehting with sheets. It's a
// Javascript object with sheet names
// as keys and data as value in form of 2D array
});
});
});
Working Example
Here is a working example of the code. You can upload a csv file and have it turned into JSON.
I hope this helps.
I'm developing a REST API with Play 2 and I'm wondering how to implement file upload functionality.
I've read the official Play documentation but it just provides a multipart/form-data example, while my backend does not provide any form... it just consists of a REST API to be invoked by a JavaScript client or whatever else.
That said, what's the correct way to implement such an API? Should I implement a PartHandler and then still use the mutipartFormData parser? How should I pass the file content to the API? Is there any exhaustive example on this topic?
Any help would be really appreciated.
You should look into BodyParsers: http://www.playframework.com/documentation/2.2.x/ScalaBodyParsers
What you are trying to do is not especially complicated, especially if you are only handling smaller files that would fit in memory. After all uploading a file is just about sending the file as a body of a POST or something like that. It is not any different from receiving some XML or JSON in a request.
Hope this helps
import org.apache.http.entity.mime._
import java.io.File
import org.apache.http.entity.mime.content._
import java.io.ByteArrayOutputStream
import play.api.libs.ws.WS
val contents ="contents string"
val file = File.createTempFile("sample", ".txt")
val bw = new java.io.BufferedWriter(new java.io.FileWriter(file)
bw.write(new_contents);
bw.close();
builder.addPart("file", new FileBody(file, org.apache.http.entity.ContentType.create("text/plain"), "sample"))
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
val entity = builder.build
val outputstream = new ByteArrayOutputStream
entity.writeTo(outputstream)
val header = (entity.getContentType.getName -> entity.getContentType.getValue)
val response = WS.url("/post/file").withHeaders(header).post(outputstream.toByteArray())
To pass your contents, depending on your client side, you can encode the contents to Base64 at client side to pass the contents as Json (You can use Json body parser). Then on the server side you can decode the contents using a Base64 decoder (e.g. Apache Commons) to get the byte array. It will be as simple as
Base64.decodeBase64(YourEncodedFileContent)
When you have the byte array you can simply write it on disk or save it into database etc. We are using this approach in production and it works fine however we only handle small file uploads.
OK, thank you all for your suggestions... here below is how I solved my issue:
object Files extends Controller {
def upload = SecuredAction[Files.type]("upload").async(parse.multipartFormData(partHandler)) { implicit request =>
future { request.body.files.head.ref match {
case Some((data, fileName, contentType)) => Ok(success(Json.obj("fileName" -> fileName)))
case _ => BadRequest
}}.recover { case e =>
InternalServerError(error(errorProcessingRequest(e.getMessage)))
}
}
...
private def partHandler = {
parse.Multipart.handleFilePart {
case parse.Multipart.FileInfo(partName, fileName, contentType) =>
Iteratee.fold[Array[Byte], ByteArrayOutputStream](
new ByteArrayOutputStream
) { (outputStream, data) =>
outputStream.write(data)
outputStream
}.map { outputStream =>
outputStream.close()
Some(outputStream.toByteArray, fileName, contentType.get)
}
}
}
}
I hope it helps.
while my backend does not provide any form... it just consists of a REST API to be invoked by a JavaScript client
Then your backend is not a REST API. You should follow the HATEOAS principle, so you should respond with links and forms along with data to every GET request. You don't have to send back HTML, you can describe these things with hypermedia json or xml media types, for example with JSON-LD, HAL+JSON, ATOM+XML, etc... So you have to describe your upload form in your preferred hypermedia, and let the REST client to turn that description into a real HTML file upload form (if the client is HTML). After that you can send a multipart/form-data as usual (REST is media type agnostic, so you can send data in any media type you want, not just in a JSON format). Check the AJAX file upload techniques for further detail...