We Keep Coding

How can I upload multiple files into Magnolia CMS assets app from a custom form

Short version
I have a webapp using Magnolia, I need to upload a comment with posibility of multiple files, I want to use AJAX, before saving the files as assets I want to be able to check the user's permission, I figured I need a custom Java-based REST endpoint, I made it work, but I have issues saving "jcr:data" into an asset.
Long version
I have a webapp, I have registered users (using PUR), I have different roles for users (for simplicity let's say User and Editor) and I have a Post and a Comment content types. Every User can create a Post and add files, every Post holds the creator's UUID, array of Comment UUIDs and array of file UUIDs (from Assets app), every Post has a comment section and every Comment can have files attached (zero or multiple) to it. Every Editor can post a comment to every Post, but Users can only post comments to their own Posts.
My form for comments looks something like this:
<form action="" method="post" id="comment-form" enctype="multipart/form-data">
<input type="file" name="file" id="file" multiple />
<textarea name="text"></textarea>
<button type="button" onclick="sendComment();">Send</button>
</form>
I tried using Javascript model to process the data and I was able to save the asset correctly, however only one. I couldn't access the other files in the model.
I tried solving it (and improving user experience) by using AJAX and a REST endpoint. I opted not to use the Nodes endpoint API, because I didn't know how to solve the permission issue. I know I can restrict access to REST for each role, but not based on ownership of the Post. So I created my own Java-based endpoint (copied from documentation).
In the sendComment() function in Javascript I create an object with properties like name, extension, mimeType, ..., and data. I read in the documentation that you should send the data using the Base64 format, so I used FileReader() to accomplish that:
var fileObject = {
// properties like name, extension, mimeType, ...
}
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
// this part is easy
};
xhttp.open("PUT", "http://localhost:8080/myApp/.rest/assets/v1/saveAsset", true);
xhttp.setRequestHeader("Content-type", "application/json");
var reader = new FileReader();
reader.onload = function() {
fileObject.data = reader.result;
// I also tried without the 'data:image/png;base64,' part by reader.result.split(",")[1];
xhttp.send(JSON.stringify(fileObject));
};
reader.readAsDataURL(file); //where file is the value of the input element.files[i]
In Java I created a POJO class that has the same properties as the javascript object. Including a String data.
The code for the endpoint looks like this:
public Response saveAsset(Asset file) {
// Asset is my custom POJO class
Session damSession;
Node asset;
Node resource;
try {
damSession = MgnlContext.getJCRSession("dam");
asset = damSession.getRootNode().addNode(file.getName(), "mgnl:asset");
asset.setProperty("name", file.getName());
asset.setProperty("type", file.getExtension());
resource = asset.addNode("jcr:content", "mgnl:resource");
InputStream dataStream = new ByteArrayInputStream(file.getData().getBytes());
ValueFactory vf = damSession.getValueFactory();
Binary dataBinary = vf.createBinary(dataStream);
resource.setProperty("jcr:data", dataBinary);
resource.setProperty("fileName", file.getName());
resource.setProperty("extension", file.getExtension());
resource.setProperty("size", file.getSize());
resource.setProperty("jcr:mimeType", file.getMimeType());
damSession.save();
return Response.ok(LinkUtil.createLink(asset)).build();
} catch (RepositoryException e) {
return Response.ok(e.getMessage()).build(); //I know it's not ok, but that's not important at the moment
}
}
The asset gets created, the properties get saved apart from the jcr:data. If I upload an image and then download it either by the link I get as a response or directly from the Assets app, it cannot be opened, I get the format is not supported message. The size is 0, image doesn't show in the Assets app, seems like the data is simply not there, or it's in the wrong format.
How can I send the file or the file data to the Java endpoint? And how should I receive it? Does anybody know what am I missing? I honestly don't know what else to do with it.
Thank you

The input stream had to be decoded from Base64.
InputStream dataStream = new ByteArrayInputStream(Base64.decodeBase64(file.getData().getBytes()));
...and it only took me less than 3 months.
Noticed it after going through the source code for REST module that for unmarshalling Binary Value it had to be encoded to Base64, so I tried decoding it and it started to work.

Javascript, Typescript, Angular 5 - Open and read file

I am working with Angular 5, I have an application in which I need to read an AMP HTML file as text. This file is contained in a component and should only be accessed from this component.
I would like to be able to open the file in read-only by giving its name.
I'm actually searching for something like this:
let file = open('amp.html');
Is it possible? If not how can I do to achieve this?

If you're writing browserside JS
You can't just simply read a file. The JS is running on your browser, and you need to think about where you're getting that file from.
If the file is on a server, you need to fetch that file from the server first by making a request for it.
If you're reading a file on the user's computer, you're gonna be using the File API on the browser to allow the user to select that file.
If you're writing backend JS
Assuming you're using NodeJS, you can conduct file operations like you would with other programming languages. Check out the fs module

If i understand you correct, you can read it as text like this:
function readFile(file){
var raw = new XMLHttpRequest(); // create a request
raw.open("GET", file, false); // open file
raw.onreadystatechange = function (){ // file is ready to read
if(raw.readyState === 4){
if(raw.status === 200 || raw.status == 0){
var allText = raw.responseText;
alert(allText); // can be also console.logged, of course.
}
}
}
raw.send(null); // return control
}
usage:
readFile('link.html')
I solved this issue thankfully to this question.

Reading/Writing to a file in javascript

I want to read and write to a file in a specific way.
An example file could be:
name1:100
name2:400
name3:7865786
...etc etc
What would be the best way to read this data in and store in, and eventually write it out?
I don't know which type of data structure to use? I'm still fairly new to javascript.
I want to be able to determine if any key,values are matching.
For example, if I were to add to the file, I could see that name1 is already in the file, and I just edit the value instead of adding a duplicate.

You can use localStorage as a temporary storage between reads and writes.
Though, you cannot actually read and write to a user's filesystem at will using client side JavaScript. You can however request the user to select a file to read the same way you can request the user to save the data you push, as a file.
localStorage allow you to store the data as key-value pairs and it's easy to check if an item exists or not. Optionally simply use a literal object which basically can do the same but only exists in memory. localStorage can be saved between sessions and navigation between pages.
// set some data
localStorage.setItem("key", "value");
// get some data
var data = localStorage.getItem("key");
// check if key exists, set if not (though, you can simply override the key as well)
if (!localStorage.getItem("key")) localStorage.setItem("key", "value");
The method getItem will always return null if the key doesn't exist.
But note that localStorage can only store strings. For binary data and/or large sizes, look into Indexed DB instead.
To read a file you have to request the user to select one (or several):
HTML:
<label>Select a file: <input type=file id=selFile></label>
JavaScript
document.getElementById("selFile").onchange = function() {
var fileReader = new FileReader();
fileReader.onload = function() {
var txt = this.result;
// now we have the selected file as text.
};
fileReader.readAsText(this.files[0]);
};
To save a file you can use File objects this way:
var file = new File([txt], "myFilename.txt", {type: "application/octet-stream"});
var blobUrl = (URL || webkitURL).createObjectURL(file);
window.location = blobUrl;
The reason for using octet-stream is to "force" the browser to show a save as dialog instead of it trying to show the file in the tab, which would happen if we used text/plain as type.
So, how do we get the data between these stages. Assuming you're using key/value approach and text only you can use JSON objects.
var file = JSON.stringify(localStorage);
Then send to user as File blob shown above.
To read you will have to either manually parse the file format if the data exists in a particular format, or if the data is the same as you save out you can read in the file as shown above, then convert it from string to an object:
var data = JSON.parse(txt); // continue in the function block above
Object.assign(localStorage, data); // merge data from object with localStorage
Note that you may have to delete items from the storage first. There is also the chance other data have been stored there so these are cases that needs to be considered, but this is the basis of one approach.
Example
// due to security reasons, localStorage can't be used in stacksnippet,
// so we'll use an object instead
var test = {"myKey": "Hello there!"}; // localStorage.setItem("myKey", "Hello there!");
document.getElementById("selFile").onchange = function() {
var fileReader = new FileReader();
fileReader.onload = function() {
var o = JSON.parse(this.result);
//Object.assign(localStorage, o); // use this with localStorage
alert("done, myKey=" + o["myKey"]); // o[] -> localStorage.getItem("myKey")
};
fileReader.readAsText(this.files[0]);
};
document.querySelector("button").onclick = function() {
var json = JSON.stringify(test); // test -> localStorage
var file = new File([json], "myFilename.txt", {type: "application/octet-stream"});
var blobUrl = (URL || webkitURL).createObjectURL(file);
window.location = blobUrl;
}
Save first: <button>Save file</button> (<code>"myKey" = "Hello there!"</code>)<br><br>
Then read the saved file back in:<br>
<label>Select a file: <input type=file id=selFile></label>

Are you using Nodejs? Or browser javascript?
In either case the structure you should use is js' standard object. Then you can turn it into JSON like this:
var dataJSON = JSON.stringify(yourDataObj)
With Nodejs, you'll want to require the fs module and use one of the writeFile or appendFile functions -- here's sample code:
const fs = require('fs');
fs.writeFileSync('my/file/path', dataJSON);
With browser js, this stackoverflow may help you: Javascript: Create and save file

I know you want to write to a file, but but consider a database instead so that you don't have to reinvent the wheel. INSERT ... ON DUPLICATE KEY UPDATE seems like the logical choice for what you're looking to do.

For security reasons it's not possible to use JavaScript to write to a regular text or similar file on a client's system.
However Asynchronous JavaScript and XML (AJAX) can be used to send an XMLHttpRequest to a file on the server, written in a server-side language like PHP or ASP.
The server-side file can then write to other files, or a database on the server.
Cookies are useful if you just need to save relatively small amounts of data locally on a client's system.
For more information have a look at
Read/write to file using jQuery

Saving file with JavaScript File API results wrong encoding

I have a problem (or may be two) with saving files using HTML5 File API.
A files comes from the server as a byte array and I need to save it. I tried several ways described on SO:
creating blob and opening it in a new tab
creating a hidden anchor tag with "data:" in href attribute
using FileSaver.js
All approaches allow to save the file but with breaking it by changing the encoding to UTF-8, while the file (in current test case) is in ANSI. And it seems that I have to problems: at the server side and at the client side.
Server side:
Server side is ASP.NET Web API 2 app, which controller sends the file using HttpResponseMessage with StreamContent. The ContentType is correct and corresponds with actual file type.
But as can be seen on the screenshot below server's answer (data.length) is less then actual file size calculated at upload (file.size). Also here could be seen that HTML5 File object has yet another size (f.size).
If I add CharSet with value "ANSI" to server's response message's ContentType property, file data will be the same as it was uploaded, but on saving result file still has wrong size and become broken:
Client side:
I tried to set charset using the JS File options, but it didn't help. As could be found here and here Eli Grey, the author of FileUplaod.js says that
The encoding/charset in the type is just metadata for the browser, not an encoding directive.
which means, if I understood it right, that it is impossible to change the encoding of the file.
Issue result: at the end I can successfully download broken files which are unable to open.
So I have two questions:
How can I save file "as is" using File API. At present time I cannot use simple way with direct link and 'download' attribute because of serverside check for access_token in request header. May be this is the "bottle neck" of the problem?
How can I avoid setting CharSet at server side and also send byte array "as is"? While this problem could be hacked in some way I guess it's more critical. For example, while "ANSI" charset solves the problem with the current file, WinMerge shows that it's encoding is Cyrillic 'Windows-1251' and also can any other.
P.S. the issue is related to all file types (extensions) except *.txt.
Update
Server side code:
public HttpResponseMessage DownloadAttachment(Guid fileId)
{
var stream = GetFileStream(fileId);
var message = new HttpResponseMessage(HttpStatusCode.OK);
message.Content = new StreamContent(stream);
message.Content.Headers.ContentLength = file.Size;
message.Content.Headers.ContentType = new MediaTypeHeaderValue(file.ContentType)
{
// without this charset files sent with bigger size
// than they are as shown on image 1
CharSet = "ANSI"
};
message.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = file.FileName + file.Extension,
Size = file.Size
};
return message;
}
Client side code (TypeScript):
/*
* Handler for click event on download <a> tag
*/
private downloadFile(file: Models.File) {
var self = this;
this.$service.downloadAttachment(this.entityId, file.fileId).then(
// on success
function (data, status, headers, config) {
var fileName = file.fileName + file.extension;
var clientFile = new File([data], fileName);
// here's the issue ---^
saveAs(clientFile, fileName);
},
// on fail
function (error) {
self.alertError(error);
});
}
My code is almost the same as in answers on related questions on SO: instead of setting direct link in 'a' tag, I handle click on it and download file content via XHR (in my case using Angularjs $http service). Getting the file content I create a Blob object (in my case I use File class that derives from Blob) and then try to save it using FileSaver.js. I also tried approach with encoded URL to Blob in href attribute, but it only opens a new tab with a file broken the same way. I found that the problem is in Blob class - calling it's constructor with 'normal' file data I get an instance with 'wrong' size as could be seen on first two screenshots. So, as I understand, my problem not in the way I try to save my file, but in the way I create it - File API

Save File From Local Data in Javascript

Heres the scenario:
User comes to my website and opens a webpage with some javascript functionality.
User edits the data through javascript
User clicks on a save button to save the data, thing is, it seems like they shouldn't need to download this data because its already in javascript on the local machine.
Is it possible to save data from javascript (executing from a foreign webpage) without downloading a file from the server?
Any help would be much appreciated!

For saving data on the client-side, without any server interaction, the best I've seen is Downloadify, is a small JavaScript + Flash library allows you to generate and save files on the fly, directly in the browser...
Check this demo.

I came across this scenario when I wanted to initiate a download without using a server. I wrote this jQuery plugin that wraps up the content of a textarea/div in a Blob, then initiates a download of the Blob. Allows you to specify both file name and type..
jQuery.fn.downld = function (ops) {
this.each(function () {
var _ops = ops || {},
file_name = _ops.name || "downld_file",
file_type = _ops.type || "txt",
file_content = $(this).val() || $(this).html();
var _file = new Blob([file_content],{type:'application/octet-stream'});
window.URL = window.URL || window.webkitURL;
var a = document.createElement('a');
a.href = window.URL.createObjectURL(_file);
a.download = file_name+"."+file_type;
document.body.appendChild(a);
a.click(); $('a').last().remove();
});
}
Default Use : $("#element").downld();
Options : $("#element").downld({ name:"some_file_name", type:"html" });
Codepen example http://codepen.io/anon/pen/cAqzE

JavaScript is run in a sandboxed environment, meaning it only has access to specific browser resources. Specifically, it doesn't have access to the filesystem, or dynamic resources from other domains (web pages, javascript etc). Well, there are other things (I/O, devices), but you get the point.
You will need to post the data to the server which can invoke a file download, or use another technology such as flash, java applets, or silverlight. (i'm not sure about the support for this in the last 2, and I also wouldn't recommend using them, depends what it's for...)

The solution to download local/client-side contents via javascript is not straight forward. I have implemented one solution using smartclient-html-jsp.
Here is the solution:
I am in the project build on SmartClient. We need to download/export data of a grid
(table like structure).
We were using RESTish web services to serve the data from Server side. So I could not hit the url two times; one for grid and second time for export/transform the data to download.
What I did is made two JSPs namely: blank.jsp and export.jsp.
blank.jsp is literally blank, now I need to export the grid data
that I already have on client side.
Now when ever user asks to export the grid data, I do below:
a. Open a new window with url blank.jsp
b. using document.write I create a form in it with one field name text in it and set data to export inside it.
c. Now POST that form to export.jsp of same heirarchy.
d. Contents of export.jsp I am pasting below are self explanatory.
// code start
<%# page import="java.util.*,java.io.*,java.util.Enumeration"%>
<%
response.setContentType ("text/csv");
//set the header and also the Name by which user will be prompted to save
response.setHeader ("Content-Disposition", "attachment;filename=\"data.csv\"");
String contents = request.getParameter ("text");
if (!(contents!= null && contents!=""))
contents = "No data";
else
contents = contents.replaceAll ("NEW_LINE", "\n");
//Open an input stream to the file and post the file contents thru the
//servlet output stream to the client m/c
InputStream in = new ByteArrayInputStream(contents.getBytes ());
ServletOutputStream outs = response.getOutputStream();
int bit = 256;
int i = 0;
try {
while ((bit) >= 0) {
bit = in.read();
outs.write(bit);
}
//System.out.println("" +bit);
} catch (IOException ioe) {
ioe.printStackTrace(System.out);
}
outs.flush();
outs.close();
in.close();
%>
<HTML>
<HEAD>
</HEAD>
<BODY>
<script type="text/javascript">
try {window.close ();} catch (e) {alert (e);}
</script>
</BODY>
</HTML>
// code end
This code is tested and deployed/working in production environment, also this is cross-browser functionality.
Thanks
Shailendra

You can save a small amount of data in cookies.