I want to insert a UTF-8 comment in a PNG. Context is in a modern browser : export -canvas- and add some metadata into PNG before user download, later import it and read metadata.
PNG specs for metadata, says about iTXt
I see a good answer here on SO about this, with all steps to achieve a tEXt chunk but without code.
I found a simple nodejs library node-png-metadata to manage PNG metadata.
With this resources, I succeeded some tricks like insert a chunk and read it, but it seem's it's not a valid iTXt chunk (same with tEXt chunk), because tools like pngchunks or pnginfo can't understand it.
See this working fiddle for playing import a PNG it will add metadata and display it ! Test with tEXt or iTXt chunk
Near line 21 some tests around creation of the chunk
var txt = {
sample: '#à$'
};
var newchunk = metadata.createChunk("tEXt", "Comment"+String.fromCharCode(0x00)+"heremycommentl"); // works but not conform
var newchunk = metadata.createChunk("TEXt", "Comment"+String.fromCharCode(0x00)+"heremycommentl"); // result invalid png
var newchunk = metadata.createChunk("iTXt", "Source"+String.fromCharCode(0x00)+"00fr"+String.fromCharCode(0x00)+"Source"+String.fromCharCode(0x00)+""+JSON.stringify(txt));// works but not conform
Beside Resulting PNG is corrupted if chunk type name first char is upper case ? TEXt
If some of you have understanding to share, you're welcome
Chunk names are case-sensitive, tEXt is the name of the chunk, TEXt is not. And since the first letter is uppercase, making the chunk critical, no PNG tools can understand the image since there is now an unknown critical chunk.
The iTXt one is broken because the compression flag and method are stored directly, not as the ASCII representation of the numbers. Changing it to:
metadata.createChunk("iTXt", "Source"+String.fromCharCode(0x00)+String.fromCharCode(0x00)+String.fromCharCode(0x00)+"fr"+String.fromCharCode(0x00)+"Source"+String.fromCharCode(0x00)+""+JSON.stringify(txt));
makes it work.
metadata.createChunk("tEXt", "Comment"+String.fromCharCode(0x00)+"heremycommentl") doesn't cause any issues with pnginfo, perhaps you confused the error there with the iTXt one?
Related
I have a text like this:
Last login: today
cat file
testMachine:root:/root# cat file
File contents
testMachine:root:/root#
And I need to retrieve the information like this:
testMachine:root:/root# cat file
File contents
Stripping away the last line is easy, but the amount of lines I need to remove at start is arbitrary, and I need to remove everything up until the first cue word, which is the machine name, that is known and stored.
I have tried substring() but it strips line by line instead of treating the whole text as one, and removes the host name too, which should remain there. I tried replace() too, but I am not familiar with regex, so the result is a memory exception.
EDIT 1: It seems to be important to note that using a JS for Java engine (In this case I'm using Rhino) means the result isn't the same as you get in web. This was found out after an answer below, which works perfectly on web, doesn't even run on the desktop app.
const text = `
Last login: today
cat file
testMachine:root:/root# cat file
File contents
testMachine:root:/root#`;
const cueWord = "testMachine:root"
const idx = text.indexOf(cueWord);
let restOftheString = text.substring(idx).split("\n");
restOftheString.pop()
console.log(restOftheString.join("\n"))
What is the difference between cy.readFile and cy.fixture in Cypress.io ? In what context should we use cy.readFile and cy.fixture ?
cy.readFile('menu.json')
cy.fixture('users/admin.json') // Get data from {fixturesFolder}/users/admin.json
There are two major differences.
First, the two functions handle filepaths differently.
cy.readFile() starts at your cypress project folder, the folder where your cypress.json is. In other words, cy.readFile("test.txt") will read from (path-to-project)\test.txt.
cy.fixture() starts in the fixtures folder. cy.fixture("test.txt") will read from (path-to-project)\cypress\fixtures\test.txt. Note that this may be different if you have set a fixtures path in your cypress.json.
Absolute file paths do not appear to be supported here.
Second, cy.fixture() tries to guess the file's encoding.
cy.fixture() assumes the encoding for certain file extensions while cy.readFile() does not, except in at least one special case (see below).
For example, cy.readFile('somefile.png') will interpret it as a text document and just blindly read it into a string. This produces garbage output when printed to console. However, cy.fixture('somefile.png') will instead read in the PNG file and convert it to a base64-encoded string.
This difference isn't in the ability of the two functions, but instead appears to be in default behavior; if you specify the encoding, both functions act identically:
cy.readFile('path/to/test.png', 'base64').then(text => {
console.log(text); // Outputs a base64 string to the console
});
cy.fixture('path/to/test.png', 'base64').then(text => {
console.log(text); // Outputs the same base64 string to the console
});
Note:
cy.readFile() doesn't always read in plain text. cy.readFile() gives back a Javascript object when reading JSON files:
cy.readFile('test.json').then(obj => {
// prints an object to console
console.log(obj);
});
I have a compact canvas-to-png download saver function (see code below).
This code works very well and I am satisfied with its output... mostly.
Would a second replace suffice? What would that replace look like?
My only other option is to post-process the file with imagemagick.
Any ideas?
More completely: I want to add metadata from javascript.
I found this link http://dev.exiv2.org/projects/exiv2/wiki/The_Metadata_in_PNG_files
which details the structures, and I may be able to figure it out with sufficient time.
If anyone has experience and can shorten this for me, I would appreciate it.
//------------------------------------------------------------------
function save () // has to be function not var for onclick to work.
//------------------------------------------------------------------
{
var element = document.getElementById("saver");
element.download = savename;
element.href = document.
getElementById(id.figure1a.canvas).
toDataURL("image/png").
replace(/^data:image\/[^;]/,'data:application/octet-stream');
}
The Base-64 representation has little to do with the internal chunks. It's just [any] binary data encoded as string so it can be transferred over string-only protocols (or displayed in a textual context).
It's perhaps a bit broad to create an example, but hopefully showing the main steps will help to achieve what you're looking for:
To add a chunk to a PNG you would first have to convert the data for it into an ArrayBuffer using XHR/fetch in the case of Data-URIs, or FileReader in case you have the PNG as Blob (which I recommend. See toBlob()).
Add a DataView to the ArrayBuffer
Go to position 0x08 in the array which will represent the start of the IHDR chunk, read the length of the chunk (Uint32) (it's very likely it has the same static size for almost any PNG but since it's possible to have changes, and you don't need to remember the chunk size we'll just read it from here). Add length to position (+4 for CRC-32 at the end of the chunk, and +4 if you didn't move the pointer while reading the length), typically this should land you at position 0x21.
You now have the position for the next chunk which we can use to insert our own text chunks
Split that first part into a part-array (a regular array) using a sub-array with the original ArrayBuffer, e.g. new Uint8Array(arraybuffer, 0, position); - you can also use the subarray method.
Produce the new chunk* as typed array and add to part-array
Add the remaining part of the original PNG array without the first part to the part-array, e.g. new Uint8Array(arraybuffer, position, length - position);
Convert the part-array to a Blob using the part-array directly as argument (var newPng = new Blob(partArray, {type: "image/png"});). This will now contain the custom chunk. From there you can use an Object-URL with it to read it back as an image (or make it available for download).
*) Chunk:
For tEXt be aware of it is limited to the Latin-1 charset which means you'll have to whitewash the string you want to use - use iTXt for unicode (UTF-8) content - we'll use tEXt here for simplicity.
The keyword and value is separated by a NUL-byte (0x00) in a tEXt chunk, and the keyword must be exactly typed as defined in the spec.
Build the chunk this way:
get byte-size from string
add 12 bytes (for length, four-cc and crc-32)
format the array this way (you can use a DataView here as well):
Uint32 - length of chunk (data only in number of bytes)
Uint32 - "tEXt" as four-cc
[...] - The data itself (copy byte-wise)
Uint32 - CRC32* which includes the FourCC but not length and itself.
All data in a PNG is big-endian.
To calculate CRC-32 feel free to use this part of my pngtoy solution (the LUT is built this way). Here is one way to format a four-cc:
function makeFourCC(n) { // n = "tEXt" etc., big-endian
var c = n.charCodeAt.bind(n);
return (c(0) & 0x7f) << 24 | (c(1) & 0x7f) << 16 | (c(2) & 0x7f) << 8 | c(3) & 0x7f
}
This question is close but not quite close enough.
My HTML5 application reads a CSV file (although it applies to text as well) and displays some of the data on screen.
The problem I have is that the CSV files can be huge (with a 1GB file size limit). The good news is, I only need to display some of the data from the CSV file at any point.
The idea is something like (psudeo code)
var content;
var reader = OpenReader(myCsvFile)
var line = 0;
while (reader.hasLinesRemaning)
if (line % 10 == 1)
content = currentLine;
Loop to next line
There are enough articles about how to read the CSV file, I'm using
function openCSVFile(csvFileName){
var r = new FileReader();
r.onload = function(e) {
var contents = e.target.result;
var s = "";
};
r.readAsText(csvFileName);
}
but, I can't see how to read line at a time in Javascript OR even if it's possible.
My CSV data looks like
Some detail: date, ,
More detail: time, ,
val1, val2
val11, val12
#val11, val12
val21, val22
I need to strip out the first 2 lines, and also consider what to do with the line starting with a # (hence why I need to read through line at a time)
So, other than loading the lot into memory, do I have any options to read line at a time?
There is no readLine() method to do this as of now. However, some ideas to explore:
Reading from a blob does fire progress events. While it is not required by the specification, the engine might prematurely populate the .result property similar to an XMLHttpRequest.
The Streams API drafts a streaming .read(size) method for file readers. I don't think it is already implemented anywhere, though.
Blobs do have a slice method which returns a new Blob containing a part of the original data. The spec and the synchronous nature of the operation suggest that this is done via references, not copying, and should be quite performant. This would allow you to read the huge file chunk-by-chunk.
Admittedly, none of these methods do automatically stop at line endings. You will need to buffer the chunks manually, break them into lines and shift them out once they are complete. Also, these operations are working on bytes, not on characters, so there might be encoding problems with multi-byte characters that need to be handled.
See also: Reading line-by-line file in JavaScript on client side
I'm trying to decode a base64 string for an image back into binary so it can be downloaded and displayed locally by an OS.
The string I have successfully renders when put as the src of an HTML IMG element with the data URI preface (data: img/png;base64, ) but when using the atob function or a goog closure function it fails.
However decoding succeeds when put in here: http://www.base64decode.org/
Any ideas?
EDIT:
I successfully got it to decode with another library other than the built-in JS function. But, it still won't open locally - on a Mac says it's damaged or in an unknown format and can't get opened.
The code is just something like:
imgEl.src = 'data:img/png;base64,' + contentStr; //this displays successfully
decodedStr = window.atob(contentStr); //this throws the invalid char exception but i just
//used a different script to get it decode successfully but still won't display locally
the base64 string itself is too long to display here (limit is 30,000 characters)
I was just banging my head against the wall on this one for awhile.
There are a couple of possible causes to the problem. 1) Utf-8 problems. There's a good write up + a solution for that here.
In my case, I also had to make sure all the whitespace was out of the string before passing it to atob. e.g.
function decodeFromBase64(input) {
input = input.replace(/\s/g, '');
return atob(input);
}
What was really frustrating was that the base64 parsed correctly using the base64 library in python, but not in JS.
I had to remove the data:audio/wav;base64, in front of the b64, as this was given as part of the b64.
var data = b64Data.substring(b64Data.indexOf(',')+1);
var processed = atob(data);