I am trying to read blns.json from this repo.
I've tried JSON.parse, I've tried turning blns.json to blns.js and requiring the file through module.exports. I've even simply tried console.log() on the array and nothing:
Invalid or unexpected token
What is the best way to read this file in node to be consumed by my tests?
The problem with blns.json file is in the fact that it contains bytes that do not conform with JSON (https://github.com/minimaxir/big-list-of-naughty-strings/issues/20).
You can instead load blns.base64.json which contains base64 representations of naughty strings and then decode them to Buffer (https://nodejs.org/dist/latest-v8.x/docs/api/buffer.html#buffer_class_method_buffer_from_string_encoding).
Keep in mind that if you will try to convert Buffers to Stings, then specific bytes will be lost and some strings will cease to be naughty. But if you are going to use blns to test a web app, then probably it does not matter.
Related
I've Protocol buffer (protoBuf) files which has models (c# class) and WADL files which has functionality for the WebAPIs controllers (controllers and methods in xml format). Now, I want to convert both (protobuf and wadl) it to Json standard format in 1 file. This json can be used by swaggerUI, SoapUI or any other tool which take json. And, when one modifies the Json in Swagger then it must also update corresponding protobuf and wadl.(I know it seems lot of work here, any easy way?) Can any one please let me know, How do I make it?
My app depends on a webservice to form it's URIs, so sometimes it comes up with (what I believe is) a windows-1250 encoded string (/punk%92d) and express fails as follows:
Connect
400 Error: Failed to decode param 'punk%92d'
at Layer.match
So I thought about converting each link to that segment into utf-8 (example: /punk’d, so there would be no reference to the offending enconding), and back again to windows-1250 to work with the external webservice.
I tried this approach using both iconv and iconv-lite but there's always something wrong with the results: /punk d, /punk�d, etc.
Here's a sample using iconv:
var str = 'punk’d';
var buf = new Buffer(str.toString('binary'), 'binary');
console.log(new Iconv('UTF-8', 'Windows-1250').convert(buf).toString('binary'));
…and iconv-lite:
console.log(iconv.decode(new Buffer(str), 'win1250'));
I know using binary is a bad approach, but I was hoping something, anything would just do the job. I obviously tried multiple variations of this code since my knowledge of Buffers is limited, an even simpler things wouldn't work, like:
console.log(new Buffer('punk’d').toString('utf-8'));
So I'm interested in either a way to handle those encoded strings in the URI within express, or an effective way to convert them within node.js.
Sorry if this seems like too simple of a thing to try, but since Node and Express are both JavaScript, have you tried simply using decodeURIComponent('punk’d')? It looks to me that it's simply a standard encoded URI. I think you're getting that weird output from iconv because you're converting from the wrong encoding.
I am using the dropbox-js API as a back-end to an application I am creating.
I need to get the contents of a file and I understand that the method "readFile" that is used to get the contents only really supports text files.
I can get the contents of a text file of type "text/plain" i.e. .txt files, using the following:
client.readFile(d2.path, {arrayBuffer: true}, function(error, contents){
var decoded = decodeUtf8(contents);
console.log(decoded);
});
The API reference for this method is here: http://coffeedoc.info/github/dropbox/dropbox-js/master/classes/Dropbox/Client.html#readFile-instance
The decode function was found here: https://gist.github.com/boushley/5471599
This does not seem to work for any other document type file. If I try and read a .docx / .doc file the result consists of what looks like scrambled characters. Should it be able to work with other document type files? How would I read it differently?
I really need it to support more than .txt files.
Edit:
This is a test document (.docx) that I tried to read:
This is how it is decoded (Contents shows that it is indeed an arrayBuffer, while Decoded is the actual string that is returned after decode:
readFile should work for any content type. Presumably the "scrambled characters" you see are exactly the content of the .docx or .doc file you're reading. (If you looked at the file via type on Windows or cat on Mac/Linux, you would see the same thing.)
So I think the issue you're having is that you want to somehow extract the text from a variety of file formats. Dropbox (and dropbox.js) won't help you with that particular problem... you'll need to find software that understands all those file formats and can convert them to the form you need. For example, textract is a Python library that can do this.
I am trying to use docx.js to generate a Word document but I can't seem to get it to work.
I copied the raw code into the Google Chrome console after amending line 247 to fix a "'textAlign' undefined error"
if (inNode.style && inNode.style.textAlign){..}
Which makes the function convertContent available. The result of which is an Object e.g.
JSON.stringify( convertContent($('<p>Word!</p>)[0]) )
Results in -
"{"string":
"<w:body>
<w:p>
<w:r>
<w:t xml:space=\"preserve\">Word!</w:t>
</w:r>
</w:p>
</w:body>"
,"charSpaceCount":5
,"charCount":5,
"pCount":1}"
I copied
<w:body>
<w:p>
<w:r>
<w:t xml:space="preserve">Word!</w:t>
</w:r>
</w:p>
</w:body>
into Notepad++ and saved it as a file with an extension of 'docx' but when I open it in MS Word but it says 'cannot be opened because there is a problem with the contents'.
Am I missing some attribute or XML tags or something?
You can generate a Docx Document from a template using docxtemplater (library I have created).
It can replace tags by their values (like a template engine), and also replace images in a paid version.
Here is a demo of the templating engine: https://docxtemplater.com/demo/
This code can't work on a JSFiddle because of the ajaxCalls to local files (everything that is in the blankfolder), or you should enter all files in ByteArray format and use the jsFiddle echo API: http://doc.jsfiddle.net/use/echo.html
I know this is an older question and you already have an answer, but I struggled getting this to work for a day, so I thought I'd share my results.
Like you, I had to fix the textAlign bug by changing the line to this:
if (inNode.style && inNode.style.textAlign)
Also, it didn't handle HTML comments. So, I had to add the following line above the check for a "#text" node in the for loop:
if (inNodeChild.nodeName === '#comment') continue;
To create the docx was tricky since there is absolutely no documentation on this thing as of yet. But looking through the code, I see that it is expecting the HTML to be in a File object. For my purposes, I wanted to use the HTML I rendered, not some HTML file the user has to select to upload. So I had to trick it by making my own object with the same property that it was looking for and pass it in. To save it to the client, I use FileSaver.js, which requires a blob. I included this function that converts base64 into a blob. So my code to implement it is this:
var result = docx({ DOM: $('#myDiv')[0] });
var blob = b64toBlob(result.base64, "application/vnd.openxmlformats-officedocument.wordprocessingml.document");
saveAs(blob, "test.docx");
In the end, this would work for simple Word documents, but isn't nearly sophisticated for anything more. I couldn't get any of my styles to render and I didn't even attempt to get images working. I've since abandoned this approach and am now researching DocxgenJS or some server-side solution.
You may find this link useful,
http://evidenceprime.github.io/html-docx-js/
An online demo here:
http://evidenceprime.github.io/html-docx-js/test/sample.html
You are doing the correct thing codewise, but your file is not a valid docx file. If you look through the docx() function in docx.js, you will see that a docx file is actually a zip containing several xml files.
I am using Open Xml SDK for JavaScript.
http://ericwhite.com/blog/open-xml-sdk-for-javascript/
Basically, on web server, I have a empty docx file as new template.
when user in browser click new docx file, I will retrieve the empty docx file as template, convert it to BASE64 and return it as Ajax response.
in client scripts, you convert the BASE64 string to byte array and using openxmlsdk.js to load the byte array as an javascript OpenXmlPackage object.
once you have the package loaded, you can use regular OpenXmlPart to create a real document. (inserting image, creating table/row ).
the last step is stream it out to end user as a document. this part is security related. in my code I send it back to webserver and gets saved temporarily. and prepare a http response to notify end user to download it.
Check the URL above, there are useful samples of doing this in JavaScript.
I'm work on a project recently, which need to pass a binary-stream from npapi plugin to javascript, I've tried following ways:
use NPN_InvokeDefault, i created a string variant which store the binary-stream, and invoke it to javascript, it failed. (i've tried to pass binary-stream read from XXX.txt file, it works!)
i tried to use NPN_NewStream, the example listed in http://www.terraluna.org/dgp/cvsweb/PluginSDK/Documentation/pi3.htm#npnnewstream workes, but the pic is loaded in a new browser tab, i don't know how to recieve it in javascript.
Is there any one have ever met similar problem before? or maybe npapi can't support such kind of data transfering?
looking forward to your suggestiongs, thanks a lot.
Unfortunately, NPAPI was never designed with this purpose in mind. There are a couple of ways you can do it, and none of them are really ideal:
You can create a javascript array and pass the data in small 1-4 byte chunks (this is really very inefficient)
You could create a webserver embedded in the plugin and request the data from there (I have done this and it can work quite well, but keep in mind that if you use this from an SSL website you'll get security warnings when the embedded webserver isn't SSL)
You can base64 encode the binary data and send it as a string.
Those are the ways I have seen it done. The reason you can't send the actual binary data directly as a string is that NPAPI requires string data to be UTF8, but if you base64 encode it then it works fine.
Sorry I can't give you a "happier" solution :-/