tensorflowjs - Is there an equivalent method for tokenizer in javascript? - javascript

I'm building an NLP classifier in python and would like to build a hosting HTML page for a demo. I want to test on a sample text to see the prediction and this is implemented in python through tokenizing the text and then padding it before predicting. Like this:
tf.tokenizer.texts_to_sequences(text)
token_list = tf.tokenizer.texts_to_sequences([text])[0]
token_list_padded = pad_sequences([token_list], maxlen=max_length, padding=padding_type)
The problem is that I'm new to javascript, so is there tokenization and padding methods in javascript like in python?

There is not yet a tf.tokenizer in js as there is in python.
A simple js.tokenizer has been described here. A more robust approach would be to use the tokenizer that comes with universal sentence encoder

There is no native mechanism for tokenization in Javascript.
You can use a Javascript library such as natural or wink-tokenizer or wink-nlp. The last library automatically extracts a number of token's features that may be useful in training.

Related

Open and save xml file from jmeter command line

I am trying to manipulate jmeter test plan in a web based tool. The problem is, it converts many characters to implicitly. For example " converts to ", 
 converts to newline.
I observed that, if i open that modified file from jmeter ui and save it without doing anything, all the characters are converted back to original. For example " converts to ".
So is it possible to do this automatically using jquery/javascript. I am using angularjs with node.js for my application. I would prefer to do this open-save-close operation in background. Please suggest , how can i achieve this. is there any jmeter-plugin available which i can run from javascript/jquery ?
Many thanks in advance
You need to escape the following characters in XML, otherwise it will result into invalid markup.
"
'
<
>
&
Given you use NodeJS you can use xml-escape function to do the trick for you.
JMeter provides __escapeXML() function out of the box just in case you're looking for Java-based implementation, see Apache JMeter Functions - An Introduction article to get familiarized with JMeter Functions concept.
What jmeter is doing with XML is correct and is done by the library XStream.
JMeter manipulate Java Objects and serializes them using XStream.
I am not sure what you are trying to do, but I don't think it is correct in terms of maintainability, indeed JMeter doesn't provide any contract on XML (XSD or DTD). Test plans should be manipulated through Java.
As far as I know, you cannot manipulate it through javascript, but you can potentially use this DSL to manipulate/generate test plans:
https://github.com/flood-io/ruby-jmeter

iPython/Jupyter Notebook: How to Embed Interactive Graph Using Desmos API?

I've recently switched from taking notes for my Calculus II course with the pen-and-paper system to using Jupyter (formerly known as iPython) notebooks. What a difference!
Anyway, as someone who learns best through visual presentations, I would really like to embed some interactive Desmos graphs in my notebooks (for anyone who is not familiar with Desmos, it is an incredibly powerful, yet easy-to-use, web-based graphing calculator).
Unfortunately, the iPython/Jupyter notebook security model prevents the execution of JavaScript embedded in Markdown cells. The HTML Sanitization library (Google Caja, I believe) strips any HTML tags and JavaScript code you put into Markdown cells.
According to a note in the security model docs, support for some sort of mechanism for allowing HTML/CSS for notebook theming is planned. But the note makes no mention of JavaScript support.
I realize cross-site scripting is a serious problem and one that is difficult to defend against, but is there really no means to loosen the security constraints for notebook authors? Perhaps in the future it might be possible to add a configuration option to the notebook metadata (which can be edited from within a notebook session) to specify a list of allowable tags.
In the meantime, does anyone know of a work-around, hack, or other method for embedding output from a third-party API using JavaScript in Markdown cells within a notebook?
If one were to print the appropriate HTML and JavaScript code using Python within a Python cell, would that avoid these restrictions? Maybe I should write a Python wrapper for the Desmos API...
You can always use an interact from IPython widgets
from IPython.html.widgets import *
import numpy as np
import matplotlib.pyplot as plt
import math
def linear(w,x,b):
return w*x + b
def logistic(z):
return 1/(1+math.e**(-z))
def plt_logistic(a, b):
x = np.linspace(-20,20, 100)
h = linear(a,x,b)
y = logistic(h)
plt.ylim(-5,5)
plt.xlim(-5,5)
plt.plot(x,h)
plt.plot(x,y)
plt.grid()
plt.show()
interact(plt_logistic, a = (-10,10,0.1), b = (-10,10,0.1))
Here is how to embed Desmos in Jupyter using jp_proxy widgets:
Please see https://github.com/AaronWatters/jp_proxy_widget
-- this code is based on the quick start example: https://www.desmos.com/api/v1.2/docs/index.html
I think there are several ways to make it
use iframe
use raw html display, which may need you write
some wrapper first to make it reuseable
use some 3-party lib: mpld3, plot.js, here is a list
use some other type 3-party lib: IPython-Dashboard

Google Spreadsheet: Encrypt cell content with Google Apps Script

I have a Google Spreadsheet and would like to encrypt the content of a few cells (I do not care which encryption method is being used as long as there is an equivalent decryption method for iOS).
Unfortunately there are no built-in encryption functions in Google Apps Script.
For this reason I would like to use a open source Javascript library like Crypto-JS and sjcl.
How can I use one of these libraries with Google Apps Script?
In the Google Apps Script documentation, I have not found any clue on how to use external JavaScript libraries with my Google Apps Script.
Well I'll say this, because this is the method that I used with Date JS. You can do the following:
Download the source .js file(s).
Open the .js file(s) in a text editor
Copy/paste all code into a new Script Project
here you can "recreate" the original .js files (copy/paste source individually) with the same names
Include the project key of that Script Project as a library of the project in which you want to use those functions.
Even if the projects are open-source you will want to make sure you comply with the licenses of those projects if you are going to use them.
This is basically a small "hack" around not being able to upload .js files into GAS Projects. Assuming that the JS is standard, this method will work with Google's system.
The other option is to simple find a light-weight one- or two-function crypto package, or a single crypto algorithm like AES-128 (taht you are given permission to use, of course). It really depends on how much encryption you want, if you need to reverse the cipher text to get the plain values, etc.
If this is a for some kind of password system, I would recommend using a simple hash. For example:
function stringHash (someString) {
var hash = 0;
if (this.length == 0) return hash;
for (i = 0; i < this.length; i++) {
char = this.charCodeAt(i);
hash = ((hash << 5) - hash) + char;
hash = hash & hash;
}
return Math.abs(hash); // Personally I don't like negative values, so I abs'd it
}
in which you would ask for a user's password, and if the password hash matched the hash stored in the spreadsheet or wherever, then you would validate. You can use this to simulate logging into a UiApp GUI, for example: store usernames/password hashes in a database and validate a user before loading the "real" app.
However, as Serge mentioned, Spreadsheets will contain revision history of the original value before it was hashed, as well as the value after it was hashed. If you want to avoid this, use ScriptDB.
PS - in addition to this work-around, I'll say that it's not currently possible to "import" a non-GAS code library into your Script Project, unless you manually copy the source file-by-file into your Script Project. There may be a feature request on the Issue Tracker already, if not you can create one and I'll star it.
EDIT: As per request, I've included an open source AES encryption "package" (contains base64 as well, which is nice) in the answer, to act as a reference for others who want to encrypt in GAS. Make sure you follow the author's request, which is to retain his original copyright and link back to the source.
Other than the AES I linked and the simple hash (equivalent to Java's String.hashCode()), whose resource can be found here, there is Crypto-JS as you mentioned in your question and, if you took the time to fully copy/paste all the code (assuming that agrees with the terms of the license - I haven't read it), you could use that by the steps I described in the top half of my answer.
MD5 in Javascript is also an algorithm that you could use. If you use the code in md5.js which is located at the top of the page, you'll have what you need. Again, make sure you're following licensing rules if you use it.
Personally I would probably just use the hash and the base-64 patterns, as most of what you would use this encryption for is probably not incredibly important. AES might take a bit longer to compute - you can probably benchmark it yourself to see if it will cause major problems with triggers running for an extended period of time, but I doubt it would be a problem anyway.
Note: base-64 is 2-way, so is AES. MD5 is a type of hash, and the simple hash function I provided is also (of course) a hash. Hash functions are one-way. So if you need two-way functionality (encrypt/decrypt), then use base-64 or AES. Base-64 is essentially the kid version of AES. And the simple hash function is the kid version of MD5. Keep this in mind :)
Edit again: I'm not familiar with iOS development or its internals, but it seems to me that iOS can at least do some cryptographic operations. You may want to read more into those methods though, because I'm not really sure how you're putting GAS and iOS together; I can't give you any more help in that area unfortunately.
The functions above don't work for me. Here is something what you can copy and paste into google sheets (spreadsheet) script editor
function enc(str) {
var encoded = "";
for (i=0; i<str.length;i++) {
var a = str.charCodeAt(i);
var b = a ^ 123; // bitwise XOR with any number, e.g. 123
encoded = encoded+String.fromCharCode(b);
}
return encoded;
}
This is what you get when you use it =ENC in your spreadsheet
Based on this post here
External libraries can be used by using JavaScript's built in eval() function
(ex. eval(UrlFetchApp.fetch('path/to/library'))).
Of course, the library must have no dependencies for this to work.

Get access to Stackoverflow's auto-suggest tagging system?

Is there anyway to get access to stackoverflow's awesome tagging system? I would like to borrow Stack's awesome auto-suggest and tag mini-explanation boxes for my own site. Obviously, I can use the jQuery UI auto-suggest for tags but I would really like to also include the cool little tag descriptions as well. If not, can someone tell me where all these explanation/descriptions came from so that I can implement a similar system?
tageditornew.js
Line 308:
$.get("/filter/tags", {q: a,newstyle: !0}, "json").done(function(c) {
C["t_" + a] = c;
StackExchange.helpers.removeSpinner();
b(c)
})
This might help you out!
It turns out that,
the API url is this:
https://stackoverflow.com/filter/tags?q=STRING&newstyle=BOOLEAN
q - Query text.
newstyle - Require new style or not. Result in new style will be returned in JSON with additional information such as synonyms and excerpt.
DEMO: http://jsfiddle.net/DerekL/bXXb7/ (with Cross Domain Requests jQuery plguin)
For example:
https://stackoverflow.com/filter/tags?q=htm
would give you:
"html|99829\nhtml5|16359\nxhtml|4143\nhtml-parsing|1461\nhtml-lists|1328\nhtml5-video|949"
where 99829 is the amount of questions. It took me 15 minutes looking at the source code to find out this api. -_-"
Putting in javascript in new style gives you this: here
[{"Name":"javascript","Synonyms":"classic-javascript|javascript-execution","Count":223223,"Excerpt":"JavaScript is a dynamic language commonly used for scripting in web browsers. It is NOT the same as Java. Use this tag for questions regarding ECMAScript and its dialects/implementations (excluding ActionScript and JScript). If a framework or library, such as jQuery, is used, include that tag as well. Questions that don't include a framework/library tag, such as jQuery, implies that the question requires a pure JavaScript answer."},{"Name":"javascript-events","Synonyms":"javascript-event","Count":5707,"Excerpt":"Creating and handling JavaScript events inline in HTML or through a script."},{"Name":"facebook-javascript-sdk","Synonyms":"","Count":992,"Excerpt":"Facebook's JavaScript SDK provides a rich set of client-side functionality for accessing Facebook's server-side API calls. These include all of the features of the REST API, Graph API, and Dialogs."},{"Name":"javascript-library","Synonyms":"","Count":675,"Excerpt":"A JavaScript library is a library of pre-written JavaScript which allows for easier development of JavaScript-based applications, especially for AJAX and other web-centric technologies."},{"Name":"javascript-framework","Synonyms":"","Count":563,"Excerpt":"A JavaScript framework is a library of pre-written JavaScript which allows for easier development of JavaScript-based applications, especially for AJAX and other web-centric technologies."},{"Name":"unobtrusive-javascript","Synonyms":"","Count":340,"Excerpt":"Unobtrusive JavaScript is a general approach to the use of JavaScript in web pages."}]
What you can get from there:
All tags start with javascript
Synonyms
Tag counts
Nice tag descriptions
If you're looking for high-level logic, in a nutshell it's just a custom auto-complete that's blazing-fast.
Whenever you type a tag (i.e. a new word or one separated by a space from previous tags), an AJAX request would be made to the server with a JSON object which is then interpreted by the client-side script and presented in the usable layout.
Comparing the autocomplete JSON objects for letter "h" and word "html" should give you enough insight into how this particular implementation works (if prompted, these can be opened with any text editor).
On a somewhat unrelated note: the autocomplete responses have to be fast. Depending on the complexity of the data autocomplete is run against, you may find how IMDb magic search works intriguing.
Update:
Seeing your comment about accessing the content of the tag library, this may in fact be more of a meta question. I struggle to think of a scenario where using an API if any or just the tag library from an external resource would be beneficial to SO - however content here is provided under Creative Commons so you may be able to use it with proper attribution. This does not constitute legal advice :)

Can JQuery or JavaScript be used to manipulate XML/DOM without a browser?

I am starting to study some web technologies to integrate content, markup, layout, styling and behaviors of stuff for personal use (NOT web developing for now) and am amazed with the power of JQuery selectors and functions.
I have heard that there are some ways to use javascript "outside" a browser, to do some DOM selection, manipulation, etc. I wonder if JQuery could be used that way too.
So, what I would like to do is:
Using some programming/scripting language (I use Python), access a XML file and parse its DOM;
Programmatically manipulate and modify the DOM with javascript/jquery selectors and functions;
Save the results to (possibly another) XML file.
If you like jQuery syntax, check out pyQuery:
from pyquery import PyQuery
_ = PyQuery('<body><p></p></body>')
_("p").text("hello").css({'color': 'red'})
print _.html()
>>> <p style="color: red">hello</p>
yeah, you just need a Javascript run time.
Check out node.js
What you're looking for is called a "headless" browser.
This SO post may help:
Real headless browser
Basically you need a javascript interpreter (ex: V8) + wrapper for your language of choice (ex: pyv8). Then you can do this (from pyv8 page):
import PyV8
ctxt = PyV8.JSContext()
ctxt.enter()
ctxt.eval("1+2") # 1+2 is a javascript code

Categories