The problem I am trying to solve is this
Given a string containing code (entered by the user via some kind of web based form), such as:
"a = b + 5; return Math.abs(a);"
Extract a list of tokens that are considered identifiers:
a, b
In general are there readily available existing solutions? If not, what is a good starting point for writing an simple algorithm for this purpose?
I tried http://ace.c9.io/#nav=about, but was not able to find a method that readily returned the token type of an entire document. According to the API reference getTokens(Number row) should return the information I want. However, this method ended up returning the entire line as type text
Thank you in advance
Related
I'm trying to use the "Change User's Password" extended operation, as defined in this RFC which states that it takes a sequence of three optional parameters. However, it seems that ldapjs's client.exop() function only allows me to provide it with a string or buffer.
This is my attempt:
const dn = `uid=${username},ou=People,dc=${orginization},dc=com`
client.exop('1.3.6.1.4.1.4203.1.11.1', [dn, null, newPassword], (err, value, res) => {
// ...
})
And this is the resulting error:
TypeError: options.requestValue must be a buffer or a string
How am I supposed to encode those values into a string? The ldapjs documentation gives very little information about passing parameters to an extended operation.
TL:DR; The extended operation parameters need to be ASN.1 values encoded with the BER standard. This isn't a trivial task, so you might want an additional npm library, such as asn1 to help with this process.
After combing through ldapjs's code, reading up a bunch about ASN.1 and how LDAP uses the ASN.1 standard, and some trial and error, I was finally able to resolve this issue. Because of the distinct lack of documentation for this, I thought I would share what I learned on stackoverflow so others don't need to go through as much trouble as I did.
A working example
This uses the asn1 npm library to encode the data being sent.
const { Ber } = require('asn1')
// ...
const CTX_SPECIFIC_CLASS = 0b10 << 6
const writer = new Ber.Writer()
writer.startSequence()
writer.writeString(dn, CTX_SPECIFIC_CLASS | 0) // sequence item number 0
// I'm choosing to omit the optional sequence item number 1
writer.writeString(newPassword, CTX_SPECIFIC_CLASS | 2) // sequence item number 2
writer.endSequence()
client.exop('1.3.6.1.4.1.4203.1.11.1', writer.buffer, (err, value, res) => {
// ...
})
What is ASN.1?
ASN.1 is a language that's used to describe an interface for an object. These interfaces are special in that they are language agnostic - i.e. javascript can create an object that conforms to one of these interfaces, encode it, and send it to a python server which then decodes and validates the object against the same interface. Much of ASN.1 is not relevant to what we're trying to accomplish, but it's important to note that what we're trying to do is make an object that conforms to one of these ASN.1 interfaces (LDAP is built all around them).
What is BER?
BER describes a standard way to represent an object that conforms to an ASN.1 interface. Using the BER standard, we can encode javascript data into a buffer that can be understood by an LDAP server.
BER basics
BER is designed to be a very compact encoding standard. I'll go over the basics here, but I highly recommend this article if you want to get into more details about the binary representation of BER (it's tailored to LDAP users). A Layman's Guide to a Subset of ASN.1, BER, and DER is another great resource.
ASN.1 describes a number of basic object types, such as strings and numbers, and It describes structured object types such as a sequence or set. It also provides the power for users to use their own custom types.
In BER, each piece of data is prefixed by two bytes (usually): An identifier byte and a length-of-data byte. The identifier byte tags the data with information about the kind of data it contains (A string? sequence? custom type?). There are four "classes" of tags: universal (such as a string), application (LDAP defined some application tags which you might encounter), context-specific (See the "BER Sequences" section below), and private (not likely applicable here). A string tag's bit sequence will always be interpreted as a string tag, but the meaning for the bit sequence of a custom tag may vary on the environment, or even within a request.
In the asn1 npm library, you can write out a string element as follows:
writer.writeString('text')
To find all of the available functions, the author of this library asks that you peek into the source code.
BER Sequences
A sequence is used to describe an object (a set of key-value pairs) with a particular shape. Some elements may be optional while others are required. The RFC I was following gave the following description for its parameters. We need to conform to this sequence's interface in order to send our password reset parameters to LDAP.
PasswdModifyRequestValue ::= SEQUENCE {
userIdentity [0] OCTET STRING OPTIONAL
oldPasswd [1] OCTET STRING OPTIONAL
newPasswd [2] OCTET STRING OPTIONAL }
The [0], [1], and [2] all refer to context-specific tag numbers. A value tagged with the context-specific tag of 1 will be interpreted as the value for the oldPasswd argument. We don't need to use the global string tag to indicate that our value is of type string - LDAP can already infer that information using the interface we're conforming to. This means when writing a string in this sequence, instead of doing writer.writeString('text') as done before (which automatically used the global string tag), a tag number must be provided as follows:
const CTX_SPECIFIC_CLASS = 0b10 << 6
writer.writeString(newPassword, CTX_SPECIFIC_CLASS | 2) // The second optional parameter allows you to set a custom tag on the data being set (instead of the default string tag).
The first two bits of the tag byte are reserved for specifying the tag class (in this case, it's the context-specific class, or bits "10"). So, CTX_SPECIFIC_CLASS | 2 refers to the newPasswd sequence item described by the RFC. Note that if I want to omit an optional sequence entry, I just don't write out a value tagged with that sequence id.
Concluding Remarks
Hopefully this should give readers enough information to be able to format and send BER-encoded parameters for an extended LDAP operation. I do want to note that I am no ASN.1/BER expert - all of this information above is just how I understood these concepts from my own research over the past couple of days. So, there are likely a few things mis-explained in this post. Feel free to edit it if you happen to be more knowledgeable than me about this topic.
On sql server : Out put : 0x5C8C8AAFE7AE37EA4EBDF8BFA01F82B8
SELECT HASHBYTES('MD5', convert(varchar,getdate(),112)+'mytest#+')
On JavaScript : Out put : 5c8c8aafe7ae37ea4ebdf8bfa01f82b8
//to get Md5 Hash bytes
vm.getMd5Hashbytes = function () {
var currentDate = moment().format('YYYYMMDD');
var md5Hash = md5.createHash(currentDate + 'mytest#+');
return md5Hash;
}
angular-md5 module
Q : Can you tell me why this difference ? SQL server shows 0x as prefix.Why ?
This is purely a formatting issue. Both versions are producing an identical sequence of bytes. SQL Server and node just have different conventions when it comes to presenting these bytes in a human readable format.
You can get similar formatting by specifically telling SQL Server how to format your binary data
declare #hashAsBinary varbinary(max)
declare #hashAsText char(32)
set #hashAsBinary = HASHBYTES('MD5', '20160818mytest#+')
set #hashAsText = LOWER(CONVERT(varchar(max), #hashAsBinary, 2))
select #hashAsText
Which outputs:
5c8c8aafe7ae37ea4ebdf8bfa01f82b8
See SQL Server converting varbinary to string
I am not sure how else to explain it but it will take more space than a comment allows for so I will post it as an answer.
Look at the source code that you are referencing. At the end (lines 210 and 212) you will see it converts the binary value to a hex string (and then to lower case which does not matter unless you opt for a string comparison at the end). End result = your JavaScript library returns a representation using the type string formatted as hex.
Your Sql function HASHBYTES on the other hand produces a varbinary typed result (which is a different type than string (varchar)).
So you have 2 different data types (each living on their own space as you have not pulled one to the other). You never mention where you are doing the comparison, ie: on the database or are you pulling from the database to script. Either way to do a comparison you need to convert one type so you are either comparing 2 strings types OR comparing two binary types. If you do not compare similar types you will get unexpected results or run time exceptions.
If you are comparing using strings AND in JavaScript then look at your library that you are referencing, it already has a call named wordToHex, copy and paste it and reuse it to convert your Sql result to a string and then do a string comparison (do not forget to compare case insensitive or also make it lower case).
Edit
WebApi is black box for me.It is a 3rd party service.I just need to send the security token as mentioned above.
Assuming that the type accepted by that web api is byt[] appending 0x to your string in javascript and then sending it to the web api should work as in the web api will then translate the incoming parameter as a byte array and execute the comparison using the correct types. As this is a black box there is no way to know for certain unless you either ask them if the accepted type is indeed a byte array or to test it.
I have data in a standalone Neo4j REST server, including an index of nodes. I want pure JavaScript client to connect to Neo4j and serve the formatted data to d3.js, a visualisation library built on Node.js.
JugglingDB is very popular, but the Neo4j implementation was done "wrong": https://github.com/1602/jugglingdb/issues/56
The next most popular option on github is: https://github.com/thingdom/node-neo4j
looking at the method definitions https://github.com/thingdom/node-neo4j/blob/develop/lib/GraphDatabase._coffee
I'm able to use "getNodeById: (id, _) ->"
> node1 = db.getNodeById(12, callback);
returns the output from the REST server, including node properties. Awesome.
I can't figure out how to use "getIndexedNodes: (index, property, value, _) ->"
> indexedNodes = db.getIndexedNodes:(index1, username, Homer, callback);
...
indexedNodes don't get defined. I've tried a few different combinations. No joy. How do I use this command?
Also, getIndexedNodes() requires a key-value pair. Is there any way to get all, or a subset of the items in the index without looping?
One of the authors/maintainers of node-neo4j here. =)
indexedNodes don't get defined. I've tried a few different combinations. No joy. How do I use this command?
Your example seems to have some syntax errors. Are index1, username and Homer variables defined elsewhere? Assuming not, i.e. assuming those are the actual index name, property name and value, they need to be quoted as string literals, e.g. 'index1', 'username' and 'Homer'. But you also have a colon right before the opening parenthesis that shouldn't be there. (That's what's causing the Node.js REPL to not understand your command.)
Then, note that indexedNodes should be undefined -- getIndexedNodes(), like most Node.js APIs, is asynchronous, so its return value is undefined. Hence the callback parameter.
You can see an example of how getIndexedNodes() is used in the sample node-neo4j-template app the README references:
https://github.com/aseemk/node-neo4j-template/blob/2012-03-01/models/user.js#L149-L160
Also, getIndexedNodes() requires a key-value pair. Is there any way to get all, or a subset of the items in the index without looping?
getIndexedNodes() does return all matching nodes, so there's no looping required. Getting a subset isn't supported by Neo4j's REST API directly, but you can achieve the result with Cypher.
E.g. to return the 6th-15th user (assuming they have a type property set to user) sorted alphabetically by username:
db.query([
'START node=node:index1(type="user")',
'RETURN node ORDER BY node.username',
'SKIP 5 LIMIT 10'
].join('\n'), callback);
Cypher is still rapidly evolving, though, so be sure to reference the documentation that matches the Neo4j version you're using.
As mentioned above, in general, take a look at the sample node-neo4j-template app. It covers a breadth of features that the library exposes and that a typical app would need.
Hope this helps. =)
Neo4j 2 lets you do indices VIA REST. Docs here
REST Indicies
I have a pretty big array of JSON objects (its a music library with properties like artist, album etc, feeding a jqgrid with loadonce=true) and I want to implement lucene-like (google-like) query through whole set - but locally, i.e. in the browser, without communication with web server. Are there any javascript frameworks that will help me?
Go through your records, to create a one time index by combining all search
able fields in a single string field called index.
Store these indexed records in an Array.
Partition the Array on index .. like all a's in one array and so on.
Use the javascript function indexOf() against the index to match the query entered by the user and find records from the partitioned Array.
That was the easy part but, it will support all simple queries in a very efficient manner because the index does not have to be re-created for every query and indexOf operation is very efficient. I have used it for searching up to 2000 records. I used a pre-sorted Array. Actually, that's how Gmail and yahoo mail work. They store your contacts on browser in a pre-sorted array with an index that allows you to see the contact names as you type.
This also gives you a base to build on. Now you can write an advanced query parsing logic on top of it. For example, to support a few simple conditional keywords like - AND OR NOT, will take about 20-30 lines of custom JavaScript code. Or you can find a JS library that will do the parsing for you the way Lucene does.
For a reference implementation of above logic, take a look at how ZmContactList.js sorts and searches the contacts for autocomplete.
You might want to check FullProof, it does exactly that:
https://github.com/reyesr/fullproof
Have you tried CouchDB?
Edit:
How about something along these lines (also see http://jsfiddle.net/7tV3A/1/):
var filtered_collection = [];
var query = 'foo';
$.each(collection, function(i,e){
$.each(e, function(ii, el){
if (el == query) {
filtered_collection.push(e);
}
});
});
The (el == query) part of course could/should be modified to allow more flexible search patterns than exact match.
I will first explain what I'm trying to do then I will explain why just in case you get bored of reading the whole scenario.
Basically I have some HTML markup stored in a variable I now need to a wait to access the different elements within the variable. For example:
var markUp = "<h3>h3 tag</h3><p>paragraph tag</p>";
What I need to know is if there is a way for me to query the variable to retrieve say the h3 tag, in a similar way you would use the query function ? I have seen some other practices where people append the var to a hidden div then query the div. I would prefer to avoid this but if that is the only way I will proceed.
I have come across this problem whilst developing a drag and drop application, on drop i use a custom creator function to change the items structure once it is dropped.
If further explanation is needed please say, thanks advance Jonathan
You can use dojo._toDom to create a DOM fragment from your string.
var markup = "<h3>h3 tag</h3><p>paragraph tag</p><p>another paragraph</p>";
var domFragment = dojo._toDom(markup);
dojo.query("p", domFragment).forEach(function(element,i) {
console.debug(element.innerHTML);
});
The underscore prefix in _toDom means that it's a "private" member method of dojo. Normally, it's bad practice to use these as if they were public (like I do here). However, in the case of _toDom I believe it's generally considered acceptable, and according to this trac entry, it sounds like it'll be made public in the next version.