I'm trying to use the "Change User's Password" extended operation, as defined in this RFC which states that it takes a sequence of three optional parameters. However, it seems that ldapjs's client.exop() function only allows me to provide it with a string or buffer.
This is my attempt:
const dn = `uid=${username},ou=People,dc=${orginization},dc=com`
client.exop('1.3.6.1.4.1.4203.1.11.1', [dn, null, newPassword], (err, value, res) => {
// ...
})
And this is the resulting error:
TypeError: options.requestValue must be a buffer or a string
How am I supposed to encode those values into a string? The ldapjs documentation gives very little information about passing parameters to an extended operation.
TL:DR; The extended operation parameters need to be ASN.1 values encoded with the BER standard. This isn't a trivial task, so you might want an additional npm library, such as asn1 to help with this process.
After combing through ldapjs's code, reading up a bunch about ASN.1 and how LDAP uses the ASN.1 standard, and some trial and error, I was finally able to resolve this issue. Because of the distinct lack of documentation for this, I thought I would share what I learned on stackoverflow so others don't need to go through as much trouble as I did.
A working example
This uses the asn1 npm library to encode the data being sent.
const { Ber } = require('asn1')
// ...
const CTX_SPECIFIC_CLASS = 0b10 << 6
const writer = new Ber.Writer()
writer.startSequence()
writer.writeString(dn, CTX_SPECIFIC_CLASS | 0) // sequence item number 0
// I'm choosing to omit the optional sequence item number 1
writer.writeString(newPassword, CTX_SPECIFIC_CLASS | 2) // sequence item number 2
writer.endSequence()
client.exop('1.3.6.1.4.1.4203.1.11.1', writer.buffer, (err, value, res) => {
// ...
})
What is ASN.1?
ASN.1 is a language that's used to describe an interface for an object. These interfaces are special in that they are language agnostic - i.e. javascript can create an object that conforms to one of these interfaces, encode it, and send it to a python server which then decodes and validates the object against the same interface. Much of ASN.1 is not relevant to what we're trying to accomplish, but it's important to note that what we're trying to do is make an object that conforms to one of these ASN.1 interfaces (LDAP is built all around them).
What is BER?
BER describes a standard way to represent an object that conforms to an ASN.1 interface. Using the BER standard, we can encode javascript data into a buffer that can be understood by an LDAP server.
BER basics
BER is designed to be a very compact encoding standard. I'll go over the basics here, but I highly recommend this article if you want to get into more details about the binary representation of BER (it's tailored to LDAP users). A Layman's Guide to a Subset of ASN.1, BER, and DER is another great resource.
ASN.1 describes a number of basic object types, such as strings and numbers, and It describes structured object types such as a sequence or set. It also provides the power for users to use their own custom types.
In BER, each piece of data is prefixed by two bytes (usually): An identifier byte and a length-of-data byte. The identifier byte tags the data with information about the kind of data it contains (A string? sequence? custom type?). There are four "classes" of tags: universal (such as a string), application (LDAP defined some application tags which you might encounter), context-specific (See the "BER Sequences" section below), and private (not likely applicable here). A string tag's bit sequence will always be interpreted as a string tag, but the meaning for the bit sequence of a custom tag may vary on the environment, or even within a request.
In the asn1 npm library, you can write out a string element as follows:
writer.writeString('text')
To find all of the available functions, the author of this library asks that you peek into the source code.
BER Sequences
A sequence is used to describe an object (a set of key-value pairs) with a particular shape. Some elements may be optional while others are required. The RFC I was following gave the following description for its parameters. We need to conform to this sequence's interface in order to send our password reset parameters to LDAP.
PasswdModifyRequestValue ::= SEQUENCE {
userIdentity [0] OCTET STRING OPTIONAL
oldPasswd [1] OCTET STRING OPTIONAL
newPasswd [2] OCTET STRING OPTIONAL }
The [0], [1], and [2] all refer to context-specific tag numbers. A value tagged with the context-specific tag of 1 will be interpreted as the value for the oldPasswd argument. We don't need to use the global string tag to indicate that our value is of type string - LDAP can already infer that information using the interface we're conforming to. This means when writing a string in this sequence, instead of doing writer.writeString('text') as done before (which automatically used the global string tag), a tag number must be provided as follows:
const CTX_SPECIFIC_CLASS = 0b10 << 6
writer.writeString(newPassword, CTX_SPECIFIC_CLASS | 2) // The second optional parameter allows you to set a custom tag on the data being set (instead of the default string tag).
The first two bits of the tag byte are reserved for specifying the tag class (in this case, it's the context-specific class, or bits "10"). So, CTX_SPECIFIC_CLASS | 2 refers to the newPasswd sequence item described by the RFC. Note that if I want to omit an optional sequence entry, I just don't write out a value tagged with that sequence id.
Concluding Remarks
Hopefully this should give readers enough information to be able to format and send BER-encoded parameters for an extended LDAP operation. I do want to note that I am no ASN.1/BER expert - all of this information above is just how I understood these concepts from my own research over the past couple of days. So, there are likely a few things mis-explained in this post. Feel free to edit it if you happen to be more knowledgeable than me about this topic.
Related
We have a data model where each entity has 600 boolean values. All of this data needs to travel over the wire from a node.js backend to an Angular frontend, via JSON.
I was thinking about various ways to optimize it (this is an internal API and is not public, so adherence to best practices is less important than performance and saving bandwidth).
I am not a native Javascript speaker, so was hoping to get some feedback on some of the options I was considering, which are:
Turning it into a bitfield and using a huge (600-bit) BigInt.
Is this a feasible approach? I can imagine it would probably be pretty horrific in terms of performance
Splitting the 600 bits into 10 integers (since JS integers are 64 bit), and putting those into an array in the JSON
Base64 encoding a binary blob (will be decoded to a UInt8Array I'm assuming?)
Using something like Protobuf? It might be overkill because I don't want more than 1-2 hours spent on this optimization; definitely don't want to make major changes to the architecture either
Side note: We don't have compression on the server end due to infrastructure reasons, which makes this more complicated and is the reason for us implementing this on the data level.
Thanks!
as Evan points out, transforming your boolean for example into a single character for true="t" and false="f", the 600 boolean will become a joined string of 600 chars which can very well be split into 3 strings of 200 of the sizes, then once received on the front just concatenate the transit and if you want to recover your Bollean values from the string, with a simple reg it becomes possible.
I don't know how the data is set and then obtained, just changing this parameter to which I think needs to be automated.
Once the final string is obtained on the front here is an example of reg ex which can convert your string to an array with your 600 boolean. It is also possible to define indexes by defining an object instead of the array.
function convert_myBool(str)
{
/*var reg = new RegExp('.{1}', 'g');
var tmpTab = str.replace(reg, function(matched){
return matched == "t"?true:false;
});*/
//map is best
tmpTab = str.split('').map((value) =>{
return value == "t"?true:false;
});
return tmpTab;
};
I wrote this dynamically so of course it can be pondered, improved replaced etc. Hoping to have helped :)
Can it be sorted in any way? If there are boolean values that always occur in conjunction with a related value you may be able to group them and simplify.
Depending on what your use for that data is, you may be able to cache some of the it or memoize based on usage frequency. There would be a space tradeoff with caching, however.
On sql server : Out put : 0x5C8C8AAFE7AE37EA4EBDF8BFA01F82B8
SELECT HASHBYTES('MD5', convert(varchar,getdate(),112)+'mytest#+')
On JavaScript : Out put : 5c8c8aafe7ae37ea4ebdf8bfa01f82b8
//to get Md5 Hash bytes
vm.getMd5Hashbytes = function () {
var currentDate = moment().format('YYYYMMDD');
var md5Hash = md5.createHash(currentDate + 'mytest#+');
return md5Hash;
}
angular-md5 module
Q : Can you tell me why this difference ? SQL server shows 0x as prefix.Why ?
This is purely a formatting issue. Both versions are producing an identical sequence of bytes. SQL Server and node just have different conventions when it comes to presenting these bytes in a human readable format.
You can get similar formatting by specifically telling SQL Server how to format your binary data
declare #hashAsBinary varbinary(max)
declare #hashAsText char(32)
set #hashAsBinary = HASHBYTES('MD5', '20160818mytest#+')
set #hashAsText = LOWER(CONVERT(varchar(max), #hashAsBinary, 2))
select #hashAsText
Which outputs:
5c8c8aafe7ae37ea4ebdf8bfa01f82b8
See SQL Server converting varbinary to string
I am not sure how else to explain it but it will take more space than a comment allows for so I will post it as an answer.
Look at the source code that you are referencing. At the end (lines 210 and 212) you will see it converts the binary value to a hex string (and then to lower case which does not matter unless you opt for a string comparison at the end). End result = your JavaScript library returns a representation using the type string formatted as hex.
Your Sql function HASHBYTES on the other hand produces a varbinary typed result (which is a different type than string (varchar)).
So you have 2 different data types (each living on their own space as you have not pulled one to the other). You never mention where you are doing the comparison, ie: on the database or are you pulling from the database to script. Either way to do a comparison you need to convert one type so you are either comparing 2 strings types OR comparing two binary types. If you do not compare similar types you will get unexpected results or run time exceptions.
If you are comparing using strings AND in JavaScript then look at your library that you are referencing, it already has a call named wordToHex, copy and paste it and reuse it to convert your Sql result to a string and then do a string comparison (do not forget to compare case insensitive or also make it lower case).
Edit
WebApi is black box for me.It is a 3rd party service.I just need to send the security token as mentioned above.
Assuming that the type accepted by that web api is byt[] appending 0x to your string in javascript and then sending it to the web api should work as in the web api will then translate the incoming parameter as a byte array and execute the comparison using the correct types. As this is a black box there is no way to know for certain unless you either ask them if the accepted type is indeed a byte array or to test it.
The problem I am trying to solve is this
Given a string containing code (entered by the user via some kind of web based form), such as:
"a = b + 5; return Math.abs(a);"
Extract a list of tokens that are considered identifiers:
a, b
In general are there readily available existing solutions? If not, what is a good starting point for writing an simple algorithm for this purpose?
I tried http://ace.c9.io/#nav=about, but was not able to find a method that readily returned the token type of an entire document. According to the API reference getTokens(Number row) should return the information I want. However, this method ended up returning the entire line as type text
Thank you in advance
I have seen OpenGL codes written like this:
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(1,1,1,1,1,-1);
glMatrixMode(GL_MODELVIEW);
and I have seen OpenGL codes written like this:
glEnable(2896);
glDisable(3042);
Notice the number values in the glEnable() and glDisable() methods.
My real question is: Does anyone have a link to a website that has a list of every method or which number value corresponds to whichever mode you are putting in the code? Like, what does glEnable(2896); actually mean?
The enum names and values live in gl.xml, off of the main spec registry page.
Given your example of glEnable(2896):
2896 in hex is 0x0B50
Searching gl.xml for that value lands you on
<enum value="0x0B50" name="GL_LIGHTING"/>
Which you can see corresponds to GL_LIGHTING
The numbers are the numeric values for OpenGL tokens as defined by the OpenGL specification. You can find the specifications for the various OpenGL versions at http://www.opengl.org/registry/
The definitions for the tokens is written down in a form usable by a C or C++ compiler in the GL/gl.h header file.
However it's strongly discouraged to use the numeric values, since in code they're just magic numbers.
Like, what does glEnable(2896); actually mean?
Just search the GL/gl.h for the token which is defined to this value. They're usually written in hexadecimal so you have to convert that decimal representation first. Like this (using a *nix style shell):
dw ~ % grep $(printf '%X' 2896) /usr/include/GL/gl.h
#define GL_LIGHTING 0x0B50
I have data in a standalone Neo4j REST server, including an index of nodes. I want pure JavaScript client to connect to Neo4j and serve the formatted data to d3.js, a visualisation library built on Node.js.
JugglingDB is very popular, but the Neo4j implementation was done "wrong": https://github.com/1602/jugglingdb/issues/56
The next most popular option on github is: https://github.com/thingdom/node-neo4j
looking at the method definitions https://github.com/thingdom/node-neo4j/blob/develop/lib/GraphDatabase._coffee
I'm able to use "getNodeById: (id, _) ->"
> node1 = db.getNodeById(12, callback);
returns the output from the REST server, including node properties. Awesome.
I can't figure out how to use "getIndexedNodes: (index, property, value, _) ->"
> indexedNodes = db.getIndexedNodes:(index1, username, Homer, callback);
...
indexedNodes don't get defined. I've tried a few different combinations. No joy. How do I use this command?
Also, getIndexedNodes() requires a key-value pair. Is there any way to get all, or a subset of the items in the index without looping?
One of the authors/maintainers of node-neo4j here. =)
indexedNodes don't get defined. I've tried a few different combinations. No joy. How do I use this command?
Your example seems to have some syntax errors. Are index1, username and Homer variables defined elsewhere? Assuming not, i.e. assuming those are the actual index name, property name and value, they need to be quoted as string literals, e.g. 'index1', 'username' and 'Homer'. But you also have a colon right before the opening parenthesis that shouldn't be there. (That's what's causing the Node.js REPL to not understand your command.)
Then, note that indexedNodes should be undefined -- getIndexedNodes(), like most Node.js APIs, is asynchronous, so its return value is undefined. Hence the callback parameter.
You can see an example of how getIndexedNodes() is used in the sample node-neo4j-template app the README references:
https://github.com/aseemk/node-neo4j-template/blob/2012-03-01/models/user.js#L149-L160
Also, getIndexedNodes() requires a key-value pair. Is there any way to get all, or a subset of the items in the index without looping?
getIndexedNodes() does return all matching nodes, so there's no looping required. Getting a subset isn't supported by Neo4j's REST API directly, but you can achieve the result with Cypher.
E.g. to return the 6th-15th user (assuming they have a type property set to user) sorted alphabetically by username:
db.query([
'START node=node:index1(type="user")',
'RETURN node ORDER BY node.username',
'SKIP 5 LIMIT 10'
].join('\n'), callback);
Cypher is still rapidly evolving, though, so be sure to reference the documentation that matches the Neo4j version you're using.
As mentioned above, in general, take a look at the sample node-neo4j-template app. It covers a breadth of features that the library exposes and that a typical app would need.
Hope this helps. =)
Neo4j 2 lets you do indices VIA REST. Docs here
REST Indicies