I have data in a standalone Neo4j REST server, including an index of nodes. I want pure JavaScript client to connect to Neo4j and serve the formatted data to d3.js, a visualisation library built on Node.js.
JugglingDB is very popular, but the Neo4j implementation was done "wrong": https://github.com/1602/jugglingdb/issues/56
The next most popular option on github is: https://github.com/thingdom/node-neo4j
looking at the method definitions https://github.com/thingdom/node-neo4j/blob/develop/lib/GraphDatabase._coffee
I'm able to use "getNodeById: (id, _) ->"
> node1 = db.getNodeById(12, callback);
returns the output from the REST server, including node properties. Awesome.
I can't figure out how to use "getIndexedNodes: (index, property, value, _) ->"
> indexedNodes = db.getIndexedNodes:(index1, username, Homer, callback);
...
indexedNodes don't get defined. I've tried a few different combinations. No joy. How do I use this command?
Also, getIndexedNodes() requires a key-value pair. Is there any way to get all, or a subset of the items in the index without looping?
One of the authors/maintainers of node-neo4j here. =)
indexedNodes don't get defined. I've tried a few different combinations. No joy. How do I use this command?
Your example seems to have some syntax errors. Are index1, username and Homer variables defined elsewhere? Assuming not, i.e. assuming those are the actual index name, property name and value, they need to be quoted as string literals, e.g. 'index1', 'username' and 'Homer'. But you also have a colon right before the opening parenthesis that shouldn't be there. (That's what's causing the Node.js REPL to not understand your command.)
Then, note that indexedNodes should be undefined -- getIndexedNodes(), like most Node.js APIs, is asynchronous, so its return value is undefined. Hence the callback parameter.
You can see an example of how getIndexedNodes() is used in the sample node-neo4j-template app the README references:
https://github.com/aseemk/node-neo4j-template/blob/2012-03-01/models/user.js#L149-L160
Also, getIndexedNodes() requires a key-value pair. Is there any way to get all, or a subset of the items in the index without looping?
getIndexedNodes() does return all matching nodes, so there's no looping required. Getting a subset isn't supported by Neo4j's REST API directly, but you can achieve the result with Cypher.
E.g. to return the 6th-15th user (assuming they have a type property set to user) sorted alphabetically by username:
db.query([
'START node=node:index1(type="user")',
'RETURN node ORDER BY node.username',
'SKIP 5 LIMIT 10'
].join('\n'), callback);
Cypher is still rapidly evolving, though, so be sure to reference the documentation that matches the Neo4j version you're using.
As mentioned above, in general, take a look at the sample node-neo4j-template app. It covers a breadth of features that the library exposes and that a typical app would need.
Hope this helps. =)
Neo4j 2 lets you do indices VIA REST. Docs here
REST Indicies
Related
In my scenario a user can "like" the profile of another user. As a result I have a subcollection called "likedBy" for each user, where I create a document for a specific user, e.g.:
users(col) -> User A(doc) -> likedBy(col) -> User B (doc), User C (doc)
So in the rare scenario of a user being deleted, I want to delete all the likes issues by that user.
I am just not sure if this is even possible (a workaround could be to just save the userId again in said document and query for that).
What I am basically looking for is something like this:
db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();
The problem is, that I can not create an index for the documentId in the Firestore console.
UPDATE 2020-01-23:
For updates on this, see a conversation with Sam Stern on the group board:https://groups.google.com/d/msgid/google-cloud-firestore-discuss/e1b47358-b106-43a0-91fb-83c97d6244de%40googlegroups.com
Much of the discussion comes from the apparent fact that there is a SINGLE "master index" of ALL records in a database based on the FULLY QUALIFIED path to the document (hence only needing to be unique "within a collection").
To "accelerate" document references, the JS SDK actually "pre-pends" the collection path onto whatever info is passed in .doc to "conveniently" use that master index
i.e. all of these are exactly equivalent
db.doc('collection/docId/collection/docId/collection/docId")
db.collection('collection").doc("docId/collection/docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.collection("collection/docId/collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId/collection/docId").collection("collection").doc("docId")
db.collection("collection/docId/collection/docId/collection").doc("docId")
--- they ALL create the same index reference 'collection/docId/collection/docId/collection/docId" to find the document in the "master index".
(in my not-at-all-humble-opinion) FieldPath.documentId() was implemented (incorrectly) to "conveniently" match this behavior thus requiring the fully-qualified path, not the docId, when it should have been implemented like any other query, and required creating and maintaining a NEW INDEX to accommodate the query.
The code for this behavior was written BEFORE collectionGroups were implemented - and never documented the hack used didn't match the METHOD NAME used.
The work around is to require the Coder to copy the docId as a field in each document, and write your queries on that. I already wrote my own layer between Firestore.js and my application to abstract the behavior, and will probably simply implement this as a basic feature of the library.
But this is clearly a MISTAKE, and so far everybody keeps trying to tell me it makes sense, and that they'll change the documentation to match the existing behavior (but not the method name).
As I wrote previously, I keep getting handed a ratty bunch of daisies, and being told "See? these are roses!! The documentation calls them roses!! Roses by any other name smell as sweet, and all that!!"
No Update Expected Unless They Get Embarrassed Enough
UPDATE 2020-01-10: I have built a demo app showing the exact bug, and have sent it to Firebase support as requested. For some dang reason, the support critter considers it a "feature request", in spite of it clearly a bug. When a URL is called in the form "/ShowInfo/showID", the App signs in to Firebase Auth anonymously; then calls a query on the collectionGroup (3 levels deep) using FieldPath.documentId() "==" showID
It makes the query 3 ways:
1) Once with only the showID- which fails with the familiar "Invalid query. When querying a collection group by FieldPath.documentId(), the value provided must result in a valid document path, but 'pqIPV5I7UWne9QjQMm72'(the actual showID) is not because it has an odd number of segments (1)."
2) Once with a "Relative Path" (/Shows/showID), which doesn't have the error, but returns no document.
3) Finally with the "Full Path" (/Artists/ArtistID/Tour/tourID/Shows/showID). This doesn't have an error, and does return a document - but if I have the full path, why do I need the query on the collectionGroup? And I don't have the full path - the showID (above) comes in as part of a URL (a link to the show data, obviously) - I hand-faked it for the test.
Waiting for response.
UPDATE 2019-12-02: Firebase support reached out to ask if I still wanted this solved. Duh.
UPDATE 2019-09-27: Firebase Support has acknowledged this is a bug. No word on when it will be fixed. documentId() should, indeed, be able to be used directly against only the document Id.
documentID can be used as part of a query but, as #greg-ennis notes above, it needs an even number of segments. As it turns out, if you truly need a collectionGroup (I do), then the "Id" to be compared needs to redundantly add the collectionGroup ID/name as a segment:
db.collectionGroup('likedBy')
.where(firebase.firestore.FieldPath.documentId(), '==', "likedBy" + "/" + "User A")
.get();
I've used it and it (*sorta) works. (admittedly, it took me 2 hours to figure it out, and the above question helped focus the search)
On the other hand, this specific case is not where you want to use collectionGroup. Remember what collection group is - a way to refer to a a set of separate collections as if they were one. In this case the "collection" that holds "User A"s likes exists as a collection under "User A"s doc. Simply delete that single collection before deleting "User A"s doc. No need to bring everybody else's likes into it.
Sorta: the field path compared apparently has to be the complete path to the document. If you know the documentId, but for "reasons" you do not know the complete path, including which document the sub-collection is a part of (kinda why you were using the collectionGroup approach in the first place), this now seems a tadly dead-end. Continuing working on it.
Verified and Bug Report filed: FieldPath.documentID() does not compare against the documentId; it compares against the fully segmented document path, exactly as you would have to provide to .doc(path):
To wit: to find a document at "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want"
using the collectionGroup "Travesty"
db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "document_id_I_want")
...will fail, but...
db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()
...will succeed. Which makes this useless, since if we had all that info, we would just use:
db.doc("TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()
There is no way you can use the following query:
db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();
And this is because collection group queries work only on document properties and not on document ids. According to the official documentation regarding collection group queries:
db.collectionGroup('landmarks').where('type', '==', 'museum');
You query the landmarks subcollection where the type property holds the value of museum.
A workaround could be to store the id of the user in an array and use array-contains but remember, for each collection group query you use, you need an index and unfortunately you cannot create such an index programmatically. Even if you can create an index in the Firebase console, it won't help you since you get those ids dynamically. So is not an option to create an index for each user separately because you'll reach the maximim number of indexes very quickly.
Maximum number of composite indexes for a database: 200
To solve this kind of problems, you should consider adding an array under each user object and use a query like this:
usersRef.where("usersWhoLikedMe", "array-contains", "someUserId")
Where usersWhoLikedMe is a property of type array.
If you add User A and B ids to the doc itself:
users(col) -> User A(doc) -> likedBy(col) -> User B ({user_id: B, profile_liked_id: A})
Then you can query using:
db.collectionGroup('likedBy').where('user_id', '==', 'B');
if the reference is a collection, the value you compare to needs to be document id (the last segment of a full path)
this is a error message if you violated the rule
Invalid query. When querying a collection by documentId(), you must
provide a plain document ID, but 'a/b/c' contains a '/' character
if the reference is a collectonGroup, the value you compare to needs to be a full document path <-- which is very redundant in my opinion
Invalid query. When querying a collection group by documentId(), the
value provided must result in a valid document path, but
'a/b/c' is not because it has an odd number of
segments (3)
tested recently
I'm trying to use the ES6 String template to query my database
My SQL is like :
SELECT * FROM table WHERE id IN ();
The in must contains each items of an array, so, i tried something like this :
return connector.query`SELECT * FROM table WHERE AnalogHistory.TagName IN (${sensor})`.then(result => {return result.recordset});
This doesn't work. But if i try something like this :
return connector.query`SELECT * FROM table WHERE AnalogHistory.TagName IN (${sensor[0]},${sensor[1]}, ${sensor[2]}, ...)`.then(result => {return result.recordset});
Well this time, it work. So do you guys know what differ and on to do it by the first way (the cleaner) ?
We can't be absolutely sure without knowing how query handles the tag parameters, but the difference is that in the first case, you've provided query with the array as a single parameter; and in the second case, you've provided query with a series of individual parameters with commas in-between them.
Apparently it's happy with the second, and not with the first. Whether there's a third option that would be less laborious but also functional depends entirely on how query is designed to be called.
Re your comment saying it's the mssql npm package: I don't see anything in its documentation to suggest that it supports passing arrays into the query tag function as parameters. Which is really unfortunate.
I have been using the Lookback API to query for user stories from the Rally environment. While the querying functionality is stronger than the WsapiDataStore, by allowing me to query using the RPM hierarchy, it does not seem to be able to return full data fields' values, such as Owner and Project. Instead, the OIDs for these fields are returned. To try to work around this, my idea was to first do a Lookback API query to get all the story OIDs within the RPM hierarchy I am concerned with. I will capture the story OIDs and keep them in an array. Then, I can use a WsapiDataStore query to get the detailed info for the stories matching the OIDs in the array. When using the Lookback API, I have the option to use the 'in' operator, so the query would look like this:
{
property: 'ObjectID',
operator: 'in',
value: [ '71352862', '44523976', '61138496' ]
}
I can't use this functionality in the WsapiDataStore however. Also, when I try to 'OR' them all together in one long query string I am getting an error about an invalid request. I assume the query string is too long since in most cases I am searching for about 1000 User Stories. I would prefer not to have to make a separate query for each OID but right now that is seeming like the only solution. Is there a way to get full details from the Lookback API, or at least filter using an array on the WsapiDataStore query?
ObjectID now supports the in operator.
From the WSAPI docs:
Here is an example usage: (ObjectID in 1,2,3)
I'm interested in using the visualsearch.js control for my website but, having read through the documentation, I am still unclear regarding how to effectively obtain the output search collection data. Based on the example, the output string is constructed through serialization of the search collection. However, I was wondering if there is a way to access the search collection in a more array-like fashion (so that for/in loops can be used) rather than having to parse a single serialized string. Ultimately, I need to construct SQL queries from the search collection data.
If there is an even more efficient or appropriate way of accessing the search collection data, please let me know!
Thanks!
as far as i know there are 2 ways to fetch data from visual search
it is also directly explained in their documentation in usage #4
like you said, the stringified version of the search.
visualSearch.searchBox.value();
// returns: 'country: "United States" state: "New York" account: 5-samuel title: "Pentagon Papers"'
or the facetted object to loop over
visualSearch.searchQuery.facets();
// returns: [{"country":"United States"},{"state":"New York"},{"account":"5-samuel"},{"title":"Pentagon Papers"}]
as you can see, this option gives you an array, per facet that was filtered on, and for each asset the value that was entered.
mhmmm.. ok, the answer is not so straightforward. I would suggest you to get some practice with backbone structure just making some modification to the todo-list app. It is a great startpoint. So you get familiar with some of the wonderful backbone.js methods for collections
The Basic idea is the following:
With visualsearch you can obtain a list of "facets", that is to say an array of key/values objects.
var myFacets = visualSearch.searchQuery.facets();
//my facets is then something like [{"field1":"value1-a"},{"field2":"value2-c"}]
after this you can use myFacets elements to iterativrely filter you collection with the WONDERFUL filter method hinerithed from _underscore lib.
How to do it? You can use the _.each method in the underscore lib
_.each(myFacets,function(facet){
myCollection=myCollection.filter(function(item){
return item.get(facet.get('category')) == facet.get('value');
});
});
}
Here you use the filter method of backbone.js, which returns only the values are true according to your clause. So, you filter your collection once for each single facet. It is like telling to javascript: "Return me only the elements of the collection which match with this facets (value)", and you do it iteratively for all the different facets you got.
Hope this helps.
Ah.. one last thing, just to mess ideas up :-) :Visualsearch is built on backbone.js, and the searchQuery object is nothing but a backbone Collection, so you can use the methods and the properties of the basic backbone collection. Read this line again if this is not clear, because this can be a key point for future implementations! :-)
I suggest you to have a look at the search_jquery.js file in the lib/js/models folder. It's very interesting...
I have a pretty big array of JSON objects (its a music library with properties like artist, album etc, feeding a jqgrid with loadonce=true) and I want to implement lucene-like (google-like) query through whole set - but locally, i.e. in the browser, without communication with web server. Are there any javascript frameworks that will help me?
Go through your records, to create a one time index by combining all search
able fields in a single string field called index.
Store these indexed records in an Array.
Partition the Array on index .. like all a's in one array and so on.
Use the javascript function indexOf() against the index to match the query entered by the user and find records from the partitioned Array.
That was the easy part but, it will support all simple queries in a very efficient manner because the index does not have to be re-created for every query and indexOf operation is very efficient. I have used it for searching up to 2000 records. I used a pre-sorted Array. Actually, that's how Gmail and yahoo mail work. They store your contacts on browser in a pre-sorted array with an index that allows you to see the contact names as you type.
This also gives you a base to build on. Now you can write an advanced query parsing logic on top of it. For example, to support a few simple conditional keywords like - AND OR NOT, will take about 20-30 lines of custom JavaScript code. Or you can find a JS library that will do the parsing for you the way Lucene does.
For a reference implementation of above logic, take a look at how ZmContactList.js sorts and searches the contacts for autocomplete.
You might want to check FullProof, it does exactly that:
https://github.com/reyesr/fullproof
Have you tried CouchDB?
Edit:
How about something along these lines (also see http://jsfiddle.net/7tV3A/1/):
var filtered_collection = [];
var query = 'foo';
$.each(collection, function(i,e){
$.each(e, function(ii, el){
if (el == query) {
filtered_collection.push(e);
}
});
});
The (el == query) part of course could/should be modified to allow more flexible search patterns than exact match.