We Keep Coding

Storing tags in a graph database

I've found some advice for setting up tagging systems in relational and document databases, but nothing for graph/multi-model databases.
I am trying to set up a tagging system for documents (let's call them "articles") in ArangoDB. I can think of two obvious ways to store tags in a multi-model (graph+document) database like Arango:
as an array within each article document (document database-style)
as a separate document class with each tag as a unique document and edges connecting tag documents to the article documents (something closer to relational database-style)
Are these in fact the two main ways to do this? Neither seems ideal. For example:
If I'm storing tags within each article document, I can index the tags and presumably ArangoDB is optimizing the space they use. However, I can't use graph features to link or traverse tags (or I have to do it separately).
If I'm storing tags as separate tag documents, it seems like extra overhead (an extra query) when I just want to get a list of tags on a document.
Which leads me to an explicit question: with regard to the latter option, is there any simple way to automatically make connected 'tag' documents show up within the article documents? E.g. have an array property that somehow 'mirrored' the tag.name properties of the connected tag documents?
General advice is also welcome.

#Joachim Bøggild linked to Mike Williamson: https://mikewilliamson.wordpress.com/2015/07/16/data-modeling-with-arangodb/
I would agree with Williamson that "Compact by default" is generally the way to go. You can then extract vertices (aka. nodes) from properties if/when the actual need emerges. It also avoids creating an overly interconnected graph structure which would be slow for all kinds of traversal queries.
However, in this case, I think having Tag vertices (i.e. "documents", in your terminology) is good to have, because you can then store meta-data on the tag (like count), and connect it to other tags and sub-tags. It seems very useful and immediately foreseeable in the particular case of tags. Having a vertex, which you can add more relationships to if/when you need them, is also very extensible, so you keep your future options more open (more easily, at least).
It seems Williamson agrees that Tags warrant special consideration:
"But not everything belongs together. Any attribute that contains a
complex data structure (like the “comments” array or the “tags” array)
deserves a little scrutiny as it might make sense as a vertex (or
vertices) of its own."
The original question by #ropeladder poses the main objection that it would require extra overhead (an extra query). I think it might be premature optimization to think too much about performance at this stage. After all; the extra query might be fast, or it might actually be joined with and included in the original query. In any case, I would quote this:
“In general, it’s bad practice to try to conflate nodes to preserve
query-time efficiency. If we model in accordance with the questions we
want to ask of our data, an accurate representation of the domain will
emerge. Graph databases maintain fast query times even when storing
vast amounts of data. Learning to trust our graph database is
important when learning to structure our graphs without denormalizing
them.”
--- from page 64, chapter 'Avoiding Anti-patterns', in the book 'Graph Databases', a book co-written by Eifrem, the founder of Neo4j, another
very popular native graph database. It's free and available online
here: https://neo4j.com/graph-databases-book/
See also this article on some anti-patterns (dense vs sparse graphs), to supplement Williamsons points: https://neo4j.com/blog/dark-side-neo4j-worst-practices/
Extra section included for completeness, to those who want to dive a little bit deeper into this question:
Answering Williamson's own criteria for deciding whether something should be a vertex/node on its own, instead of leaving it as a property on the document vertex:
Will it be accessed on it’s own? (ie: showing tags without the document)
Yes. Browsing tags available in the system could be useful.
Will you be running a graph measurement (like GRAPH_BETWEENNESS) on it?
Unsure. Likely not.
Will it be edited on it’s own?
Yes, probably. A user could edit it separately. Maybe an admin/moderator wants to clean up the tag names (correct spelling errors), or clean up their structure (if you have sub-tags).
Does/could the tags have relationships of it’s own? (assuming you care)
Yes. They could. Sub-tags, or other kinds of content than merely documents. Actually, it's very useful to be able to click a tag and immediately see all documents with that tag. That would presumably be sub-optimal with tags stored as a property array on each document. Whereas a graph database is fundamentally optimized for the case of querying vertices adjacent to other vertices (aka. nodes).
Would/should this attribute exist without it’s parent vertex?
Yes. A tag could/should exist even if the last tagged document was deleted. Someone might want to use that tag later on, and it represents domain information you might want to preserve.

You already mention most of the available decision criterias. Maybe I can add some more:
Relational tags inside the documents could use array indices to filter on them, which could make queries on them fast. However, if you like to add a rating or an explanation to each item of that tag array, there is no way to. If you want to count the documents tagged, this may also be more expensive than counting all edges that originate from a specific tag, or maybe find all tags matching a search criteria.
One of the powers of multi model is, that you don't need to decide between the both aproaches. You can have an edge collection connecting tags with attributes to your documents, and have an indexed array with the same (flat) tags inside of the document. If you find all (or most) of your queries just use one method, try to convert the rest and remove the other solution. If that doesn't work, your application simply needs both of them.
In both cases finding other tagged documents alongside could be done in a subequery:
LET docs=(FOR ftDoc IN FULLTEXT(articles, 'text', 'search')
COLLECT tags = ftDoc.tags INTO tags RETURN {tags, ftDoc})
LET tags = FLATTEN(FOR t IN docs[*].tags RETURN t)
LET otherArticles = (FOR oneTag IN tags
FOR oneD IN articles FILTER oneTag IN oneD.tag RETURN oneD._key)
RETURN {articles: docs, tags: tags, otherArticles: otherArticles}

The answer to your explicit question about if a connected document could automatically show up inside your document is unfortunately no. I have made an ArangoDB graph with separate tag documents, but I am seriously considering just turning it into properties on the individual items since the tags seem to follow the criteria for being properties, not related items.
Mike Williamson has done a nice blog post about this: https://mikewilliamson.wordpress.com/2015/07/16/data-modeling-with-arangodb/
He argues that having a lot of edges from a single vertex is slow, and that would be the case with the number of edges from a popular Tag vertex.

Is it right to put data in html5 data-*?

I have got a case like this:
There are many 'Rocks' on the screen, each one has properties like gram, score and speed. Once click the rock, we give out its properties.
Now i put all rocks' properties on the dom with data-*, like data-gram, data-score and data-speed. 'e.target' is used to get the dom and so are all properties.
<div class="rocks" data-gram="10" data-score="100" data-speed="10"></div>
I want to know is it right, and is there any disadvantage?
Also,is there any other better way?

It is right, these data- fields are made for these kind of uses.
Another way to do it is simply creating an array or object in Javascript that holds that information. As you're accessing the DOM when you click the element, it simply won't matter.
If you were trying to access the data without a DOM access, using a Javascript array/object should be faster in that case.
It's up to you really.

Yes. It's perfectly valid to store data associated with HTML attributes in this way. MDN has a whole article about it. https://developer.mozilla.org/en/docs/Web/Guide/HTML/Using_data_attributes The question I would be asking is does this fit with how your consuming / interacting with that data. Are you using a javascript framework? Have you considered loading this data in from an external source as JSON? Would that be a better fit? In my case I would tend to load this data in from an external json feed. I find that more flexable and it suits libraries like angular.

Adobe DTM and arrays with varying numbers of elements

On our site, we have pages that can be assigned tags, and I'm having issues with figuring out how to handle this with Adobe DTM/Analytics. Each page can have between 1 and n tags being captured in an array. I can see them in the developer console just fine, ex:
> data.page.termIds
< [513, 1787, 115, 4330]
Each element corresponds to the tag ID.
I've been searching online but can't find any answers that really help me. This blog post is close, but it looks like they need to create a separate data element for each element in the array. I also found this question in the adobe forums, but it's similar to the blog post.
Does anyone know if this is even possible? Are we expected to create a block of data elements, with each one assigned to an array index?

Do not bother with the JS Object option unless you want to target a top level js object/variable. So for example foobar is okay but foo['bar'] or foo.bar is not. If you want to target anything "nested", skip JS Object and use Custom Script to check for it and return it, instead. This is because DTM doesn't do a good job of parsing or checking if the parent nodes exist before returning it. This is basically what was already said in the blog link you mentioned.
As for what the Data Element is returning and how to use it...the Data Element itself will have a return value of exactly what you return. So if you return an object, it will be an object. If you return an array, it will be an array. The trick (or caveat, or bug, depending on how you want to look at it) is how you reference or use the Data Element within the tools and rules.
The long story short of it is this: If you intend to use the %data_element_name% syntax (e.g. in any of the built in form fields in the rules, config settings, etc.) then you should only have your Data Element return a string type value (this is basically the issue the forum post link had, and I assume the issue you are running into).
If you want to be able to access it as some other type (e.g. the array, or js object in general), you need to use _satellite.getVar('data_element_name') syntax. The obvious drawback to this is it basically makes using DTM in a "non-coder" way useless in that you cannot use this in any of the built-in fields in rules. You can only use it in contexts where you can write javascript, e.g. Data > Custom type Conditions or in Javascript / 3rd Party Tag containers.
But one question for you is, what are you ultimately using the Data Element for? For example, if the end game here is you are looking to populate a prop or eVar or some other variable with a comma delimited string.. well just return your array as such in your Data Element and then you can use the %data_element_name% syntax.

Capturing the data is one thing and formatting is another.
My question is how do you need to see this data organized in reporting? Capturing the array itself doesn't do much unless you know what format you need it in. Are you hoping to capture these IDs and classify them? Do you want to see the array list by page?
Can you provide more detail on how you'll be reporting on these values?

Optimal way to pass system values to javascript

What is the most effective way to pass object and category ids or other system variables which shouldn't be presented to the user, from server to the browser?
Let's say I have a list of items where I can do something with each of them by javascript, for example show tooltip html or add to favorites by ajax, or display on a map. Where is it best to save that tooltip html, or database id, or geoposition?
Some options I can think of are:
some dictionary within <script></script> tag for each item,
microformats,
inline xml,
rel attributes,
css class names with specific information, e.g. class="lat-12345 uid-45678",
one <script></script> with a dictionary of html ids mapping dictionaries with system values in the template,
javascript generated from the database and included via <script src="..."></script> with a dictionary of html ids mapping dictionaries with system values in the template,
ajax requests for all cases when I need more information than just id,
event handlers with parameters within html tags, e.g. onmouseover="tooltip(this, 123, 'Hello world!')".
P.S. I prefer unobtrusive solutions and also the loading/execution time is important.

Perhaps I am missing something... why not just JSON?
How you "send" it (either in the initial page load as "javascript" or via AJAX or whatnot) is really just a trivial detail determined mostly by when the data is available. (JSON is a subset of legal JavaScript syntax.)
Then it's just a matter of the correct transformation. Of course, by pushing this to JSON/JS, you may render some non-JS clients non-interoperable, if that's a consideration for you. If such is indeed the case, why not just perform the transformation server-side using well, any number of the techniques you put at top?
You can also use arbitrary attributes in HTML (the HTML5 spec may include "data-*" which is formally legalized) -- while not technically "correct", all major web-browsers will accept unknown attributes which can be accessed through the DOM API.

I'd prefer a single AJAX call to fetch whatever data you know you need at the outset, so you can have a simple JSON object available in your script. You can, of course, supplement that with additional calls should you find you need more information.
If that's impractical, then "hardcoding" a JavaScript object in a <script>...</script> tag is the next best option. Of course, "hardcoding" is from the browser's perspective. The actual content would surely be written by server-side script from your database.

One method you can use is custom attributes. I think you refer to this as micro-formats, but I am not entirely sure if they are the same thing so I have written a description below.
Having hit the same question before, I basically use something like the following:
<div data-pid="1234">
<a href="#" class="add-to-favourites">
<img src="addToFavourites.png" />
</a>
</div>
$("a.add-to-favourites").click(function() {
$.load("/Favourites/Add.php?prodID="+$(this).parent().attr("data-pid"));
});
This should do exactly what you want to do. The reason I have put the pid in the div, not the a tag, is that you can then place all the other product information within the div with other actions the user can take, for example displaying a tooltip on mouseover using data-description, or displaying on a map using data-geo-x and data-geo-y. Of course you can name these anything you want.
Support / Acceptance
This is becoming a perfectly accepted way to do what you want to do. HTML 5 supports this for precisely the kind of thing you are trying to achieve.
So it is supported by HTML 5, but what about HTML 4?
It may make HTML 4 invalid, but the world is moving on to bigger and better things. Older browsers (IE6 and before, FF1 / 2, Opera 7 / 8 / 9) are becoming less common so it shouldnt be a problem. It wont actually break older browsers - the functionality will still work.
Important validity note
Make sure you prepend the data- onto the attribute name. This will make the attribute perfectly valid in HTML 5.
A few extra hints
In jQuery 1.5, I have heard from an answer to my question that you can simply specify attr("pid") to return the value of data-pid. If this is the case then I would be careful when naming the second part of the attribute name after the name of an actual attribute (for example, instead of data-id, use data-pid - especially if the id attribute is specified. I am not sure what effect it would have if you didn't do this, but its better to avoid the problem in the first place than have issues with the site at a later date due to this.
Hope this is what you were looking for.

ASP.NET offers a very convenient way to do this. You can simply write a JavaScript object. I am sure other templating engines offer similar ways to do this.
var person = {
Name : <%= _person.Name %>,
Age : <%= _person.Age %>
};

I would implement a Javascript singleton AppCacheManager that initializes in the document.ready event. A bit JS oop and you have a fully fledged OOP datastore.
Whenever information is needed, you load it through Ajax / RESTful Webservice and cache it in the AppCache Manager. So you have 2 caches: 1. Browser Cache, possible due to RESTful webservice URL caching, and 2: the JS Cache Manager.
You access all requests to the AppCacheManager which transparently fetches the new data or returns the cached data, so that the client doesnt need to know anything of the caching.
in short:
write a JS CacheManager
don't fetch the whole data at once but in small parts when needed and cache them
define a convenient interface for the cachemanager
Example usage:
linktext
Unobtrusiveness is a very difficult thing in JS and i'd be eager to know something about that, too.
hope that helped.

should I ALSO keep my data in a Javascript data structure? or just in the DOM?

I am very new to Javascript and Jquery, so my apologies for this beginner's question.
In a simple ajax web app, I am creating an HTML page which is mainly a big table. Each row in this table describes an event (party, show, etc). My page not only displays this information but is meant to let the user do a bunch of things with it, in particular to dynamically search and filter the table according to a variety of criteria.
First an abstract beginner's question: in this broad kind of situation (by which I mean that you want your javascript code to run a bunch of operations on the information you retrieve from the webserver) would you use the DOM as a data structure? The ease with which one can search and manipulate it (using Jquery) makes that a possibility. (E.g., "find me table rows describing an event with date column = 2010-01-01 and event type column = 'private party'.) Or would you keep the same information in a traditional Javascript data structure, search/filter/operate on that using plain javascript code and then update the DOM accordingly to display the results to the user?
(As a newbie, I imagine the first, DOM-only approach to be slower while the latter to be take up a good deal of memory. Right? Wrong?)
Assuming the second strategy is reasonable (is it?), then a practical question: can I simply store in my Javascript objects a pointer to the corresponding Jquery object? Eg, can I do
var events = new Array();
// ....
var event3094 = new Event('party','2010-01-01' /*, ... */);
event3094.domElement = $("#correctIdOfTheEventRowInMyTable");
events.push(event3094)
Does this store just a reference (pointer?) to the Jquery object in each Event object or is it creating a new copy of the Jquery object?
I am just wondering "how the pros" do it. : )
Thank you for any advice and insight.
cheers
lara

There are so many ways to do this, but DOM manipulation will almost always be slower than JS manipulation.
To answer your question, anytime you use $(selector) a new jQuery object is created and a match to find the element is performed.
I would recommend two approaches:
FIRST OPTION
Load data in a normal HTML table
Read through the rows, and store just the data (each cell's contents) in an array similar to your code example.
Store a reference to the tr in that object.
Then you can process filter, etc, and only apply changes and searches to the DOM as needed.
SECOND OPTION
Load the page without the table
Load the data as JSON from the server, and generate a table from the data
Store reference to the tr element
Basically, you don't want to perform a $(selector) a 1000 times. The concept is something like this:
var $rows = $("table tr");
// ...
event.domElement = $rows[0]; // Stores reference to existing DOM node, not a new jQuery object.
Then when you need to use jQuery methods on the object, you could use $(yourEvent.domElement) to wrap it in a jQuery wrapper.

Depending on the number of rows you might expect to be shown for most of your users (let's assume it's no more than a few hundred), I myself would probably aim to just keep everything in the DOM table that you're already building. (If you are expecting to be dealing with thousands of rows on one page, you might want to explore a different solution rather than sending it all to the browser.)
There are a few things that you did not mention in your original post. First, how are you creating this table? I imagine using a server-side solution. How easy is that to modify? How much extra work would it be to go through and generate all of your data a second time in a different format, as XML or JSON? Does this add a bunch of complexity on the server-side, only so that you can add more complexity client-side to match? Certain platforms may make this trivial, but is something to consider.
Now, in regards to your alternatives to the DOM:
I agreed and mentioned in a comment above that I don't think JSON would be very optimal "out of the box" for what you want to do. A javascript array is no better. XML is nice in that you can use jquery to easily traverse/filter, but then you still have to deal with your DOM. Sure, you can store references to your DOM elements, but that just seems like a bunch of work up front and then some more work later when matching them up. And without necessarily guaranteeing any major performance boost.
So, to answer your question directly as it is phrased, should you ALSO keep your data in a JavaScript data structure, or just in the DOM: You did mention this was a "simple" ajax web app. My recommendation is to try and keep this simple, then! Your example of how you can so easily use jquery to find rows and cells based on search criteria should be enough to convince you to give this a try!
Best of luck!-Mike

I think you'l find that the DOM method is close to the same speed if I follow your logic right. The second method will require you to manipulate the data, and then apply the changes to the DOM, where the first method allows both operations at the same time.

If youve got alot of data i would for go making objects and just supply it as XML. that way you get most of the same features as operating on the HTML DOM but you dont have a crazy markup structure like a table to navigate through.

I think it's largely a personal preference, but I like to store the objects in JavaScript and then render them to HTML as needed. From my understanding
event3094.domElement = $("#correctIdOfTheEventRowInMyTable");
will store a reference to the jQuery wrapper of your row element (not a copy).
One benefit of storing is JS is that if you have lost of objects (eg 10,000 ) and only render a small fraction of them, the browser will perform a lot better if you're not creating 10,000 * (number of DOM elements per object) elements.
Edit: Oh, and if you can, you might want to send the data from the server to the client as JSON. It's very compact (compared to XML) and in the new browsers it can be parsed quickly and safely using JSON.parse(). For older browsers, just use this library http://www.json.org/js.html . The library will only create a global JSON namespace if the browser doesn't supply it.

One thing to consider is how you need to access your data. If all the data for an element's event is contained in the element (an event operating solely on a table cell for example), then storing in the DOM makes sense.
However, if the calculation of one element depends on data form other elements (a summation of all table cells in a particular column, for example), you may find it difficult to gather up all the data if it's scattered about in DOM elements compared to a single data structure.