We Keep Coding

Is there a way to get a list of all global HTML attributes programmatically?

The list of global HTML attributes is quite large if you include every possible event listener, plus it's subject to change as browsers evolve.
Is it possible to use JavaScript to generate a comprehensive list of these attributes programmatically without explicitly defining them? (e.g. from a built-in class or element instance)

You can get most of them by looking at HTMLElement.prototype, or Element.prototype:
console.log(Object.keys(HTMLElement.prototype));
console.log(Object.keys(Element.prototype));

Attribute / Property names on custom elements

I'm part of a team that develops a custom element library for a big customer (according to custom element spec v1). We've developed like 30 components so far, half of those being autonomous custom elements, the other half being customized built-ins.
We're currently in the phase of refactoring and unifying the API of those components. Almost all of these components have custom attributes / properties.
With standard HTML elements the convention is to use data-attributes which are reflected to and from the dataset of the element.
I'm aware of the fact that autonomous custom elements allow for free attribute / property naming as long as the name doesn't conflict with what HTMLElement already has.
The idea behind data- attributes as I understand is to make sure that no libary will begin implementing a custom attribute which might conflict with future versions of the HTML standard.
The same, though, should be valid for property names. Imagine the HTML standard would get a new universal attribute, let's abstractly call it attr. Of course that would also reflect to a same-name property on the HTMLElement prototype.
My questions are:
What is the suggested approach to evade future conflicts here? Is it to always only use data--attributes for custom attribute implementations, and thus, avoid the conflict by working with the dataset instead of properties directly attached to the element?
If so, how would you then implement "boolean" data-attributes (where the attribute value doesn't matter, the state is implemented via absence/presence of that attribute)?
Is it possible to define own getters and setters for properties in the dataset of an element?
Please ask for clarification if anything is unclear. I tried to make the question as concise as possible.

I don't worry about future-proofing my custom element attributes or properties against what may become a predefined attribute or property of HTMLElement. If the spec changes, then I will deal with it then. But to try to guess everything that might be used in the future is not possible nor worth my time.
I avoid using data- attributes for custom elements. The dataset values are only strings and do not convert well. For example if I do this:
el.dataset.dog = {woof:10};
then my tag get's an attribute like this: data-dog="[object Object]" and this console.log(el.dataset.dog) displays "[object Object]". That defeats the purpose of many properties that may need to deal with objects.
I only create attributes for things that must be attributes. If I must allow the HTML to define default values or if I need to create a CSS attribute selector then I will create an attribute. But only under these two conditions.
I create properties (getter/setter) or functions for everything I need. It is easier to pass in data of any format into a property or function then it is to convert it into a string to pass in through an attribute.
I have even taken over one or more of the existing HTMLElement properties and attributes. This is rare, but sometimes I need to do something with the value or I just don't want the default operation to happen. Again, this is rare.

Prefer id attribute or data attribute in page with many elements?

I'm creating a HTML page with many, many "widgets" (can be in the magnitude of 1000, many not visible by display:none which get's modified dynamically by user interaction).
Each widget must carry some information regarding it.
At the moment I'm building that DOM piece by piece and add a jQuery .data() to each widget. But this building of little blocks is too slow (lots of time spent in Parse HTML), so I want to switch to a new approach where I first build a string of all widgets and then add that to the DOM with innerHTML at one go. But this doesn't allow me to use the .data() during the build anymore.
As all widgets have a unique ID anyway (currently only stored in the .data()...) I could think of two obvious solutions:
Give every widget an id attribute with the widget ID as content
Give every widget an data-widgetid attribute with the ID as content
(Parallel to that I'd have a big JavaScript hash that links the widget-id to the per widget data.)
What approach should I prefer?
Performance and resources is an issue (also on mobile devices).
By the id attribute I'm interfering with the DOM, so I don't know if that could have any negative effects. (E.g. it could be that browsers are starting to build an extra look up structure for that internally that I could avoid otherwise. I'm not a browser developer, so I don't know if that is the case though)

Use ID. If you want to protect the ID namespace of the DOM, use a prefix. For example, , where "123" is the unique ID. If your application is driven by a database, the primary key conveniently provides non-colliding ID values. Otherwise use a client-side counter to generate unique IDs.
Accessing the element by ID is much faster via document.getElementById. Modern browsers allow this kind of access really fast. The internal storage is also more efficient comparing to a custom attribute. To access a custom attribute, you'll have to traverse all the matching nodes and compare them with the values of the custom attribute. Some old IE also don't support custom attributes.

If you don't need to get those elements by their id, and you will only use it to associate data with the elements, I would recommend not using id.
Since ids are unique, browsers build a hash table to retrieve elements by id so fast.
Therefore, adding ids to lots of elements that don't need them will increase the size of the hash table unnecessarily.
In average, a search in a hash table is O(1), so the size doesn't matter. However, on the worst case, it's O(n), so better have a small table if you can.
This way, if you use document.getElementById to get other elements, it may be faster.

using hidden elements in pages to store data

I need to store data in html documents that are associated with elements, and that I can get to with javascript. I think I want to avoid arbitrary attributes on elements, since after reading various posts here I don't trust them. I can't use id or class, since I those are used for other things and I don't want to mess with them (my project has to work on wide varieties of html so I can't make any assumptions as to the design of the class and id structure). Another thing I need is for whatever I do, it needs to survive round-tripping through innerHTML (of a parent or ancestor element of the element I need to tag with data), with data intact.
I have considered various hidden elements (including script tags and html comments), which I insert into the document right before the element I need to tag with data. Currently my favorite is hidden form elements (i.e. input with type "hidden"). I can stick any data I want into the "value" of the element, and I can find all such elements easily enough with getElementsByTagName(). Also importantly, it doesn't seem to affect the layout of the page.
I'm curious about the ramifications of this, and if anyone can think of any any problems with it. I need to be able to put them anywhere in body of the page, before any element. For instance I might need them associated with an option element (a child of a select element), so that rules out using a hidden div, since putting one of those inside a select element is illegal. I don't think they will affect form behavior, since they don't have a name attribute (I can avoid using id's if that helps).
Any thoughts on this, or suggestions for other ways of accomplishing the same that meets my needs?

If this is going to be data for JavaScript's purposes, why even try to put it in the HTML at all? Here's a simple example of what I mean
<script type="text/javascript">
var nodeData = {
foo: {/* Data to associate with div#foo */}
, bar: {/* Data to associate with div#bar */}
};
</script>
<div id="foo">Foo!</div>
<div id="bar">Bar!</div>
It's simple and elegant since the principle of IDs being unique per document nicely matches with javascript's dictionaries requiring unique keys per entry.

Is there a good way to attach JavaScript objects to HTML elements?

I want to associate a JavaScript object with an HTML element. Is there a simple way to do this?
I noticed HTML DOM defines a setAttribute method and it looks like this is defined for arbitrary attribute name. However this can only set string values. (You can of course use this to store keys into a dictionary.)
Specifics (though I'm mostly interested in the general question):
Specifically, I have HTML elements representing nodes in a tree and I'm trying to enable drag-and-drop, but the jQuery drop event will only give me the elements being dragged and dropped.
The normal pattern for getting information to event handlers seems to be to create the HTML elements at the same time as you are creating JavaScript objects and then to define event handlers by closing over these JavaScript objects - however this doesn't work too well in this case (I could have a global object that gets populated when a drag begins... but this feels slightly nasty).

JavaScript objects can have arbitrary properties assigned to them, there's nothing special you have to do to allow it. This includes DOM elements; although this behaviour is not part of the DOM standard, it has been the case going back to the very first versions of JavaScript and is totally reliable.
var div= document.getElementById('nav');
div.potato= ['lemons', 3];

If you're already using jQuery, you can use its data() method for this. This allows you to assign complex objects to the element if you want or you can leverage that method to hold a reference to an object (or some other data) at the very least.
It's worth noting that, under the hood, jQuery implements data() in precisely the way that bobince described in his answer, so you always use that directly, whether or not you're using jQuery.