I'm trying to learn the basics of IndexedDB by creating a trivial notepad application. I'm having difficulties using an ordered list in this environment.
The feature I'm not sure how to implement is having an ordered list of notes.
I first tried implementing the notepad application in WebSQL, and I found it quite easy to select the notes like this:
select * from notes order by position
And when inserting a note at a specified position, I first did ...
update notes set position = position + 1 where position >= insert_position
... to shift each note to make space for the new note at position insert_position.
But I saw that WebSQL is actually deprecated.
What are the possibilities to achieve such a feature in IndexedDB? I don't fully understand how to create an ordered list in an environment such as IndexedDB since a quick query like the above is not applicable.
As a side note, I know it's possible to store an array in IndexedDB, but then I would just have one record which I'm using each time. I'm rather looking for a way to somehow have an ordered list of all records (each record representing a note), and to be able to update the ordering (like the shifting query above).
Could someone shed some light on the IndexedDB way of an ordered list?
As with many things there are a few ways to crack this nut.
If you were creating an app that orders notes based on creation time, it would be as simple as using an auto-incrementing key (this flag is specified on objectStore creation). Note one would have the id (aka primaryKey) of 1, the second 2 and so forth. This would use the default keyPath, so you could open up a cursor without having to create an index.
To order notes by something that could change, such as modified on time, you'd create an index on that field and be sure to specify it when adding or putting objects. You would open up a cursor with a lower bound of, say 0 (lexicographically ordered keys means this comes before all strings) and leave the upper bound open. You'd then cursor across each row one by one firing onsuccess handlers until you exhaust your cursor and it returns null in event. target.result.
It sounds like you might be looking to have a field such as "position" and order on that. That's totally doable with a regular index and cursor, as above. One note of advice would be to make position field a floating point number rather than an integer as with the former you can update the order without having to alter any other rows (position n = ( ( position 1 + position 2 ) / 2 )).
Related
I have an array of RGB value integers/whole numbers, ranging from 0 to 255. Eight Different lists.
For example one list has 8,349,042 values in it. OR 2,783,014 RGB sets of three.
The core objective is the user selects a pixel in an image.
That pixel's (R,G,B) value is grabbed and searched for within these lists. It exists in one of these lists, as all the lists together contain all the possible RGB combinations (16,777,216)
I'm trying to figure out what's the best way to store and search through these values.
again: these values don't change, they are hard coded lists(see Bullet Point below), with a known range.
The search query would be at minimum 3 times every event which would be every 10-30 seconds or so if the user was spamming.
OR Best case scenario, if the storage and search technique is fast enough I would like to: run the search on every pixel in an image (of maybe 800 x 600 or smaller resolutions) to have more data to play with. If I run into any memory limitations, I plan to work with it and use it as restrictions for my game design.
I used Javascript to generate these lists, going through and assigning each value based on how close it was to a base color.
[maybe unimportant how I made these lists]
I first assigned black and white RGB lists based on hard numeric limitations, then the rest of the RGB values were looped through and assigned to their closest base colors, red, yellow, cyan, green, blue and magenta. If there was a tie in distance I gave it to currently shortest list to try to keep it somewhat balanced. I may try to optimize this later and generate a new list, but not during runtime, just raw data.
I saved the results in hard text, and they are currently stored as a text that I can dump into large array.
At first I was trying to store the data as a JSON file along with my scripts. But I struggled to read the data and save it to an array. I ran into issues with using fetch and async and not being able to have the array where I needed it. Testing with console.log(arr) and getting undefined. I'm guessing because it wasn't loaded yet.
I can just paste it hard coded into the array but it's ginormous and I know there has to be a better way.
Also, hearing about differences in arrays vs sets vs objects
and different searching techniques within them.
Most of them seemed to be more tactful for multi variable arrays like name and age and location databases.
Since my data is all numeric I was thinking it could be a more bit/byte based approach?
I was reading some things on trees and hashs, bit crunching encryption?
Trees seemed nice for quicker searching as I could try to assign each branch of the tree to each R, G and B of the value, but I would also need to figure out how to convert my Giant single list of numbers into that, it also could be just the search type and that may depend on how I store the data.
I also struggled to understand the difference between front end and back end. I believe everything I've tried would be considered front end as I'm only testing my code in a browser.
I was pointed towards Node for backend but got lost in trying to get the console to run things.
I'm willing to give any of these things a try but I don't want to go down a path and find out it can't do what I want, or not optimally enough, like a server burden, or user burden with waiting too long or unable to do things because of user data security, requiring the user to do something more than just load the game, permissives wise.
SO I'm hoping someone can give me suggestions on what I should pursue so I can knuckle down and have a better focus on what I need to learn to be well versed and best tackle problem.
EDIT: Simplified question: In Javascript, I have an array of 2 million (x,y,z) numbers. What's the best way to search that array for a specific (x,y,z) value?
Would it be better to store the data in a different format than an array for constant searches?
I'm not certain I've understood the overall goal here but have a suggestion to consider if you are trying to assign a predetermined value to some (almost random as it is picked as a pixel colour from an image) rgb number set.
I assume the list/dictionary you have made allocates some value to each rgb number set and that it can be regenerated or reformatted if needed.
There are a maximum of 16,777,216 rgba values (256^3). Javascript arrays can have up to 2^32-2 (almost 4.3 billion) elements. Therefore the suggestion is to reformat your dictionary to be a 3-dimensional array where each dimension is indexed 0-255. The array can be declared and assigned as an array literal in a regular js script (text file) like colourDictionary= [[[val0]..[val255]]..[[]..[]],...[[val0]..[val255]]..[[]..[]]]; and each value accessed arithmetically in constant time using the pixel values as colourDictionary[r][g][b]
To be useful without writing lines to cater for missing values, your gaps (you mention a list of 8,349,042, around half the available number combinations) could be filled with the values of nearest neighbours.
Apologies if I've missed the point.
I need to generate several objects on page load with unique names that includes a method that generates the following code in my html:
<a-entity gltf-model="#filename" modify-materials position="0 0 !insert location here!"></a-entity>
The !insert location here! is a z-axis value, which can be positive or negative. As the user moves across the z-axis, I would like it to endlessly generate new objects with location values at intervals of 12 units that are within a range, and unload the objects as they get out of range.
I need to be able to add child objects to each placed model object, and I will need to go back and add (and store) properties to specific instances as well, so including a couple of those in the code to generate at the same time would be nice too.
It needs to be efficient, with as few server requests and user-position checks as possible. Possibly lazy-loading. I am having a breakdown with this seemingly herculean task, I'm sure I'm missing something that makes it easy.
I am willing to use libraries and the like, but prefer the lightest weight option available, even if it means writing more code by hand. (just need to figure out how) PLEASE, YOUR GUIDANCE IS APPRECIATED.
Every time I try and code this up, it gets suuuper bloated, and at some point the code just breaks down and I get lost.
can someone please tell me the benefits of using UintArrays and also why 8,16, and 32 bit arrays are used?
As if we just want to store 1's and 0's then we can just use the 8 bit one whats the use of others.
Also can you provide a short description of the flow of GROUP functionality in crossfilter.
Also is there any source where i can easily understand the entire flow of crossfilter's internal code.
Many thanks.
IIRC, the Uint arrays are just used for filters. The width of the array is determined by the number of dimensions you have and each bit of each row of the array is flipped to 1 if the record at that position in the array is excluded by the filter on the dimension corresponding to the bit position. The fact that the width of the array needs to be equal to or greater than the number of dimensions is the source of Crossfilter's 32-dimension cap. There are a few advantages, again, if I remember correctly:
The array takes up a lot less space in memory
Applying a new filter becomes a matter of bit operations and doesn't require updating the full integer
Checking if a given record should be displayed under the current filters is just a matter of comparing the filter array to 0 at the index corresponding to the record
This is currently just conseptual, but it bothers my brain.
If I have a list of items - in my mind it's a HTML/JS implementation, but that's just because I'm a visual thinker ;)
I want to use drag and drop to sort this list, with the aim of storing the new order when I'm done. Is there a way to do this without numbering the items, and then updating the number of the dropped item + every single item that follows it? Isn't that very inefficient?
As far as performance goes, changing the numbering of the elements is nothing next to actually rendering the transition (while you're dragging an element), so no, it's not inefficient.
You can use a doubly linked list in order to minmize the amount of operations needed to change the order of the collection.
I would recommend using a JavaScript framework to do the job.
KnockoutJS should fit your needs (from the website):
Knockout is a JavaScript library that helps you to create rich, responsive display and editor user interfaces with a clean underlying data model. Any time you have sections of UI that update dynamically (e.g., changing depending on the user’s actions or when an external data source changes), KO can help you implement it more simply and maintainably.
If you left gaps in the numbers, you could probably do it efficiently -- for instance, let the first element be 10, the second be 20, the third 30, etc. Then when you drag something in front of the second one and after the first, call it 15 (or something).
Rather than incrementing numbers every time, you'd only have to do it if you ran out of space. At the end, you could just order the objects by lowest number.
Not including any code here, because this is a conceptual question.
You have to have a number for every element in the list anyway, since they have to have a Total Ordering among themselves to be represented as a list.
If you don't expect the number of elements in the list to be large, a Bubble Sort should work very well for you.
Currently for "group" management you can click the name of the group in a list of available groups and it will take you to a page with two side by side multi-select list boxes. Between the two list boxes are Add and Remove buttons. You select all the "users" from the left list and click 'Add' and they will appear in the right list, and vice versa. This works fairly well for a small amount of data.
The problem lies when you start having thousands of users. Not only is it difficult and time consuming to search through (despite having a 'filter' at the top that will narrow results based on a string), but you will eventually reach a point where your computer's power and the number of list items apex and the whole browser starts to lag horrendously.
Is there a better interface idea for managing this? Or are there any well known tricks to make it perform better and/or be easier to use when there are many 'items' in the lists?
Implement an Ajax function that hooks on keydown and checks the characters the user has typed into the search/filter box so far (server-side). When the search results drop below 50, push those to the browser for display.
Alternatively, you can use a jQuery UI Autocomplete plugin, and set the minimum number of characters to 3 to trigger the search. This will limit the number of list items that are pushed to the browser.
I would get away from using the native list box in your browser and implement a solution in HTML/CSS using lists or tables (depending on your needs). Then you can use JavaScript and AJAX to pull only the subset of data you need. Watch the user's actions and pull the next 50 records before they actually get to them. That will give the illusion of all of the records being loaded at runtime.
The iPhone does this kind of thing to preserve memory for it's TableViews. I would take that idea and apply it to your case.
I'd say you hit the nail on the head with the word 'filter'. I'm not the hugest fan of parallel multi-selects like what you are describing, but that is almost beside the point, whatever UX element you use, you are going to run into a problem given thousands of items. Thus, filtering. Filtering with a search string is a fine solution, but I suspect searching by name is not the fastest way to get to the users that the admin here wants. What else do you know about the users? How are they grouped.
For example, if these users were students at a highschool, we would know some meta-data about them: What grade are they in? How old are they? What stream of studies are they in? What is their GPA? ... providing filtering on these pieces of metadata is one way of limiting the number of students available at a time. If you have too many to start with, and it is causing performance problems, consider just limiting them, have a button to load more, and only show 100 at a time.
Update: the last point here is essentially what Jesse is proposing below.