indexeddb partial key search get next - javascript

Does the indexeddb CursorWithValue store what the next or prev record will be BEFORE I call cursor.continue()? Can I look at the IDBCursorWithValue object and then store the pointer to the next record?
Is it possible to get the first record via a partial key, then get the next record ONLY when the user clicks for the next record without buffering a collection of the records in an array?
I understand I can use cursor.continue() to get all the matching records and store in an array. I also understand that being asynchronous, if I just take the first matching record, and terminate my onsuccess function that call to the db is terminated and I'm fairly sure that I then lose the ability to link to the next record.
The following works and I can get one or all matching records of the partial key. With the \uffff I basically get matching alpha and all greater records.
storeGet = indexTITLE.openCursor(IDBKeyRange.bound(x.value, x.value, '\uffff'), 'next');
This is all new to me, perhaps I'm looking at this all wrong. Any advice is appreciated. I've been reading every thread on here and github that I can, hoping someone else already was doing this with indexeddb.

Let me try and restate the problem:
You've iterated a cursor part-way through a range. Now you want to stop and wait for user input before continuing. But the transaction will close, so you can't just continue on the click. What do you do instead?
First off: great question! This is tricky. You have a handful of different options.
In the simplest case, you have a unique index (or an object store) so there are no duplicate keys.
var currentKey = undefined;
// assumes you open a transaction and pass in the index to query
function getNextRecord(index, callback) {
var range;
if (currentKey === undefined) {
range = null; // unbounded
} else {
range = IDBKeyRange.lowerBound(currentKey, true); // exclusive
}
var request = index.openCursor(range);
request.onsuccess = function(e) {
var cursor = request.result;
if (!cursor) {
// no records found/hit end of range
callback();
return;
}
// yay, found a record. remember the key for next time
currentKey = cursor.key;
callback(cursor.value);
};
}
If you have a non-unique index it is more tricky since you need to store the index key and primary key, and there's no way to open the cursor right at that position. (See the feature request: https://github.com/w3c/IndexedDB/issues/14) So you need to advance the cursor just past the previously seen key/primaryKey position:
var currentKey = undefined, primaryKey = undefined;
// assumes you open a transaction and pass in the index to query
function getNextRecord(index, callback) {
var range;
if (currentKey === undefined) {
range = null; // unbounded
} else {
range = IDBKeyRange.lowerBound(currentKey, true); // exclusive
}
var request = index.openCursor(range);
request.onsuccess = function(e) {
var cursor = request.result;
if (!cursor) {
// no records found/hit end of range
callback();
return;
}
if (indexedDB.cmp(cursor.key, currentKey) === 0 &&
indexedDB.cmp(cursor.primaryKey, primaryKey) <= 0) {
// walk over duplicates until we are past where we were last time
cursor.continue();
return;
}
// yay, found a record. remember the keys for next time
currentKey = cursor.key;
primaryKey = cursor.primaryKey;
callback(cursor.value);
};
}
I'm assuming there's not an upper bound, e.g. we want all records in the index. You can replace the initialization of range as appropriate.

Related

How to create a function to count items from a set and store counts in an array parallel to one containing related items?

I am having trouble completing one of the last assignments in my semester-long high school-level programming class. I have been assigned to create a JavaScript program which counts the amount of time different ZIP codes appear in a set and output parallel arrays containing the zip codes and their counts. I am having difficulty getting the values to output. I believe that the respective zips and counts aren't being entered into their arrays at all.
I'm not looking for an original solution to the problem. I'd just like someone to tell me why my code isn't working, and possibly what I can change in my code specifically to fix it.
Usually I would never ask for help like this. I actually took the class last semester and now that I'm at the end of the year I have the option of completing it to earn college credit. I have never been the best at working with functions, and that remains true now. In the code below are all the moving parts I'm allowed to work with. I know it looks messy and rudimentary, but it's all I know. I'd appreciate it if any answers use only the sorts of things I used in my code. Another note, I am required to use functions for 'all identifiable processes', but I'm pretty sure my instructor only cares about the final product, so I'm not sure that the functions really matter, even if they could help.
var records = openZipCodeStudyRecordSet(),
uniqueZips = [],
zipCounts = [],
output = "";
function project62Part1() {
table = document.getElementById("outputTable");
function countZips(zip) {
var currentZip,
count;
while (records.readNextRecord()) {
currentZip = records.getSampleZipCode();
if (zip === currentZip) {
count++;
}
}
return count;
}
function processZip(zip) {
var currentZip;
while (records.readNextRecord()) {
currentZip = records.getSampleZipCode();
for (i = 0; i < uniqueZips.length; i++) {
if (uniqueZips[i] === "") {
uniqueZips.push(currentZip);
zipCounts[i] = countZips(currentZip);
break;
}
if (zip !== uniqueZip[i]) {
uniqueZips.push(currentZip);
zipCounts[i] = countZips(currentZip);
}
}
}
}
function createOutput(string) {
for (i = 0; i < uniqueZips.length; i++) {
string += "<tr><td>" + uniqueZips[i] + "</td><td>" + zipCounts[i] +
"</td></tr>";
}
return string;
}
processZip();
output = createOutput(output);
table.innerHTML = "<tr><td>Zip Code</td><td>Count</td></tr>" + output;
}
The output is supposed to be additional rows of zips and counts added to a table that is already set up on the page. There are no important technical errors in the code.
This is to be accomplished through the function processZip, which is meant to add respective zip and count into table rows. However, it appears as though the zip and count arrays its getting info from haven't had anything put into them by the other functions. I don't know if it is because of error in calling the functions, or what's in the functions themselves.
The HTML page this is connected to calls the function project62Part1().
That code is kind of all over the place but here's the logic you ideally want to follow:
Loop over each record in your table (outer loop) to get the zip code.
Declare an 'isFound' variable and set it to false
For each iteration of the outer loop, loop over your entire array of zip codes (inner loop).
3a. If you get a match, set isFound to true, increment your zipcode counter += 1 on the same index (since they're parallel arrays)
3b. If, at the end of your inner loop, isFound is still false, add the zipcode to your array of zip codes, and add a new array element to your zip code counters setting it to 1.
Since your zip code array and your zip code counter are parallel arrays to each other, when isFound is false, you are creating entries in both arrays, keeping them parallel to each other.
If, on 3a isFound is true, you are on the index of the zip code array that the zip code belongs to, so it should be the same index for your counter array.
In your current process zip function, the first condition will never be true, because starting out, your array size is 0 and after you start populating that array, you will never have an empty string (unless, of course, the zip code itself was an empty string)
The second if statement you have that checks if zip !== uniqueZip[i] - you are only checking that current value of uniqueZips and ignoring every other value in the array, so you will almost always have the second condition as true
I've been playing with the newer JavaScript language and syntax and your item was a good candidate for me to try out.
I did approach the code a little differently such as making the use of a Set for the unique values. Saves on code by not having to check and see if the value exists because the Set will never allow duplicate values in.
var uniqueZips = new Set();
const zipcodes = [21060, 22422, 25541, 43211, 21060, 22422, 22422, 43211, 43211, 43211];
function project62Part1() {
function processZipCodes() {
for(let index in zipcodes){
// We add every value because a SET will only allow you to add it once.
uniqueZips.add(zipcodes[index]);
}
}
// Structure our zipcode data information
function organizeZipCodeData() {
let response = {data:[]};
uniqueZips.forEach(function(zip) {
response.data.push( { 'zipcode':zip, 'appears': countZipAppearances(zip) })
});
return response;
}
function countZipAppearances(zip) {
// Default to zero even though you never expect an undefined
let count = 0;
zipcodes.forEach(function(zval) {
if (zip === zval) {
count++;
}
});
return count;
}
function showZipcodeInformation(data){
for (var index in data) {
if (data.hasOwnProperty(index)) {
var entry = [data[index]][0];
console.log(entry.zipcode, entry.appears);
}
}
}
// UI CONTENT: Construct the UI view from the data
function generateHtmlView(data){
let htmlview = "<table><tr><td>Zip Code</td><td>Count</td></tr>";
for (var index in data) {
if (data.hasOwnProperty(index)) {
var entry = [data[index]][0];
htmlview+="<tr><td>"+entry.zipcode+"</td><td>"+entry.appears+"</td></tr>";
}
}
htmlview += "</table>";
console.log(htmlview);
return htmlview;
}
// //////////////////////////////////////////////////////
// Call to gather the zipcodes
processZipCodes();
// Call to organize the zipcode data
let output = organizeZipCodeData();
// See what we have in the organized data
showZipcodeInformation(output.data);
// See what we have in the html content
generateHtmlView(output.data);
}
// Initiate the process
project62Part1();

localStorage array is being overwritten after refresh

So my localStorage values in array are being overwritten after the refresh. While in session, I can change values on front end and array in localStorage constantly will be updated or if more inputs changed, then more values to array added. If for example I had 3 inputs changed and all their values was added to localStorage array in the session then after webpage refresh if I will edit one of the inputs again, all the arrays in localStorage will be deleted and overwritten by the new's session update of input. I'm trying to figure out this for a long time, but can't figure out how to make it work so that after the refresh all the input values would be stored in the old array and not overwritten.
var presetsArr = [];
if (!localStorage.getItem("presetsArr") || localStorage.getItem("presetsArr").length < 0) {
init();
} else {
initIfLocalStorage();
}
function initIfLocalStorage () {
presetsArr = JSON.parse(localStorage.getItem('presetsArr'));
for (var i = 0; i < presetsArr.length; i++) {
if (presetsArr[i].freq) {
osc.frequency.value = presetsArr[i].freq;
freqSlider.value = presetsArr[i].freq;
freqDisplay.innerHTML = freqSlider.value;
}
if (presetsArr[i].detune) {
osc.detune.value = presetsArr[i].detune;
detuneSlider.value = presetsArr[i].detune;
detuneDisplay.innerHTML = detuneSlider.value;
}
if (presetsArr[i].gain) {
volume.gain.value = presetsArr[i].gain;
}
}
freqSlider.addEventListener("click", function(e) {
osc.frequency.value = this.value;
freqDisplay.innerHTML = this.value;
bookmark["freq"] = osc.frequency.value;
presetsArr.push(bookmark);
presetsArr = localStorage.setItem("presetsArr", JSON.stringify(presetsArr)) || [];
})
detuneSlider.addEventListener("click", function(e) {
osc.detune.value = this.value;
detune.innerHTML = this.value;
bookmark["detune"] = osc.detune.value;
presetsArr.push(bookmark);
presetsArr = localStorage.setItem("presetsArr", JSON.stringify(presetsArr)) || [];
})
Here is what I mean: First session, I changed both frequency and detune values, they got stored in the array. http://imgur.com/0GIeQPj
Now after refresh in the second session, I changed the detune value again and the whole localStorage array got overwritten. Now frequency key is gone and only detune left. So after refresh next time I want to play my oscillator, I won't get any sound out of it, because frequency value in the array was removed.
http://imgur.com/crTywnN
For anyone running in the same kind of problem, here is the solution I came across.
So like Patrick Evans said, I made a mistake by assigning .setItem() to a variable, even though .setItem() doesn't return any value. So I just always create a new empty array when I click after the resfresh (in the new session). But that is not enough. Now I was running into a problem, where I was trying to push values into non-existing array and getting null error. I changed my click handler to:
presetsArr = JSON.parse(localStorage.getItem('presetsArr')) || [];
presetsArr.push(bookmark);
localStorage.setItem("presetsArr", JSON.stringify(presetsArr));
|| [] helped me to get rid of Null error. Even though I created en empty array at the top of my program, for some reason inside the click hander I couldn't push anything to it, so this little shortcut helped me to check if there is an array to push to and if there is not, then create one before pushing.
Now I can refresh the webpage and continue changing input values without overwriting and recreating localStorage array.

Google Script to remove duplicates exceeds processing time

I have a Google Sheet that has 10k+ rows of data. While it should be rare, there could be instances of duplicate data being entered into the tab, and I have written a script to search for and remove those duplicates. For a while, this script has been running nicely and doing exactly what I expected it to do. But now that the tab has grown to over 10k rows, the script is exceeding the 6 minute time limit.
I've based this function on this tutorial.
// remove duplicates on Ship Details Complete
function duplicateShipDetailsComplete() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sourceSheet = ss.getSheetByName("Shipment Details Complete");
var sourceRange = sourceSheet.getRange(2, 1, sourceSheet.getLastRow(), 16)
var sourceData = sourceRange.getValues();
var keepData = new Array();
var deleteCount = 0;
for(i in sourceData) { // look for duplicates
var row = sourceData[i];
var duplicate = false; // initialize as not a duplicate
for(j in keepData) { // compare the current row in data to the rows in newData
if(row[2] == keepData[j][2] // duplicate Partner Invoice?
&& row[4] == keepData[j][4] // duplicate vPO?
&& row[5] == keepData[j][5] // duplicate SKU?
&& row[7] == keepData[j][7]) { // duplicate qty?
duplicate = true; // only if ALL criteria are duplicate, set row as a duplicate
}
}
if(!duplicate) { // If the row is NOT a duplicate
keepData.push(row); // add to newData
} else {
deleteCount++; // keep track of duplicates being deleted
}
}
sourceRange.clear();
sourceSheet.getRange(2, 1, keepData.length, keepData[0].length).setValues(keepData); // paste the keepData into the Working sheet
return deleteCount;
}
I've thought about breaking it up into pieces; process 1/3 of the data in each of 3 different calls. But this is actually being called from a different function that emails the returned deleteCount value, if it's greater than 0.
// Nightly email after checking for duplicates in Ship Details Complete
function sendEmailShipDetails() {
var deleted = duplicateShipDetailsComplete();
var update = parseFloat(SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Dept/Class").getRange(1,6).getValue()).toFixed(2);
if(deleted > 0) {
MailApp.sendEmail(
"me#myoffice.com",
"Shipment Details Cleaned Up",
deleted + " Shipment Detail line(s) were deleted from Complete as duplicates.\n" +
"Updated Value: " + update + "\n"
);
}
}
Even without that email function, it's exceeding the limit when I call duplicateShipDetailsComplete() directly. I suppose I could write three different functions (first 1/3, second 1/3, third 1/3) and update a cell somewhere with the results for each, and then call the email function separately to get that value. I'd feel a little better about that if I could write 1 function and pass parameters to it, but this is all coming from a Time Based Trigger, and you can't pass parms from those. But before I started to do that, I thought I'd check to see if someone had other suggestions on how I could make the existing code more efficient. Or, see if someone had totally different ideas on how I can do this.
Thanks

Store a table row index as an array index

There a simple function:
selected_row = []; // global scope
function toggleRowNumber(rowIndex) {
if(selected_row[rowIndex]) selected_row.splice(rowIndex, 1);
else selected_row[rowIndex] = 1;
}
usage
toggleRowNumber(50000); // click the row - write the index
toggleRowNumber(50000); // click the row again - remove the inxed
alert(selected_row.length);
50001
OK
Delightful feature!
So is there a way to direct write|read an index without any searchin/looping? And without this huge feat as decribed above.
Thanks.
If I understoold correctly, you want to store and index where you can check/set whether an item is selected or not. If that is the case, you are looking for a "key - value" data structure. Then, why not use a map?
var selected_row = {};
function toggleRowNumber(rowIndex) {
if(selected_row[rowIndex]) selected_row[rowIndex] = 0; //or = undefined;
else selected_row[rowIndex] = 1;
}
That is better because hash map will save you time and space.
Space becuase you are not storing hundreds of 'undefined' values in a vector.
Time because, hash function used to access elements is pretended to hit the right position in many cases.

Indexeddb : How to limit number of objects returned?

I'm using a cursor with a lower bound range query. I can't find a way to limit the number of objects returned, similar to a "LIMIT n" clause in a databse.
var keyRange = IDBKeyRange.lowerBound('');
Does it not exist ?
As you're iterating through the results, you can stop at any time. Something like this should work:
var results = [];
var limit = 20;
var i = 0;
objectStore.openCursor().onsuccess = function (event) {
var cursor = event.target.result;
if (cursor && i < limit) {
results.push(cursor.value);
i += 1;
cursor.continue();
}
else {
// Do something with results, which has at most 20 entries
console.log(results);
}
};
Also, in the special case where you are selecting based on a key that is made up of sequential numbers, you could use a keyRange to explicitly return only a certain range. But that's generally not the case.
An important fix its replace:
if (cursor && i < limit) {
for
if (cursor && results.length < limit) {
Remember that its an async call, and it could be adding elements on "results" at the same time and "i" would not have the proper value.
You can even search by range in indexedDB with IDBKeyRange.bound function
IDBKeyRange.bound(keyValueFrom, keyValueTo, false, true);
You can set the start value, end value and if you should include the start and end value in the returned items. In my case i want the first value of the range, however i want to exclude the last value.
For details of IDBKeyRange please visit the W3C IndexedDB page

Categories