I currently save a bunch of objects (thousands) into the chrome.storage.local and then when on a specific web page checking whether specific IDs on the web page are in fact saved in local storage.
Here's a pseudo code
Bakcground script:
var storage = chrome.storage.local;
var json = '[{"kek1": {"aaa": "aaaValue", "bbb": "bbbValue", "ccc": "cccValue"}},{"kek2": {"ddd": "dddValue", "eee": "eeeValue", "fff": "fffValue"}}]';
var jsonParsed = JSON.parse(json);
jsonParsed.forEach(function(object) {
storage.set(object);
});
Content script (when on a specific page):
ids.forEach(function(id) {
storage.get(id, function(result){
if(!isEmpty(result)) {
//we found it, nice, now flag it as found
}
});
});
function isEmpty(obj) {
for(var key in obj) {
if(obj.hasOwnProperty(key))
return false;
}
return true;
}
Which is easy and nice since I only have to do storage.get(id, ...
Unfortunately, I save a lot of stuff in storage, some of it I need to be removing periodically, which then becomes a hustle since I have to loop through all the objects and determining whether that particular object needs to be removed or it needs to remain.
So i decided I would do like these "parent object". Ie one object for settings, containing an array of objects with different settings the user would save. One object for the stuff that needs to be removed, containing an array objects. Etc
Like so - all relevant info that I want to remove periodically will be under one key "test" (temp name):
var json = '{"test":[{"kek1": {"aaa": "aaaValue", "bbb": "bbbValue", "ccc": "cccValue"}},{"kek2": {"ddd": "dddValue", "eee": "eeeValue", "fff": "fffValue"}}]}';
I know how to access the nested objects and their values:
var jsonParsed = JSON.parse(json);
jsonParsed.test[0].kek1.aaa
But I don't know how I would easily check for the keys saved in the storage since I would have to specify the "element number" ([i]).
Do I just do a for loop itterating over the array like so?
for (i = 0; i < jsonParsed.test.length; i++) {
var getKey = Object.keys(jsonParsed.test[i]);
if (getKey[0] == 'theKeyImLookingFor') {
//do stuff
}
}
To me that feels like non ideal solution since the for loop would have to run for each of the ids on the page and there could sometimes be close to 4000 of them. (4000 for loops back to back)
Is it a good idea to save a single object holding an array of thousands of other objects?
Am I doing it wrong or is this the way to go?
But I don't know how I would easily check for the keys saved in the storage
Use the standard Array methods like find or findIndex:
const i = arrayOfObjects.findIndex(o => 'someKey' in o);
Is it a good idea to save a single object holding an array of thousands of other objects?
It's a bad idea performance-wise.
What you probably need here is an additional value in the storage that would contain an array with ids of other values in the storage that need to be processed in some fashion e.g. expired/removed. It's basically like a database index so you would update it every time when writing an individual object. Since it contains only the ids, updating it is cheaper than rewriting the entire data.
Also, instead of performing lots of calls to the API, do just a single call:
// writing
chrome.storage.local.set(Object.assign({}, ...arrayOfObjects));
// reading
chrome.storage.local.get(arrayOfIds, data => {
for (const id of arrayOfIds) {
const value = data[id];
if (value !== undefined) {
// ok
}
}
});
So this follows on from my previous question:
knockout js, add additional elements to an array
Basically I have an app where a user fills in certain data, clicks next and this is added to an array. However, what I'd like to do is add some items into the array before the user even begins using the app (these items I get from a database). The idea being that at the start they can view each item in the array and then choose and an item and edit this item. I've got a feeling I'm missing something blindingly obvious but I cannot seem to figure it out
Knockout observable arrays have equivalent functions to native JavaScript arrays. See: http://knockoutjs.com/documentation/observableArrays.html
So you need just to use arr.pop(item) or arr.push(item).
In case you need to replace all items and want to avoid multiple events to raise, use observableArray.valueWillMutate() and valueHasMutated() functions. See sample where I do swap the entire array:
ko.observableArray.fn.replaceWith = function (valuesToPush) {
// NOTE: base on - ko.observableArray.fn.pushAll
var underlyingArray = this();
var oldItemcount = underlyingArray.length;
this.valueWillMutate();
// adding new items to obs. array
ko.utils.arrayPushAll(underlyingArray, valuesToPush);
// removing old items (using KO observablearray fnc.)
if (oldItemcount > 0)
this.removeAll(underlyingArray.slice(0, oldItemcount));
this.valueHasMutated();
return this; //optional
};
I'm just getting started with JavaScript objects. I'm trying to store catalog inventory by id and locations. I think a single object would look something like this:
var item = {
id: number,
locations: ["location1", "location2"]
};
I've started to read a bit about it but am still trying to wrap my head around it. Not sure what is the fastest way add new items to a list with a location, add a new location to an existing item, all while checking for dupes. Performance of getting the locations later isn't as critical. This is part of a process that is running thousands of checks to eventually get items by id and location, so performance is key.
Final question, I'm not even sure if it's possible to store this in local storage. From another similar question, I'm not sure.
Using lodash, something like this should work to determine if an item id exists and append either a new item to the array, or just add a new location:
var item = [{
id: 1,
locations: ["location1", "location2"]
},{
id: 2,
locations: ["location2", "location4"]
}];
function findItem(id){
return _.findIndex(item, function(chr) {
return chr.id == id;
});
}
function addItem(id,locations) {
var position = findItem(id);
if (position<0) {
item.push({
id: id,
locations: locations
})
} else {
item[position].locations = _.uniq(item[position].locations.concat(locations));
}
}
addItem(2,['location292']);
addItem(3,['location23']);
console.log(item);
What it basically does is to search the array of objects (item) for an id as the one we are passing to the addItem() function, if it is found we add the new locations array to the existing item, if not it's creating a new object with a new id and location.
You've asked a question that contains some tradeoffs:
The simplest and fastest way to retrieve a list of locations is to store them in an array.
The fastest way to check something for a duplicates is not an array, but rather a map object that maintains an index of the key.
So, you'd have to discuss more about which set of tradeoffs you want. Do you want to optimize for performance of adding a non-duplicate or optimize for performance of retrieving the list of locations. Pick one or the other.
As for localStorage, you can store any string in LocalStorage and you can convert simply non-reference objects to a string with JSON.stringify(), so yes this type of structure can be stored in LocalStorage.
For example, if you want to use the array for optimized retrieval, then you can check for duplicates like this before adding:
function addLocation(item, newLocation) {
if (item.locations.indexOf(newLocation) === -1) {
item.locations.push(newLocation);
}
}
Also, you can store an array of items in LocalStorage like this:
localStorage.setItem("someKey", JSON.stringify(arrayOfItems));
And, then some time later, you can retrieve it like this:
var arrayOfItems = JSON.parse(localStorage.getItem("someKey"));
I'm trying to test out Firebase to allow users to post comments using push. I want to display the data I retrieve with the following;
fbl.child('sell').limit(20).on("value", function(fbdata) {
// handle data display here
}
The problem is the data is returned in order of oldest to newest - I want it in reversed order. Can Firebase do this?
Since this answer was written, Firebase has added a feature that allows ordering by any child or by value. So there are now four ways to order data: by key, by value, by priority, or by the value of any named child. See this blog post that introduces the new ordering capabilities.
The basic approaches remain the same though:
1. Add a child property with the inverted timestamp and then order on that.
2. Read the children in ascending order and then invert them on the client.
Firebase supports retrieving child nodes of a collection in two ways:
by name
by priority
What you're getting now is by name, which happens to be chronological. That's no coincidence btw: when you push an item into a collection, the name is generated to ensure the children are ordered in this way. To quote the Firebase documentation for push:
The unique name generated by push() is prefixed with a client-generated timestamp so that the resulting list will be chronologically-sorted.
The Firebase guide on ordered data has this to say on the topic:
How Data is Ordered
By default, children at a Firebase node are sorted lexicographically by name. Using push() can generate child names that naturally sort chronologically, but many applications require their data to be sorted in other ways. Firebase lets developers specify the ordering of items in a list by specifying a custom priority for each item.
The simplest way to get the behavior you want is to also specify an always-decreasing priority when you add the item:
var ref = new Firebase('https://your.firebaseio.com/sell');
var item = ref.push();
item.setWithPriority(yourObject, 0 - Date.now());
Update
You'll also have to retrieve the children differently:
fbl.child('sell').startAt().limitToLast(20).on('child_added', function(fbdata) {
console.log(fbdata.exportVal());
})
In my test using on('child_added' ensures that the last few children added are returned in reverse chronological order. Using on('value' on the other hand, returns them in the order of their name.
Be sure to read the section "Reading ordered data", which explains the usage of the child_* events to retrieve (ordered) children.
A bin to demonstrate this: http://jsbin.com/nonawe/3/watch?js,console
Since firebase 2.0.x you can use limitLast() to achieve that:
fbl.child('sell').orderByValue().limitLast(20).on("value", function(fbdataSnapshot) {
// fbdataSnapshot is returned in the ascending order
// you will still need to order these 20 items in
// in a descending order
}
Here's a link to the announcement: More querying capabilities in Firebase
To augment Frank's answer, it's also possible to grab the most recent records--even if you haven't bothered to order them using priorities--by simply using endAt().limit(x) like this demo:
var fb = new Firebase(URL);
// listen for all changes and update
fb.endAt().limit(100).on('value', update);
// print the output of our array
function update(snap) {
var list = [];
snap.forEach(function(ss) {
var data = ss.val();
data['.priority'] = ss.getPriority();
data['.name'] = ss.name();
list.unshift(data);
});
// print/process the results...
}
Note that this is quite performant even up to perhaps a thousand records (assuming the payloads are small). For more robust usages, Frank's answer is authoritative and much more scalable.
This brute force can also be optimized to work with bigger data or more records by doing things like monitoring child_added/child_removed/child_moved events in lieu of value, and using a debounce to apply DOM updates in bulk instead of individually.
DOM updates, naturally, are a stinker regardless of the approach, once you get into the hundreds of elements, so the debounce approach (or a React.js solution, which is essentially an uber debounce) is a great tool to have.
There is really no way but seems we have the recyclerview we can have this
query=mCommentsReference.orderByChild("date_added");
query.keepSynced(true);
// Initialize Views
mRecyclerView = (RecyclerView) view.findViewById(R.id.recyclerView);
mManager = new LinearLayoutManager(getContext());
// mManager.setReverseLayout(false);
mManager.setReverseLayout(true);
mManager.setStackFromEnd(true);
mRecyclerView.setHasFixedSize(true);
mRecyclerView.setLayoutManager(mManager);
I have a date variable (long) and wanted to keep the newest items on top of the list. So what I did was:
Add a new long field 'dateInverse'
Add a new method called 'getDateInverse', which just returns: Long.MAX_VALUE - date;
Create my query with: .orderByChild("dateInverse")
Presto! :p
You are searching limitTolast(Int x) .This will give you the last "x" higher elements of your database (they are in ascending order) but they are the "x" higher elements
if you got in your database {10,300,150,240,2,24,220}
this method:
myFirebaseRef.orderByChild("highScore").limitToLast(4)
will retrive you : {150,220,240,300}
In Android there is a way to actually reverse the data in an Arraylist of objects through the Adapter. In my case I could not use the LayoutManager to reverse the results in descending order since I was using a horizontal Recyclerview to display the data. Setting the following parameters to the recyclerview messed up my UI experience:
llManager.setReverseLayout(true);
llManager.setStackFromEnd(true);
The only working way I found around this was through the BindViewHolder method of the RecyclerView adapter:
#Override
public void onBindViewHolder(final RecyclerView.ViewHolder holder, int position) {
final SuperPost superPost = superList.get(getItemCount() - position - 1);
}
Hope this answer will help all the devs out there who are struggling with this issue in Firebase.
Firebase: How to display a thread of items in reverse order with a limit for each request and an indicator for a "load more" button.
This will get the last 10 items of the list
FBRef.child("childName")
.limitToLast(loadMoreLimit) // loadMoreLimit = 10 for example
This will get the last 10 items. Grab the id of the last record in the list and save for the load more functionality. Next, convert the collection of objects into and an array and do a list.reverse().
LOAD MORE Functionality: The next call will do two things, it will get the next sequence of list items based on the reference id from the first request and give you an indicator if you need to display the "load more" button.
this.FBRef
.child("childName")
.endAt(null, lastThreadId) // Get this from the previous step
.limitToLast(loadMoreLimit+2)
You will need to strip the first and last item of this object collection. The first item is the reference to get this list. The last item is an indicator for the show more button.
I have a bunch of other logic that will keep everything clean. You will need to add this code only for the load more functionality.
list = snapObjectAsArray; // The list is an array from snapObject
lastItemId = key; // get the first key of the list
if (list.length < loadMoreLimit+1) {
lastItemId = false;
}
if (list.length > loadMoreLimit+1) {
list.pop();
}
if (list.length > loadMoreLimit) {
list.shift();
}
// Return the list.reverse() and lastItemId
// If lastItemId is an ID, it will be used for the next reference and a flag to show the "load more" button.
}
I'm using ReactFire for easy Firebase integration.
Basically, it helps me storing the datas into the component state, as an array. Then, all I have to use is the reverse() function (read more)
Here is how I achieve this :
import React, { Component, PropTypes } from 'react';
import ReactMixin from 'react-mixin';
import ReactFireMixin from 'reactfire';
import Firebase from '../../../utils/firebaseUtils'; // Firebase.initializeApp(config);
#ReactMixin.decorate(ReactFireMixin)
export default class Add extends Component {
constructor(args) {
super(args);
this.state = {
articles: []
};
}
componentWillMount() {
let ref = Firebase.database().ref('articles').orderByChild('insertDate').limitToLast(10);
this.bindAsArray(ref, 'articles'); // bind retrieved data to this.state.articles
}
render() {
return (
<div>
{
this.state.articles.reverse().map(function(article) {
return <div>{article.title}</div>
})
}
</div>
);
}
}
There is a better way. You should order by negative server timestamp. How to get negative server timestamp even offline? There is an hidden field which helps. Related snippet from documentation:
var offsetRef = new Firebase("https://<YOUR-FIREBASE-APP>.firebaseio.com/.info/serverTimeOffset");
offsetRef.on("value", function(snap) {
var offset = snap.val();
var estimatedServerTimeMs = new Date().getTime() + offset;
});
To add to Dave Vávra's answer, I use a negative timestamp as my sort_key like so
Setting
const timestamp = new Date().getTime();
const data = {
name: 'John Doe',
city: 'New York',
sort_key: timestamp * -1 // Gets the negative value of the timestamp
}
Getting
const ref = firebase.database().ref('business-images').child(id);
const query = ref.orderByChild('sort_key');
return $firebaseArray(query); // AngularFire function
This fetches all objects from newest to oldest. You can also $indexOn the sortKey to make it run even faster
I had this problem too, I found a very simple solution to this that doesn't involved manipulating the data in anyway. If you are rending the result to the DOM, in a list of some sort. You can use flexbox and setup a class to reverse the elements in their container.
.reverse {
display: flex;
flex-direction: column-reverse;
}
myarray.reverse(); or this.myitems = items.map(item => item).reverse();
I did this by prepend.
query.orderByChild('sell').limitToLast(4).on("value", function(snapshot){
snapshot.forEach(function (childSnapshot) {
// PREPEND
});
});
Someone has pointed out that there are 2 ways to do this:
Manipulate the data client-side
Make a query that will order the data
The easiest way that I have found to do this is to use option 1, but through a LinkedList. I just append each of the objects to the front of the stack. It is flexible enough to still allow the list to be used in a ListView or RecyclerView. This way even though they come in order oldest to newest, you can still view, or retrieve, newest to oldest.
You can add a column named orderColumn where you save time as
Long refrenceTime = "large future time";
Long currentTime = "currentTime";
Long order = refrenceTime - currentTime;
now save Long order in column named orderColumn and when you retrieve data
as orderBy(orderColumn) you will get what you need.
just use reverse() on the array , suppose if you are storing the values to an array items[] then do a this.items.reverse()
ref.subscribe(snapshots => {
this.loading.dismiss();
this.items = [];
snapshots.forEach(snapshot => {
this.items.push(snapshot);
});
**this.items.reverse();**
},
For me it was limitToLast that worked. I also found out that limitLast is NOT a function:)
const query = messagesRef.orderBy('createdAt', 'asc').limitToLast(25);
The above is what worked for me.
PRINT in reverse order
Let's think outside the box... If your information will be printed directly into user's screen (without any content that needs to be modified in a consecutive order, like a sum or something), simply print from bottom to top.
So, instead of inserting each new block of content to the end of the print space (A += B), add that block to the beginning (A = B+A).
If you'll include the elements as a consecutive ordered list, the DOM can put the numbers for you if you insert each element as a List Item (<li>) inside an Ordered Lists (<ol>).
This way you save space from your database, avoiding unnecesary reversed data.
I'm new to jQuery and just playing for fun. I have some code that I want to try to modify for my needs but the current js file is getting its data from google spreadsheets and then returning each item as objects. I don't use json to pass data from my server to jQuery so I'm wondering how I can convert json to objects.
The current way its doing it is(tabletop is the name of their js program that gets data from google docs):
Tabletop.init({
key: timelineConfig.key,
callback: setupTimeline,
wanted: [timelineConfig.sheetName],
postProcess: function(el){
//alert(el['photourl']);
el['timestamp'] = Date.parse(el['date']);
el['display_date'] = el['displaydate'];
el['read_more_url'] = el['readmoreurl'];
el['photo_url'] = el['photourl'];
}
});
I have added alerts all over the file and I think this is the area that gets the data and passes it on. I was thinking of trying to replace items in their object with objects from my json and see if it changes anything, but I'm unsure. Typrically I pass individual items via json,hashmaps, and lists, not sure how it works with objects or how to access objects(I simply call url's that I create for the requests, $("#user-history").load("/cooltimeline/{{ user.id }}");). But where do I start if I want to turn json data into objects?
If it helps, here's the demo of what I'm trying to do(but by having it use json data).
p.s. I'm really looking for the logic of how to complete what I'm trying to do and perhaps some ideas I'm missing so I can google them and learn.
Use use function JSON.parse(json) :) Or jQuery.parseJSON(json)
var json = '{"a":2}';
var object = JSON.parse(json);
alert(object.a);
You should see alert with message: 2
I don't realy know if I understand your comment, but maybe you want just do this:
postProcess: function(el){ //here el is JSON string
el = JSON.parse(el); // now el is an object
el.timestamp = Date.parse(el.date);
el.display_date = el.displaydate;
el.read_more_url = el.readmoreurl;
el.photo_url = el.photourl;
return el;
}
Btw. you do not need to use brackets on know property names without not standard names:
el['timestamp'] === el.timestamp
It will be easier if you paste your JSON