I have a CouchDB database with the following type of documents, representing events that happen within a building:
{ person: 1
timestamp: 1
event: { type: enter
room: b }
}
and
{ person: 2
timestamp: 5
event: { type: leave
room: b }
}
The problem that I want to solve is the following: I want to know the total amount of time that every other person spent in the same room as person 1. Note that any person can enter and leave many rooms at many different times. I honestly don't know whether MapReduce is the best paradigm for this, or if I should just export my data and write a separate script to figure this stuff out (although this is probably not a feasible solution for our production environment).
As a starting solution lets assume that all the data is sane, and thus someone entering a room will also leave that room at a later time. However, in a final solution this requirement will probably have to be relaxed, because some events may be missing.
I have thought of a potential solution, but I have no idea whether this is at all possible or how to do this in couchdb. Here is an outline.
Create a view that emits the following format, for every person entering a room event:
{ [room, person, timestamp], null }
Create a view that emits { [room, timestamp], null} for every time person 1 exits the room (could be for all people, but is unnecessary).
Create a view that for each exiting a room event for any person except person 1, does the following. In the mapping step:
Queries the first view to find the last timestamp when that person entered the room.
Queries the first view to find all times before the exiting the room event that person 1 entered that room
For each of those times, queries the second view to find all exit times for that room, and for each interval checks what the overlap is.
Sum these overlaps together and emit as { person, time }
Reduce:
for every person, sum all the times together.
However, this relies on me being able to figure out how to query a different view from within a view. Does anybody know if that is possible, and if so, how?
The only way I have found for doing this within the CouchDB structure is by using a list function. I create a view that simply emits all documents with [building, timestamp] as key. This helps me query the view to ensure I have a single day and a single building, with startkey and endkey.
I now create a list function that simply takes all the documents returned by the view, and performs the processing in a javascript function. This bypasses the map-reduce framework for the most part, but was the only way I could think of doing this within the CouchDB framework. Obviously the same could be done with any other script instead of the list function, using CouchDB's RESTful API.
Related
I'm stuck. Stuck because Firestore is not (yet) capable to handle my (to me) relatively simple query. I don't want anything fancy. All I wish to get from the database are the meetings that have not ended yet. But, I would like the list to be limited to 5 items, and ordered by the starting date of the meeting, to display a small "Up next" style Agenda.
However, the following is not a valid query:
where("end", "<", now), orderBy("start", "asc"), limit(5));
So, how do I achieve this rather simple query for my Agenda?
And, while we're here, maybe we can dig into the other queries, too:
:: Display the last meeting (Already Over)
where("end", "<", now), orderBy("end", "desc"), limit(1))
:: Display the current meeting (Now) - Started, but not Ended.
.... ?????
:: Display the meetings which have not yet started (Next)
where("start", ">", now), orderBy("start", "asc"), limit(5))
The only thing that I can think of right now for the "current", is to grab the array of all the meetings that have ended. Grab the array of all the future meetings, and an array of all meetings. Substract the (previous) and (future) arrays from the (all) list, and I'll have the one meeting that hasn't ended, but has already started. Theres gotta be a more efficient way to do this. No?
The common approach for this is to define buckets, and then assign each meeting to the relevant buckets.
For example, say that you show the events that are going on today, and then style the events that have already started/ended differently.
In such a scenario, you could have an array of meetings_days that contains the days that the meeting is active: meeting_days: ['2022-12-01', '2022-12-02', '2022-12-03', '2022-12-04', '2022-12-05'].
Now you can use an array-contains filter to select the events for a given day, and then determine the styling in your client-side application code. If you don't show all events (e.g. not ones that have already finished), determining the right bucket size to limit overreads while keeping the data size reasonable.
An alternative data model could be a may of the meeting days: meeting_days: { '2022-12-01': true, '2022-12-02': true, '2022-12-03': true, '2022-12-04': true, '2022-12-05': true }. Now you can do an AND type query, like finding only events that run the first 2 days of December.
The correct data model here depends on the use-cases of your app, and will likely change/evolve as your app evolves.
Beginner and self-taught coder here (always open to learning please correct me) and I'm making a web app through pretty much exclusively HTML CSS and Javascript (I don't really want to use PHP or hosting-side processing because I don't know much about web security and it makes me nervous about uploading data to my hosted site).
Very unsure about the most efficient way to do this so I'm going to try to describe it below and I'd really appreciate your input.
My main question: Is there a more efficient way to do this?
The app eventually will have a javascript canvas, where it will draw an object ('track') at a specific location. This object will then move to another location based off nested data in an array ('step') when the user moves to the next item in an array.
As of now, how I'm going about it is having:
storing the location values in the steps array
have an array of 'tracks' for what shape/color/etc will be drawn on the canvas
linking the two elements by an arbitrary ID that is in both 'steps array' and 'tracks' array
A visual representation of what this might look like
steps[stepNumber].movedTracksInStep[movedTracksInStepNumber] holds object:
{track ID,
X location,
y location}
separate array trackList
trackList[trackNumber] holds object:
{track ID,
shape,
color,
bunchastuff}
I choose to do it like this because I figured it would be better to store the location in the steps array, store the visual data in a separate array, so that way it's not repeating the same data every step.
My question:
Is there a more efficient way to do this, especially in terms of search functions? I'm a newbie so there very well might be something I am missing.
Currently, I just have to search through all of the ID tracks in the step and see if there is a match. I'm wondering if there is a more direct way to link the two together than having to search each time.
I've thought about perhaps having all the data for the visual representation in the first step and then not having to repeat it (though I'm not quite sure how that would work), or having the numbers of arrays match up (but this would change if the user deletes a track or adds a track).
Thank you! Let me know if you need me to explain more.
Objects in JS are stored and copied "by reference", so if you assign value of one object to another, value will not be copied, but reference link will be created. Below is the example close to your code, check inline comments. And you can adopt this behavior to your task:
// Your tracks information
const trackList = {
1: {
shape: "rect",
color: "green",
bunchastuff: "foo"
}
};
// Your steps data
const steps = {
1: {
1: {
// Here we create reference to track 1 in
// trackList object data, without copying it
track: trackList[1],
x: 100,
y: 50
}
}
};
// Print step info
console.log("before track info edit:", steps[1][1].track);
// Update data in track 1
trackList[1].shape = "round";
// Print step info again and we'll
// see, that it also updated
console.log("after track info edit:", steps[1][1].track);
You can read more about object references here: https://javascript.info/object-copy
I'm using Javascript, Node.js, Express and MongoDB to make a webapplication. In the application You can make an account with fields name and lastnames, but I also want a field that keeps track of completed steps (boolean: false if uncompleted, true if completed). It's not a possibility to hardcode the steps as separate boolean fields because I want to make the list of steps adjustable. So I was thinking of making another collection (=progress) with al the names of the steps and where you can add some and know I am searching for a way to include these boolean steps for each account I make.
sketch of how I think the MongoDB collection Schema's should look like:
account
- name (String)
- lastname (String)
- progress:
- stepName (String) , completed (boolean)
- stepName (String) , completed (boolean)
- stepName (String) , completed (boolean)
- ...
progress
- stepName (String)
- completed (boolean)
Many thanks in advance
You can -- since it looks like you are using Mongoose, you can see https://mongoosejs.com/docs/subdocs.html for some info on how to do something like you are outlining here.
However, based on my experience doing similar things (tracking progress through a flow), I would suggest setting things up a little differently. The biggest issue I see is that if you add a new step, you don't want to have to go through and update every account document to put the new step into its progress field. Or similarly, if you decide to remove a step, you don't want to have to go to each account and remove that step's entry from its progress field.
Instead of thinking of it as including the progress collection in each document of the account collection, I would think of it as having a step collection that contains any adjustable/configurable data about each step. There is no need to have a completed field there, because that's something specific to each account. Depending on your needs, I would suggest something like the following:
step
- id (String) -- something that will never change
- name (String) -- a user-facing name that you might want to change later
- position (number) -- used to control the order of steps presented to the user
- ... other things, depending on what you need/want
Now, for account, you will want a map (https://mongoosejs.com/docs/schematypes.html#maps) showing the progress (completion state of each step):
account
- name (String)
- lastname (String)
- progress: (Map of boolean)
- key: step id
- value: boolean showing whether it has been completed
- ...
This means any time you're checking an account's progress, you'll want to loop through the documents from the step collection (ordered based on the position field), and check that account's progress map against each step id to see if there are any that aren't completed. Although that sounds like extra work, there are a lot of advantages to this sort of approach. It means if you decide to add, remove, or reorder a step within the step collection, you don't need to go make the same change to every existing account document to match -- you'll just automatically pick up the change the next time you check the account's progress field.
One further note -- the way I'm suggesting you set up the schema, instead of having an explicit false value for each step that isn't completed, most of the time you just wouldn't have any value in the map for uncompleted steps. In your code, you can easily treat any missing step progress values the same as a false (if (!account.progress.firstStep) { works by default with javascript). This also gives you the option, if you want, of setting a step's progress to explicitly false only when a user has been presented with that step. Thus you have some easier visibility into what step they stopped on (even if you later change the order of the steps) and/or if they've skipped any steps (if some are considered optional), rather than only being able to see which steps have been completed.
What you describe is called an "Embedded Document" and is supported by mongodb.
Mongodb documentation includes an example of what you want:
{
...
name: { first: "Alan", last: "Turing" },
contact: { phone: { type: "cell", number: "111-222-3333" } },
...
}
And in mongodb it's accessed by:
"<embedded document>.<field>"
Source: https://docs.mongodb.com/manual/core/document/
So, all of the code works quite well. A database a queried, the node/parent IDs are lazily passed to jsTree, and, if one types a term into the search bar—a similar process goes on, but the nodes passed to jsTree are those returned by another SQL query (using something like SELECT nodeID FROM table WHERE name LIKE %searchTerm%).
There's only one problem:
If I type too quickly into the search bar, the results get all mixed up with one another. If I type slowly (I'd estimate 2 letters a second max), everything works well. Any faster and everything is blurred together. (That is, if I'm searching for names which contain the term "test", and type this quickly, I'll get names that contain "t", "te", "tes", and "test" instead of just names that contain "test".)
Anyone have this problem before? Is there some "wait until ready" function I should be using?
I can suggest you to do the 'Timeout' workaround. Basically, you create a SetTimeout function with a delay of 200-400 miliseconds and launch your lazyload ajax there. Every inputbox.change event restarts the time. See example in pseudo-javascript:
$('#your-input').keydown(function(){
if(ajaxTimer != undefined) {
clearTimeout(ajaxTimer);
}
ajaxTimer = setTimeout(function(){
$.ajax({...}).done(function() {...}
},400);
})
Use a throttle on the typing input, basically guarantees that a function will be called a maxmimum of once every X seconds. Sure you could write your own however there is a great library out there already.
check it out if you're interested http://benalman.com/projects/jquery-throttle-debounce-plugin/
I have a scenario on my web application and I would like suggestions on how I could better design it.
I have to steps on my application: Collection and Analysis.
When there is a collection happening, the user needs to keep informed that this collection is going on, and the same with the analysis. The system also shows the 10 last collection and analysis performed by the user.
When the user is interacting with the system, the collections and analysis in progress (and, therefore, the last collections/analysis) keep changing very frequently. So, after considering different ways of storing these informations in order to display them properly, as they are so dynamic, I chose to use HTML5's localStorage, and I am doing everything with JavaScript.
Here is how they are stored:
Collection in Progress: (set by a function called addItem that receives ITEMNAME)
Key: c_ITEMNAME_Storage
Value: c_ITEMNAME
Collection Finished or Error: (set by a function called editItem that also receives ITEMNAME and changes the value of the corresponding key)
Key: c_ITEMNAME_Storage
Value: c_Finished_ITEMNAME or c_Error_ITEMNAME
Collection in the 10 last Collections (set by a function called addItemLastCollections that receives ITEMNAME and prepares the key with the current date and time)
Key: ORDERNUMBER_c_ITEMNAME_DATE_TIME
Value: c_ITEMNAME
Note: The order number is from 0 to 9, and when each collection finishes, it receives the number 0. At the same time, the number 9 is deleted when the addItemLastCollections function is called.
For the analysis is pretty much the same, the only thing that changes is that the "c" becomes an "a".
Anyway, I guess you understood the idea, but if anything is unclear, let me know.
What I want is opinions and suggestions of other approaches, as I am considering this inefficient and impractical, even though it is working fine. I want something easily maintained. I think that sticking with localStorage is probably the best, but not this way. I am not very familiar with the use of Design Patterns in JavaScript, although I use some of them very frequently in Java. If anyone can give me a hand with that, it would be good.
EDIT:
It is a bit hard even for me to explain exactly why I feel it is inefficient. I guess the main reason is because for each case (Progress, Finished, Error, Last Collections) I have to call a method and modify the String (adding underline and more information), and for me to access any data (let's say, the name or the date) of each one of them I need to test to see which case is it and then keep using split( _ ). I know this is not very straightforward but I guess that this whole approach could be better designed. As I am working alone on this part of the software, I don't have anyone that I can discuss things with, so I thought here would be a good place to exchange ideas :)
Thanks in advance!
Not exactly sure what you are looking for. Generally I use localStorage just to store stringified versions of objects that fit my application. Rather than setting up all sorts of different keys for each variable within localStore, I just dump stringified versions of my object into one key in localStorage. That way the data is the same structure whether it comes from server as JSON or I pull it from local.
You can quickly save or retrieve deeply nested objects/arrays using JSON.stringify( object) and JSON.parse( 'string from store');
Example:
My App Object as sent from server as JSON( I realize this isn't proper quoted JSON)
var data={ foo: {bar:[1,2,3], baz:[4,5,6,7]},
foo2: {bar:[1,2,3], baz:[4,5,6,7]}
}
saveObjLocal( 'app_analysis', data);
function saveObjLocal( key, obj){
localStorage.set( key, JSON.stringify(obj)
}
function getlocalObj( key){
return JSON.parse( localStorage.get(key) );
}
var analysisObj= =getlocalObj('app_analysis');
alert( analysisObj.foo.bar[2])