Hello I'm having trouble trying to understand how to write this query
My collection is a series of entries like this:
{
id:1,
name:"peter",
number:3
}
I want to be able to write a query which will return all items except for documents where the name='peter' and number=3
I know I can write something like:
db.test.find({$and:[{'name':'peter'},{'num':3}]})
to return all matching items, but is there any way of rewriting this query to return everything but the matching elements?
The $not operator requires a field to be bound to , but in this case it wont work.
Basically I had to rethink my query, using DeMorgan's law
¬(A^B) = ¬(A)V¬(B)
NOT (A AND B) = NOT(A) OR NOT(B)
so my query is
db.test.find({ $or:[{name:{$not:{$eq:'peter'}}},{num:{$not:{$eq:3}}}]});
Boolean algebra to the rescue!!
You can use a trick involving $nor with only one statement. Your only statement is then the original query. This works because $nor means that all conditions must be false; if you have only one condition, you get the negation.
So try:
db.test.find({$nor:[{$and:[{'name':'peter'},{'num':3}]}]})
I think this is nice because it's the negation of your original query exactly as it was
You can use the $ne operator to test for not equality:
find({$and: [{name: {$ne: peter}}, {num: {$ne: 2}}]})
Related
Bit of a lengthy one so those of you who like a challenge (or I'm simply not knowledgeable enough - hopefully it's an easy solution!) read on!
(skip to the actual question part to skip the explanation and what I've tried)
Problem
I have a site that has a dataset that contains an object with multiple objects inside. Each of those objects contains an array, and within that array there are multiple objects. (yes this is painful but its from an API and I need to use this dataset without changing or modifying it.) I am trying to filter the dataset based of the key-value pairs in the final object. However, I have multiple filters being executed at once.
Example of Path before looping which retrieves the key-value pair needed for one hall.
["Hamilton Hall"]["Hire Options"][2].Commercial
After Looping Path of required key-value pair for all halls, not just one (the hall identifier is stored):
[0]["Hire Options"][2].Commercial
Looping allows me to check each hall for a specific key-value pair (kind of like map or forEach, but for an object).
After getting that out of the way back to the question.
How would I go about filtering which of the looped objects are displayed?
What I have Tried
(userInput is defined elsewhere - this happens on a btn click btw)
let results = Object.keys(halls);
for (key of results) {
let weekend = [halls[ `${key}` ][ 'Hire Options' ][4][ 'Weekend function' ]];
if(userInput == weekend) {
outputAll([halls[ `${key}` ]]);
}
}
That filters it fine. However, I run into an issue here. I want to filter by multiple queries, and naturally adding an AND into the if statement doesn't work. I also dont want to have 10 if statements (I have 10+ filters of various data types I need to sort by).
I have recently heard of ternary operators, but do not know enough about them to know if that is the correct thing to do? If so, how? Also had a brief loook at switches, but doesnt seem to look like what I want (correct me if I am wrong.)
Actual Question minus the babble/explanation
Is there a way for me to dynamically modify an if statements conditions? Such as adding or removing conditions of an if statement? Such as if the filter for 'a' is set to off, remove the AND condition for 'a' in the if statement? This would mean that the results would only filter with the active filters.
Any help, comments or 'why haven't you tried this' remark are greatly appreciated!
Thanks!
Just for extra reference, here is the code for retrieving each of the objects from the first object as it loops through them:
(Looping Code)
halls = data[ 'Halls' ];
let results = Object.keys(halls);
for (key of results) {
let arr = [halls[ `${key}` ]];
outputAll(arr);
}
You can use Array.filter on the keys array - you can structure the logic for a match how you like - just make it return true if a match is found and the element needs to be displayed.
let results = Object.keys(halls);
results.filter(key => {
if (userInput == halls[key]['Hire Options'][4]['Weekend function']) {
return true;
}
if (some other condition you want to match) {
return true;
}
return false;
}).forEach(key => outputAll([halls[key]]));
How can I write a where condition like the following using Dexie.js?
field1=0 AND field2!='test' AND field3 IN(1,2,3,4)
I tried this one but I got an error....
.where('field1').equals(0).and('field2').notEquals('test').and('field3').anyOf([1,2,3,4]).toArray()
I suppose because the ".and" method works only with a function.
But if I have to write a multiple where condition with different types of conditions (e.g.: equals, notEquals, anyOf)... How can I do that?
I apreciate your support,
thanks in advance
Collection.and() is identical to Collection.filter() and takes a JS callback.
The reason is that an index can only be used for one criteria. A typical SQL database has the same limitation but it has some intelligent strategies ( query plan ) that tries to find out the which field to use for index lookup and which ones to use for manually filtering the result. Dexie does not have query plans so you need to do that logic yourself.
For cases where you are combining an equals filter with another filter, a compound index can be used also to get more efficient AND queries.
So as an example we can take your exact example and translate it into the most optimal way using dexie queries:
const db = new Dexie("multipleCriteriasSample");
db.version(1).stores({
someTable: 'id, [field1+field3]'
});
function doTheQuery() {
return db.someTable.where('[field1+field3]').anyOf([
[0, 1],
[0, 2],
[0, 3],
[0, 4]
]).and(item => item.field2 !== "test");
}
The logic for the above is this:
field1 is an equals operator - which makes it possible to combine with another criteria if we have it in a compound index ('[field1+field3]')
field2 is a notEqual which generally never has any gain of using an index - so filter it using the and() method.
field3 is an anyOf(). Use the compound index and do a single anyOf().
I will start off by saying while I am not new to CouchDB, I am new to querying the views using JavaScript and the web.
I have looked at multiple other questions on here, including CouchDB - Queries with params, couchDB queries, Couchdb query with AND operator, CouchDB Querying Dates, and Basic CouchDB Queries, just to list a few.
While all have good information in them, I haven't found one that has my particular problem in it.
I have a view set up like so:
function (docu) {
if(docu.status && docu.doc && docu.orgId.toString() && !docu.deleted){
switch(docu.status){
case "BASE":
emit(docu.name, docu);
break;
case "AIR":
emit(docu.eta, docu);
break;
case "CHECK":
emit(docu.checkTime, docu);
break;
}
}
}
with all documents having a status, doc, orgId, deleted, name, eta, and checkTime. (I changed doc to docu because of my custom doc key.
I am trying to query and emit based on a set of keys, status, doc, orgId, where orgId is an integer.
My jQuery to do this looks like so:
$.couch.db("myDB").view("designDoc/viewName", {
keys : ["status","doc",orgId],
success: function(data) {
console.log(data);
},
error: function(status) {
console.log(status);
}
});
I receive
{"total_rows":59,"offset":59,"rows":[
]}
Sometimes the offset is 0, sometimes it is 59. I feel I must be doing something wrong for this not to be working correctly.
So for my questions:
I did not mention this, but I had to set docu.orgId.toString() because I guess it parses the URL as a string, is there a way to use this number as a numeric value?
How do I correctly view multiple documents based on multiple keys, i.e. if(key1 && key2) emit(doc.name, doc)
Am I doing something obviously wrong that I lack the knowledge to notice?
Thank you all.
You're so very close. To answer your questions
When you're using docu.orgId.toString() in that if-statement you're basically saying: this value must be truthy. If you didn't convert to string, any number, other than 0, would be true. Since you are converting to a string, any value other than an empty string will be true. Also, since you do not use orgId as the first argument in an emit call, at least not in the example above, you cannot query by it at all.
I'll get to this.
A little.
The thing to remember is emit creates a key-value table (that's really all a view is) that you can use to query. Let's say we have the following documents
{type:'student', dept:'psych', name:'josh'},
{type:'student', dept:'compsci', name:'anish'},
{type:'professor', dept:'compsci', name:'kender'},
{type:'professor', dept:'psych', name:'josh'},
{type:'mascot', name:'owly'}
Now let's say we know that for this one view, we want to query 1) everything but mascots, 2) we want to query by type, dept, and name, all of the available fields in this example. We would write a map function like this:
function(doc) {
if (doc.type === 'mascot') { return; } // don't do anything
// allow for queries by type
emit(doc.type, null); // the use of null is explained below
// allow queries by dept
emit(doc.dept, null);
// allow for queries by name
emit(doc.name, null);
}
Then, we would query like this:
// look for all joshs
$.couch.db("myDB").view("designDoc/viewName", {
keys : ["josh"],
// ...
});
// look for everyone in the psych department
$.couch.db("myDB").view("designDoc/viewName", {
keys : ["psych"],
// ...
});
// look for everyone that's a professor and everyone named josh
$.couch.db("myDB").view("designDoc/viewName", {
keys : ["professor", "josh"],
// ...
});
Notice the last query isn't and in the sense of a logical conjunction, it's in the sense of a union. If you wanted to restrict what was returned to documents that were only professors and also joshs, there are a few options. The most basic would be to concatenate the key when you emit. Like
emit('type-' + doc.type + '_name-' + doc.name, null);
You would then query like this: key : ["type-professor_name-josh"]
It doesn't feel very proper to rely on strings like this, at least it didn't to me when I first started doing it, but it is a quite common method for querying key-value stores. The characters - and _ have no special meaning in this example, I simply use them as delimiters.
Another option would be what you mentioned in your comment, to emit an array like
emit([ doc.type, doc.name ], null);
Then you would query like
key: ["professor", "josh"]
This is perfectly fine, but generally, the use case for emitting arrays as keys, is for aggregating returned rows. For example, you could emit([year, month, day]) and if you had a simple reduce function that basically passed the records through:
function(keys, values, rereduce) {
if (rereduce) {
return [].concat.apply([], values);
} else {
return values;
}
}
You could query with the url parameter group_level set to 1 or 2 and start querying by year and month or just year on the exact same view using arrays as keys. Compared to SQL or Mongo it's mad complicated and convoluted, but hey, it's there.
The use of null in the view is really for resource saving. When you query a view, the rows contain an _id that you can use in a second ajax call to get all the documents from, for example, _all_docs.
I hope that makes sense. If you need any clarification you can use the comments and I'll try my best.
I have 2 collections, and I'm trying to get all records from Coll_A where a certain field (Field1) is not present in Coll_B.
The issue I'm running into is that this Field1 in Coll_A, which contains a string value, is padded at the end with white spaces up to a certain length, but this padding is absent in the other collection.
So, for the below query, the vals array has records that aren't padded, so when I use the db.Coll_A.find, it returns incorrect results.
vals = db.Coll_B.find({},
{"Field1" : 1, _id: 0})
.map(function(a){
return a.Field1;
});
db.Coll_A.find({ "Field1": { $nin: vals }});
How can I get around this? I looked up using regex for ignoring whitespaces, but I'm not sure how that can be used here. Any assistance would be really appreciated. Thanks.
You could simply, modify the return value to an array of regular expressions and use it in your $nin operator.
code:
var exp = db.Coll_B.find({}, {"Field1" : 1, _id: 0}).map(function(a){
return new RegExp(a.Field1+"\\s*$");
});
db.coll_A.find({"Field1":{$nin:exp}});
You cannot use $regex operator inside the $nin or $in operators.
I would suggest to fix inconsistent data in the collection. Update all the records with trimming white space around the field.
If for some reason you can't or don't want to do it, then you can use forEach and manually do the checking
var results = [];
db.Coll_A.find({}).forEach(function(row){
if(row.Field1.trim().indexOf(vals) === -1){
results.push(row);
}
});
Can you suggest me an algorithm for filtering out data.
I am using javascript and trying to write out a filter function which filters an array of data.I have an array of data and an array of filters, so in order to apply each filter on every data, I have written 2 for loops
foreach(data)
{
foreach(filter)
{
check data with filter
}
}
this is not the proper code, but in short that what my function does, the problem is this takes a huge amount of time, can someone suggest a better method.
I am using the Mootools library and the array of data is JSON array
Details of data and Filter
Data is JSON array of lets say user, so it will be
data = [{"name" : "first", "email" : "first#first", "age" : "20"}.
{"name" : "second", "email" : "second#second", "age" : "21"}
{"name" : "third", "email" : "third#third", "age" : "22"}]
Array of filters is basically self define class for different fields of data
alFilter[0] = filterName;
alFilter[1] = filterEmail;
alFilter[2] = filterAge;
So when I enter the first for loop, I get a single JSON opbject (first row) in the above case.
When I enter the second for loop (filters loop) I have a filter class which extracts the exact field on which the current filter would work and check the filter with the appropriate field of the data.
So in my example
foreach(data)
{
foreach(filter)
{
//loop one - filter name
// loop two - filter email
// loop three - filter age
}
}
when the second loop ends i set a flag denoting if the data has been filtered or not and depending on it the data is displayed.
You're going to have to give us some more detail about the exact structure of your data and filters to really be able to help you out. Are the filters being used to select a subset of data, or to modify the data? What are the filters doing?
That said, there are a few general suggestions:
Do less work. Is there some way you can limit the amount of data you're working on? Some pre-filter that can run quickly and cut it down before you do your main loop?
Break out of the inner loop as soon as possible. If one of the filters rejects a datum, then break out of the inner loop and move on to the next datum. If this is possible, then you should also try to make the most selective filters come first. (This is assuming that your filters are being used to reject items out of the list, rather than modify them)
Check for redundancy in the computation the filters perform. If each of them performs some complicated calculations that share some subroutines, then perhaps memoization or dynamic programming may be used to avoid redundant computation.
Really, it all boils down to the first point, do less work, at all three levels of your code. Can you do less work by limiting the items in the outer loop? Do less work by stopping after a particular filter and doing the most selective filters first? Do less work by not doing any redundant computation inside of each filter?
That's pretty much how you should do it. The trick is to optimize that "check data with filter"-part. You need to traverse all your data and check against all your filters - you'll not going to get any faster than that.
Avoid string comparisons, use data models as native as possible, try to reduce the data set on each pass with filter, etc.
Without further knowledge, it's hard to optimize this for you.
You should sort the application of your filters, so that two things are optimized: expensive checks should come last, and checks that eliminate a lot of data should come first. Then, you should make sure that checking is cut short as soon as an "out" result occurs.
If your filters are looking for specific values, a range, or start of a text then jOrder (http://github.com/danstocker/jorder) will fit your problem.
All you need to do is create a jOrder table like this:
var table = jOrder(data)
.index('name', ['name'], { grouped: true, ordered: true })
.index('email', ['email'])
.index('age', ['age'], { grouped: true, ordered: true, type: jOrder.number });
And then call table.where() to filter the table.
When you're looking for exact matches:
filtered = table.where([{name: 'first'}, {name: 'second'}]);
When you're looking for a certain range of one field:
filtered = table.where([{age: {lower: 20, upper: 21}}], {mode: jOrder.range});
Or, when you're looking for values starting with a given string:
filtered = table.where([{name: 'fir'}], {mode: jOrder.startof});
Filtering will be magnitudes faster this way than with nested loops.
Supposing that a filter removes the data if it doesn't match, I suggest, that you switch the two loops like so:
foreach(filter) {
foreach(data) {
check data with filter
}
}
By doing so, the second filter doesn't have to work all data, but only the data that passed the first filter, and so on. Of course the tips above (like doing expensive checks last) are still true and should additionally be considered.