Write multiple where condition using Dexie.js - javascript

How can I write a where condition like the following using Dexie.js?
field1=0 AND field2!='test' AND field3 IN(1,2,3,4)
I tried this one but I got an error....
.where('field1').equals(0).and('field2').notEquals('test').and('field3').anyOf([1,2,3,4]).toArray()
I suppose because the ".and" method works only with a function.
But if I have to write a multiple where condition with different types of conditions (e.g.: equals, notEquals, anyOf)... How can I do that?
I apreciate your support,
thanks in advance

Collection.and() is identical to Collection.filter() and takes a JS callback.
The reason is that an index can only be used for one criteria. A typical SQL database has the same limitation but it has some intelligent strategies ( query plan ) that tries to find out the which field to use for index lookup and which ones to use for manually filtering the result. Dexie does not have query plans so you need to do that logic yourself.
For cases where you are combining an equals filter with another filter, a compound index can be used also to get more efficient AND queries.
So as an example we can take your exact example and translate it into the most optimal way using dexie queries:
const db = new Dexie("multipleCriteriasSample");
db.version(1).stores({
someTable: 'id, [field1+field3]'
});
function doTheQuery() {
return db.someTable.where('[field1+field3]').anyOf([
[0, 1],
[0, 2],
[0, 3],
[0, 4]
]).and(item => item.field2 !== "test");
}
The logic for the above is this:
field1 is an equals operator - which makes it possible to combine with another criteria if we have it in a compound index ('[field1+field3]')
field2 is a notEqual which generally never has any gain of using an index - so filter it using the and() method.
field3 is an anyOf(). Use the compound index and do a single anyOf().

Related

Use multiple key-value filters on an object of objects?

Bit of a lengthy one so those of you who like a challenge (or I'm simply not knowledgeable enough - hopefully it's an easy solution!) read on!
(skip to the actual question part to skip the explanation and what I've tried)
Problem
I have a site that has a dataset that contains an object with multiple objects inside. Each of those objects contains an array, and within that array there are multiple objects. (yes this is painful but its from an API and I need to use this dataset without changing or modifying it.) I am trying to filter the dataset based of the key-value pairs in the final object. However, I have multiple filters being executed at once.
Example of Path before looping which retrieves the key-value pair needed for one hall.
["Hamilton Hall"]["Hire Options"][2].Commercial
After Looping Path of required key-value pair for all halls, not just one (the hall identifier is stored):
[0]["Hire Options"][2].Commercial
Looping allows me to check each hall for a specific key-value pair (kind of like map or forEach, but for an object).
After getting that out of the way back to the question.
How would I go about filtering which of the looped objects are displayed?
What I have Tried
(userInput is defined elsewhere - this happens on a btn click btw)
let results = Object.keys(halls);
for (key of results) {
let weekend = [halls[ `${key}` ][ 'Hire Options' ][4][ 'Weekend function' ]];
if(userInput == weekend) {
outputAll([halls[ `${key}` ]]);
}
}
That filters it fine. However, I run into an issue here. I want to filter by multiple queries, and naturally adding an AND into the if statement doesn't work. I also dont want to have 10 if statements (I have 10+ filters of various data types I need to sort by).
I have recently heard of ternary operators, but do not know enough about them to know if that is the correct thing to do? If so, how? Also had a brief loook at switches, but doesnt seem to look like what I want (correct me if I am wrong.)
Actual Question minus the babble/explanation
Is there a way for me to dynamically modify an if statements conditions? Such as adding or removing conditions of an if statement? Such as if the filter for 'a' is set to off, remove the AND condition for 'a' in the if statement? This would mean that the results would only filter with the active filters.
Any help, comments or 'why haven't you tried this' remark are greatly appreciated!
Thanks!
Just for extra reference, here is the code for retrieving each of the objects from the first object as it loops through them:
(Looping Code)
halls = data[ 'Halls' ];
let results = Object.keys(halls);
for (key of results) {
let arr = [halls[ `${key}` ]];
outputAll(arr);
}
You can use Array.filter on the keys array - you can structure the logic for a match how you like - just make it return true if a match is found and the element needs to be displayed.
let results = Object.keys(halls);
results.filter(key => {
if (userInput == halls[key]['Hire Options'][4]['Weekend function']) {
return true;
}
if (some other condition you want to match) {
return true;
}
return false;
}).forEach(key => outputAll([halls[key]]));

How mongodb indexing reduces the the number of docs to be scanned?

I have a collection like: [{a: 'eXeD9', b: 399}, {a: 'eXe9', b: 35399} , xOBJs].
I am gonna search 24823293 in b field. So as I know I am have to traverse all docs, until there is a match with 24823293.
So I am confused if I create an index for b field, how it can reduces the number of docs for scanning?
Because maybe the 24823293 is not within those reduced docs.
As I am a mobile application developer, I am confused here any help.
Because with an index the scan will be performed against the possible values of { b } (which would be stored in a time efficient data structure, like a B-tree) rather than on your whole set of documents.
Creating an index on { b } can be seen as making the value of { b } an access key to the documents themselves.
You end up with an index scan instead of a full scan, which can dramatically make the difference.

pg-promise, format with json-object

From pg-promise's example, one can format a query like below, where ${this~} becomes all of the keys in the object that is the second parameter of "format()".
// automatically list object properties as sql names:
format('INSERT INTO table(${this~}) VALUES(${one}, ${two})', {
one: 1,
two: 2
});
//=> INSERT INTO table("one","two") VALUES(1, 2)
Is it possible to also get all of the values of the object, without explicitly typing all of them? I want to do it like below (should do the same thing as the snippet above, but without typing all of the values):
format('INSERT INTO table(${this~}) VALUES(${this#})', {
one: 1,
two: 2
});
Is it possible to also get all of the values of the object, without explicitly typing all of them?
No, it is not possible, because while column names require the same-type SQL Name escaping, values do not, they require templating that's possible only via explicitly defined variables.
I want to do it like below...
For that you should use the helpers methods of the library:
const cs = new pgp.helpers.ColumnSet(['one', 'two'], {table: 'my-table'});
const query = pgp.helpers.insert(values, cs);

Mongo get inverse of query

Hello I'm having trouble trying to understand how to write this query
My collection is a series of entries like this:
{
id:1,
name:"peter",
number:3
}
I want to be able to write a query which will return all items except for documents where the name='peter' and number=3
I know I can write something like:
db.test.find({$and:[{'name':'peter'},{'num':3}]})
to return all matching items, but is there any way of rewriting this query to return everything but the matching elements?
The $not operator requires a field to be bound to , but in this case it wont work.
Basically I had to rethink my query, using DeMorgan's law
¬(A^B) = ¬(A)V¬(B)
NOT (A AND B) = NOT(A) OR NOT(B)
so my query is
db.test.find({ $or:[{name:{$not:{$eq:'peter'}}},{num:{$not:{$eq:3}}}]});
Boolean algebra to the rescue!!
You can use a trick involving $nor with only one statement. Your only statement is then the original query. This works because $nor means that all conditions must be false; if you have only one condition, you get the negation.
So try:
db.test.find({$nor:[{$and:[{'name':'peter'},{'num':3}]}]})
I think this is nice because it's the negation of your original query exactly as it was
You can use the $ne operator to test for not equality:
find({$and: [{name: {$ne: peter}}, {num: {$ne: 2}}]})

Algorithm for data filter

Can you suggest me an algorithm for filtering out data.
I am using javascript and trying to write out a filter function which filters an array of data.I have an array of data and an array of filters, so in order to apply each filter on every data, I have written 2 for loops
foreach(data)
{
foreach(filter)
{
check data with filter
}
}
this is not the proper code, but in short that what my function does, the problem is this takes a huge amount of time, can someone suggest a better method.
I am using the Mootools library and the array of data is JSON array
Details of data and Filter
Data is JSON array of lets say user, so it will be
data = [{"name" : "first", "email" : "first#first", "age" : "20"}.
{"name" : "second", "email" : "second#second", "age" : "21"}
{"name" : "third", "email" : "third#third", "age" : "22"}]
Array of filters is basically self define class for different fields of data
alFilter[0] = filterName;
alFilter[1] = filterEmail;
alFilter[2] = filterAge;
So when I enter the first for loop, I get a single JSON opbject (first row) in the above case.
When I enter the second for loop (filters loop) I have a filter class which extracts the exact field on which the current filter would work and check the filter with the appropriate field of the data.
So in my example
foreach(data)
{
foreach(filter)
{
//loop one - filter name
// loop two - filter email
// loop three - filter age
}
}
when the second loop ends i set a flag denoting if the data has been filtered or not and depending on it the data is displayed.
You're going to have to give us some more detail about the exact structure of your data and filters to really be able to help you out. Are the filters being used to select a subset of data, or to modify the data? What are the filters doing?
That said, there are a few general suggestions:
Do less work. Is there some way you can limit the amount of data you're working on? Some pre-filter that can run quickly and cut it down before you do your main loop?
Break out of the inner loop as soon as possible. If one of the filters rejects a datum, then break out of the inner loop and move on to the next datum. If this is possible, then you should also try to make the most selective filters come first. (This is assuming that your filters are being used to reject items out of the list, rather than modify them)
Check for redundancy in the computation the filters perform. If each of them performs some complicated calculations that share some subroutines, then perhaps memoization or dynamic programming may be used to avoid redundant computation.
Really, it all boils down to the first point, do less work, at all three levels of your code. Can you do less work by limiting the items in the outer loop? Do less work by stopping after a particular filter and doing the most selective filters first? Do less work by not doing any redundant computation inside of each filter?
That's pretty much how you should do it. The trick is to optimize that "check data with filter"-part. You need to traverse all your data and check against all your filters - you'll not going to get any faster than that.
Avoid string comparisons, use data models as native as possible, try to reduce the data set on each pass with filter, etc.
Without further knowledge, it's hard to optimize this for you.
You should sort the application of your filters, so that two things are optimized: expensive checks should come last, and checks that eliminate a lot of data should come first. Then, you should make sure that checking is cut short as soon as an "out" result occurs.
If your filters are looking for specific values, a range, or start of a text then jOrder (http://github.com/danstocker/jorder) will fit your problem.
All you need to do is create a jOrder table like this:
var table = jOrder(data)
.index('name', ['name'], { grouped: true, ordered: true })
.index('email', ['email'])
.index('age', ['age'], { grouped: true, ordered: true, type: jOrder.number });
And then call table.where() to filter the table.
When you're looking for exact matches:
filtered = table.where([{name: 'first'}, {name: 'second'}]);
When you're looking for a certain range of one field:
filtered = table.where([{age: {lower: 20, upper: 21}}], {mode: jOrder.range});
Or, when you're looking for values starting with a given string:
filtered = table.where([{name: 'fir'}], {mode: jOrder.startof});
Filtering will be magnitudes faster this way than with nested loops.
Supposing that a filter removes the data if it doesn't match, I suggest, that you switch the two loops like so:
foreach(filter) {
foreach(data) {
check data with filter
}
}
By doing so, the second filter doesn't have to work all data, but only the data that passed the first filter, and so on. Of course the tips above (like doing expensive checks last) are still true and should additionally be considered.

Categories