Pymongo count all documents where a field exists with map reduce - javascript

I'm trying to calculate all documents where a certain field exists (in this case it's "country" field) with Map+Reduce, and the only solution that worked for me is this:
mapper = Code("""
function () {
if (typeof this.country != 'undefined') {
var key = 1
emit(key, {count: 1})
};
};
""")
I'm not really interested in keys, just if field exists, so I just passed 1.
But I'm sure that's wrong.
reducer = Code("""
function (key, values) {
var sum = 0;
values.forEach(function (value) {
sum += value['count'];
});
return {count: sum};
};
""")
And then calling map_reduce:
results = dbHandle.cards.map_reduce(mapReduce.mapper, mapReduce.reducer, "resultsMR")
for doc in results.find():
print "Found %s documents." % int(doc.get('value').get('count'))
Also I'm thinking on how to get the amount of docs where their creation date is > than other date, should I use a "query" option in map_reduce function?
query = {"foundationDate":{"$gt":datetime.datetime(2012, 1, 1, 00, 00, 00)}}
Thank you :)

Per Chien-Wei Huang comment, why not use the in built functionality e.g.
db.somName.find({"country":{"$exists":True}}).count()
Else if you want some token map-reduce code you cold simply mimic the count functionality, via the group() function e.g.
db.somName.group(
{ cond: { "country" : {$exists : true} }
, key: { }
, initial: {count : 0}
, reduce: function(doc, out){ out.count++}
}
);
Note: If you also want a count by country value, then stick the following in the key:
, key { "country" : 1 }

Related

get GroupBy count in Dexie

I have an IndexedDB table that follows accepts following structured JSON as a row:
{
id : 1,
name : 'doc1',
createdDate : '2018-08-08'
}
I want to get count for each available date in the table. ie: groupby:date then count. Expected example output is in the format of:
{
'2018-08-08' : 5,
'2018-08-18' : 19,
...
}
Table contains large number of records. So, how can efficiently achieve this requirement using Dexie?
If you index createdDate,
const db = new Dexie("yourDB");
db.version(1).stores({
yourTable: "id, name, createdDate"
});
Then you could do the following:
const result = {}
await db.yourTable.orderBy('createdDate').eachKey(date => {
result[date] = (result[date] || 0) + 1;
});

How to get specific data in JSON result

Here my JSON result of my "modeles" of car:
[
{
"idModele":1,
"modele":"M3",
"marque":{
"idMarque":1,
"marque":"BMW"
}
},
{
"idModele":2,
"modele":"RS6",
"marque":{
"idMarque":2,
"marque":"Audi"
}
},
{
"idModele":3,
"modele":"C63 AMG",
"marque":{
"idMarque":3,
"marque":"Mercedes"
}
},
{
"idModele":4,
"modele":"Clio RS Trophy",
"marque":{
"idMarque":4,
"marque":"Renault"
}
},
{
"idModele":5,
"modele":"Scirocco Type R",
"marque":{
"idMarque":5,
"marque":"Volkswagen"
}
},
{
"idModele":6,
"modele":"118d",
"marque":{
"idMarque":1,
"marque":"BMW"
}
}
]
I just want to get the "modeles" that have the "idMarque:1" (BMW) (in my result they have 2 "modeles") but I don't know how to do it.
My backend : API REST with SpringBoot
My frontend : Angular
Assuming the json array is stored in the variable result, you may simply:
Loop over the json-array.
Check each json-object for the desired condition.
for (let i = 0; i < result.length; i++) {
if (result[i].marque.idMarque === 1) {
console.log('Found it ', result[i]);
}
}
Even simpler:
result.filter(e => e.marque.idMarque === 1);
First, just for clarification, this is a javascript question. It doesn't matter what your backend or frontend is.
Answering your question, you can filter your result to get only the elements you're seeking:
filteredCars = cars.filter(car => car.marque.idMarque === 1)
This will filter the cars with marque.idMarque = 1.
You can find about the filter function on the docs.
You can get the model having idMarque:1 using filter operator. For example you get the JSON result in result class variable. Then you can use filter as follows.
let BMWCars = this.result.filter(e => e.marque.idMarque == 1);
Is a good idea to check, if you have searched object values, so:
let filtered = models
.filter(item => item.marque && item.marque.idMarque
? item.marque.idMarque === 1
: '')
In this case, if you did not get error, when marque key is missing from server response.

refactor two $.each() loops to one

I have a JSON object like this...
{
"tasks":[
{
"id":"task_3",
"taskName":"Task A",
"assignee":"Barrack Obama",
"timeReqOptimisitic":"4",
"timeReqNormal":"8",
"timeReqPessimistic":"14",
"timeUnit":"Days",
"timeReq":"8.33",
"positionX":493,
"positionY":101,
"lockStatus":"unlocked"
}
],
"milestones":[
{
"id":"task_1",
"milestoneName":"Start",
"positionX":149,
"positionY":109,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-07"
},
{
"id":"task_2",
"milestoneName":"Finish",
"positionX":989,
"positionY":367,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-22"
}
],
"connections":[
{
"connectionId":"con_10",
"pageSourceId":"task_1",
"pageTargetId":"task_3"
},
{
"connectionId":"con_20",
"pageSourceId":"task_3",
"pageTargetId":"task_2"
}
]
}
...this is a minimal version. In practice, there are numerous items in "tasks", "milestones" and "connections".
I need to iterate through the object and determine the "id" of the "milestones" item with the lowest/earliest "milestoneDate", then identify the "connections" item that has the same value for its "pageSourceId" and return its "pageTargetId".
So in the above example:
Step 1) Iterate through the object and determine the "id" of the "milestones" item with the lowest/earliest "milestoneDate".
Answer: milestones.id = "task_1"
Step 2) Identify the "connections" item that has the same value for its "pageSourceId".
Answer: connections.pageSourceId = "task_1"
Step 3) Return its "pageTargetId".
Answer: "task_3"
I have a working example here. However, I would like to know if there is a way to accomplish this without using the extremely high start date and also in one loop.
As you are not parsing the same array on these two loops, there is no way to merge your loops.
Anyway, you can yet remove the loops to access to the arrays:
http://jsfiddle.net/gael/sruvtwre/2/
$.each(object.milestones, function( index, value ) {
if(startDate > parseDate(value.milestoneDate)) {
startDate = parseDate(value.milestoneDate);
id = value.id
}
});
$.each(object.connections, function( index, value ) {
if(id == value.pageSourceId) {
pageTargetId = value.pageTargetId;
}
});
May be also sorting, and indexing your datas. Then you would need no loops:
Elements in milestones should be sorted, so the earliest milestones element would be milestones[0].
Elements in connections should be indexed by their pageTargetId property, so the requested element should be connections[id].
Your two loops would become:
var pageTargetId= object.connections[ object.milestones[0].id ].pageTargetId;
http://jsfiddle.net/gael/sruvtwre/4/
As said in comments, sorting is not an optimal solution, even if that does not really matter for small sets.
Roughly, there is no no needs to sort all the datas, just the latest matters.
You can use array reduce method, as an comparable alternative to a simple loop:
var latestMilestone= object.milestones.reduce(function(milestone1, milestone2){
if( parseDate(milestone1.milestoneDate) > parseDate(milestone2.milestoneDate) )
return milestone1;
else
return milestone2;
//convert date to timestamp
function parseDate(date) {
var parts = date.split('-');
return Date.UTC(parts[0], parts[1]-1, parts[2]); // Note: months are 0-based
}
});
How about this:
Assuming you get the milestones.id = "task_1" in first loop; outside the loop we can have use jQuery grep. As connections will have unique pageSourceId, grep will return an array with only one object.
var filteredData = jQuery.grep('CONNECTIONS_ARRAY', function(element, index){
return element.pageSourceId == 'MILESTONES_ID'; // Which you get in the loop earlier
});
Then we can access pageTargetId like this:
if(filteredData.length){
filteredData[0].pageTargetId;
}
Try
var dates = []
, ids = []
, filtered = $.map(data.milestones, function(value, index) {
dates.push(new Date(value.milestoneDate).getTime());
ids.push(value.id);
if (dates.length === data.milestones.length) {
var id = ids[$.inArray(Math.min.apply(Math, dates), dates)]
, res = $.grep(data.connections, function(task, key) {
return task.pageSourceId === id
})[0].pageTargetId;
return res
}
})[0]; // `"task_3"`
var data = {
"tasks":[
{
"id":"task_3",
"taskName":"Task A",
"assignee":"Barrack Obama",
"timeReqOptimisitic":"4",
"timeReqNormal":"8",
"timeReqPessimistic":"14",
"timeUnit":"Days",
"timeReq":"8.33",
"positionX":493,
"positionY":101,
"lockStatus":"unlocked"
}
],
"milestones":[
{
"id":"task_1",
"milestoneName":"Start",
"positionX":149,
"positionY":109,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-07"
},
{
"id":"task_2",
"milestoneName":"Finish",
"positionX":989,
"positionY":367,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-22"
}
],
"connections":[
{
"connectionId":"con_10",
"pageSourceId":"task_1",
"pageTargetId":"task_3"
},
{
"connectionId":"con_20",
"pageSourceId":"task_3",
"pageTargetId":"task_2"
}
]
};
var dates = []
, ids = []
, filtered = $.map(data.milestones, function(value, index) {
dates.push(new Date(value.milestoneDate).getTime());
ids.push(value.id);
if (dates.length === data.milestones.length) {
var id = ids[$.inArray(Math.min.apply(Math, dates), dates)]
, res = $.grep(data.connections, function(task, key) {
return task.pageSourceId === id
})[0].pageTargetId;
return res
}
})[0];
document.write(filtered);
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js">
</script>

Look for a value for a given key in JSON and change the value using javascript

I am looking to write a function which can look up a value based on a key and replace that value with another. The key is a tree from the start node of JSON. Here is the example.
var myData = {
name : 'Dan',
address: {
city : 'Santa Clara',
details : {
'prevhouse' : ''
}
}
}
Input to the function is a key tree. For eg, myData-address-details-prevhouse
When I pass this key with a new value, say 'Texas', the prevhouse value will get changed to the new value I am sending.
and new JSON will be
var myData = {
name : 'Dan',
address: {
city : 'Santa Clara',
details : {
'prevhouse' : 'Texas'
}
}
}
Here is what I wrote so far
var tree = key.split("-");
now the tree variable contains ["myData","address", "details","prevhouse"]
I know that we can access the object using myData[tree[0]][tree[1]][tree[2]], but somehow not able to get it dynamic from parsed value.
how do we generate this dynamically since the length of the depth is not known till runtime.
Hope to get a help.
try with this code:
var myData = {
name: 'Dan',
address: {
city: 'Santa Clara',
details: {
prevhouse: ''
}
}
};
function setAttribute(obj, key, value) {
var i = 1,
attrs = key.split('-'),
max = attrs.length - 1;
for (; i < max; i++) {
attr = attrs[i];
obj = obj[attr];
}
obj[attrs[max]] = value;
console.log('myData=', myData);
}
setAttribute(myData, "myData-address-details-prevhouse", "Texas");
here a working jsfiddle demo; see the console for the result
You should be able to iterate through each key because your JSON is just a JS object. So go through each key, check if it's defined, if it is, use that object for your next check. That'll get you where you want to go. Keep in mind you'll be setting the last key to your value.
basic psuedo-code without dealing with setting:
obj = data;
for (key in keys) {
obj = obj[key]
}
Something like this would do:
function update(node, path, value) {
path = path.split('-');
do {
node = node[path.splice(0, 1)];
} while(path.length > 1);
node[path[0]] = value;
}
Given that myData is the object, I think you should be using myData[tree[1]][tree[2]][tree[3]] and throwing away the first item in the array.
Something like this should work recursively (untested)
function updateValue(obj, key, value)
{
var keys = key.split('-');
updateObjectValue(obj, keys.shift(), value);
}
function updateObjectValue(obj, keyArray, value)
{
if (keyArray.length == 1) {
obj[keyArray[0]] = value;
}
else if (keyArray.length > 1) {
updateObject(obj[keyArray[0]], keyArray.shift(), value);
}
}

JavaScript: How do I dynamically "filter" my objects

How do I user the JavaScript "filter" attribute as filter my JavaScript object?
I've been reading the following StackOverflow post, and am in a similar situation.
I have the following JavaScript object:
{
'cars' :
[{
"car_id" : "1",
"price" : "42999",
"make_id" : "050",
"year_built" : "2007",
"color_id" : "832"
},
..........
]}
I'm using JQuery to display controls to allow people to filter based on: Price, Make, Year Built, Color
Per that other post, I can use the following code:
// if using an old browser, define the 'filter' attribute
if (!Array.prototype.filter)
{
Array.prototype.filter = function(fun /*, thisp*/)
{
var len = this.length >>> 0;
if (typeof fun != "function")
throw new TypeError();
var res = new Array();
var thisp = arguments[1];
for (var i = 0; i < len; i++)
{
if (i in this)
{
var val = this[i]; // in case fun mutates this
if (fun.call(thisp, val, i, this))
res.push(val);
}
}
return res;
};
}
then to perform the actual filter, I can do:
result = cars.
filter(function(p) { return p.price >= 15000 }).
filter(function(p) { return p.price <= 40000 }).
filter(function(p) { return p.year_built >= 2000 }) etc
What I don't understand is, how do I use my JQuery controls to dynamically change the filter once the filter has already been set? Meaning, let's say I have the filter applied from above, then the user changes there mind and wants to increase the maximum they are willing to pay for a car from $40,000 to $50,000.
How would I problematically modify my filter from :
filter(function(p) { return p.price <= 40000 }).
to:
filter(function(p) { return p.price <= 50000 }).
how do I use my JQuery controls to dynamically change the filter once the filter has already been set?
You don't set a filter. You call filter() with a filter function and get a filtered array back; you can't change the filter that was applied to the array afterwards. Instead you must call filter() again, and pass a different filter function.
Or the same filter function with a closure over a variable that has changed:
var minprice= 10000;
var minpricefilter= function(p) { return p.price>=minprice };
result= cars.filter(minpricefilter);
minprice= 20000;
result= cars.filter(minpricefilter);
You could use a function generator.
function createPriceFilter(price)
{
filter = function(){ return p.price >= price };
return filter;
}
Then, when you filter, always use a function generator.
cars
.filter( createPriceFilter( mySelectedPrice ) )
.filter( createSomethingFilter(...) )
. (...)
Instead of filter, how about a plain old loop:
var min_year = 2000;
var min_price = 15000;
var max_price = 40000;
function fillTable() {
clearTheTable();
for (i = 0; i < cars.length; i++) {
var car = cars[i];
if (p.price >= min_price && p.price <= max_price && p.year_built >= min_year)
addCarToTable(car);
}
}
Each time your parameters change, just call fillTable() again to regenerate the whole table.
(There are much cleverer things you can do but this is the simplest thing I could think of.)
Forget callback based filtering. Enter jOrder: http://github.com/danstocker/jorder.
Filtering by iterating over your entire table is tedious and slow. With jOrder, you search by index:
var table = jOrder(json.cars)
.index('id', ['car_id'])
.index('price', ['price'], { grouped: true, ordered: true, type: jOrder.number })
.index('maker', ['maker_id'], { grouped: true })
.index('year', ['year_built'], { grouped: true, ordered: true, type: jOrder.number })
.index('color', ['color_id'], { grouped: true });
Then you get the records you want by:
var filtered = table.where([{ price: { lower: 15000, upper: 40000 } }], { mode: jOrder.range });
Note that you can only apply one inequality filter at a time. To do more, use filtered as an input for a different jOrder table, put only the necessary index on it, and perform the second inequality filter on that one. And so on. Even if you stack up a couple of filters like this, it will be still faster than iteration by a factor of about 10 to 100 depending on the size of your table.

Categories