We Keep Coding

get GroupBy count in Dexie

I have an IndexedDB table that follows accepts following structured JSON as a row:
{
id : 1,
name : 'doc1',
createdDate : '2018-08-08'
}
I want to get count for each available date in the table. ie: groupby:date then count. Expected example output is in the format of:
{
'2018-08-08' : 5,
'2018-08-18' : 19,
...
}
Table contains large number of records. So, how can efficiently achieve this requirement using Dexie?

If you index createdDate,
const db = new Dexie("yourDB");
db.version(1).stores({
yourTable: "id, name, createdDate"
});
Then you could do the following:
const result = {}
await db.yourTable.orderBy('createdDate').eachKey(date => {
result[date] = (result[date] || 0) + 1;
});

How to declare a Hash/Dictionary of Array

I have a program that pushes values into one data structure like this:
if(symbolType == "C" || symbolType == "P") // The calls and puts
stocks.push({
symbol: symbol,
undsymbol: undSymbol,
open: 0,
type: symbolType,
expiry: expiry,
days: days,
strike: strike
});
}
else // The stock
{
stocks.push({
symbol: symbol,
open: 0,
type: symbolType
});
}
So this is the key: NOT A STRING!
{
symbol: symbol,
open: 0,
type: symbolType
}
And the values of which are many look like this:
{
symbol: symbol,
undsymbol: undSymbol,
open: 0,
type: symbolType,
expiry: expiry,
days: days,
strike: strike
}
The problem is that stocks and calls and puts are being put into one collection. Instead, I want to add the the stocks and their corresponding calls and puts into a dictionary/map, where the stocks are the keys, and the calls and puts get pushed into an array indexed by it's stock.
At the end, I want to be able to iterate and get the keys and values.
How do I declare this object
Index into it to see if the key[stock] already exists, if it doesn't add it with an empty array.
If I get a "C" or "P", I want to get the corresponding array that holds the Calls/Puts for this key [stock] and push the call/put into the array.
Initially I thought the declaration was something like this:
var stockCallsPutDict = {[]}
stockCallsPutDict[stock] = [];
stockCallsPut[stock].push(call);
// Pretty print the dict of keys and its options =
stockCallsPutDict.forEach(function kvp) {
...
}

If ES6 is an option, you can either build an object yourself or use a Map.
Here's some quick code I came up with:
const stocks = {};
const addCallAndPut = callAndPut => {
const symbol = callAndPut.symbol;
if (!stocks[symbol]) {
stocks[symbol] = [];
}
stocks[symbol].push(callAndPut);
}
const showStuff = () => {
for (const symbol in stocks) {
// output stuff using stocks[symbol]
}
}
OR WITH A MAP
const stocks = new Map();
// basic implementation
const addCallAndPut = callAndPut => {
const stockCallsAndPuts = stocks.get(callAndPut.symbol) || [];
stockCallsAndPuts.push(callAndPut);
stock.set(callAndPut.symbol, stockCallsAndPuts);
}

There are a few ways to go about this, and the best depends on how the data needs to be processed later, but from your description I'd go with something along the lines of
var stocks = {};
var stockCallsPut = {};
// loop over stocks and actions
if (!(symbol in stocks)) {
stocks[symbol] = [];
}
if (!(symbol in stockCallsPut)) {
stockCallsPut[symbol] = {};
}
if (!(symbolType in stockCallsPut[symbol])) {
stockCallsPut[symbol][symbolType] = [];
}
// accumulated stock json items here
stocks[symbol].push(new_stock_item);
// accumulated push/call json items of stock here
stockCallsPut[symbol][symbolType].push(new_action);

I'm still not sure I actually understood what your data looks like, but sounds kind of like this to me:
// Not sure if data is an object or array
var data = {
'one': {
'name': 'one-somename',
'number': 'one-somenumber',
'symbol': 'C'
},
'two': {
'name': 'two-somename',
'number': 'two-somenumber',
'symbol': 'P'
},
'three': {
'name': 'three-somename',
'number': 'three-somenumber',
'symbol': 'C'
}
};
var stocks = {};
for (var name in data) {
// It sounded like you wanted a call/put array for each object but I'm not sure if that's true since it wouldn't be possible... if so can just divide this part up into it's appropriate place in the if statement below
// Checking that the property is set on the object, if it is, it uses itself, otherwise it adds it with the call/put arrays created
stocks[name] = stocks[name] ? stocks[name] : {'calls': [], 'puts': []};
var type;
if (data[name]['symbol'] === 'C') {
type = 'calls';
} else if (data[name]['symbol'] === 'P') {
type = 'puts';
}
stocks[name][type].push(data[name]);
}

MongoDB regex replace

My table name is : table_video
My field name in db is : url_value
Whose value is : http://192.168.1.124/test/abcd/abcd.m3u8
Value which is needed by me : http://192.168.1.124/test/abcd_NEW/abcd_NEW.m3u8
There are multiple values in place of "abcd", above url is just one example.
var cursor = db.table_video.find();
while (cursor.hasNext()) {
var x = cursor.next();
print("\n\n-----------------------------------");
print("Before : url_value : "+x['url_value']);
x['url_value'] = x['url_value'].replace(/^(.*?)\/test\/(.*?)\/(.*?)\.m3u8$/g, $1/test/$2_NEW/$2_NEW.m3u8);
print("After : url_value : "+x['url_value']);
db.table_video.update({_id : x._id}, x);
}
When I execute above command in mongo console, it gives an error : 2015-11-28T12:40:08.342+0530 ReferenceError: $1 is not defined
Any help is greatly appreciated

Your regex is wrong you can get the expected result using this regex1.
But the best way to do this is using "bulk" operations for maximum efficiency.
var bulk = db.table_video.initializeUnorderedBulkOp();
var count = 0;
db.table_video.find().forEach(function(doc) {
var newUrlValue = doc.url_value.replace(/^(.*?)\/test\/(.*?)\/(.*?)\.m3u8$/, '$1/test/$2_NEW/$3_NEW.m3u8');
bulk.find( { '_id': doc._id } ).updateOne( {
'$set': { 'url_value': newUrlValue }
});
count++;
if (count % 100 === 0) {
// Execute per 100 operation and re-init
bulk.execute();
bulk = db.table_video.initializeUnorderedBulkOp();
count = 0;
}
})
// Clean up queues
if (count > 0) bulk.execute();
If your MongoDB version is older that 2.6 you need to use a while loop.
var cursor = db.table_video.find();
while (cursor.hasNext()) {
var x = cursor.next();
print("\n\n-----------------------------------");
print("Before : url_value : "+x['url_value']);
var newUrlValue = x['url_value'].replace(/^(.*?)\/test\/(.*?)\/(.*?)\.m3u8$/, '$1/test/$2_NEW/$2_NEW.m3u8');
print("After : url_value : "+newUrlValue);
db.table_video.update({ _id : x._id }, { '$set': { 'url_value': newUrlValue } } );
}
Regex provided by #Uchiha.

in mongodb version 4.2 you have regexFind project operator which can be used to get the matches, then you ocan use substr to replace parts of the pattern

mongodb mapreduce does incorrect reducing

I'm running into some trouble with a very simple mapreduce, I can't figure out what I've done wrong. I'm trying to merge two collections together, and this first, db.Pos looks like this
"chr" : "chr1", "begin" : 39401, "end" : 39442
The other collection, db.Gene has the following format
"chr" : "chr1", "begin" : 39401, "end" : 39442, "gene" : "GENE1"
My code looks like this:
var mapPos = function(){
emit({chr: this.chr, begin:this.begin, end:this.end},{gene:""});
}
var mapGene = function() {
emit({chr: this.chr, begin:this.begin, end:this.end},{gene:this.gene});
}
r = function(key,values){
var result = {gene:""}
values.forEach(function(value){
result.gene = value.gene;
});
return result;
}
res = db.Pos.mapReduce(mapPos, r, {out: {reduce: 'joined'}});
res = db.Gene.mapReduce(mapGene, r, {out: {reduce: 'joined'}});
So what I'd like to see is a collection where entries that are matching by chr, begin, and end are merged and the gene field is filled in from the db.Gene collection.
Instead, I'm getting the "gene" field in my "joined" collection updated to something other than 0 even when there is no matching doc in db.Gene that has a gene field.
What did I do wrong?

After reflexion, i think you should use merge and not reduce for your out.
The reason why you don't have the good value :
The problem is when the reduce is applied between the joined collection content and the result of the db.Gene.mapReduce.
The function reduce don't know which value is the newest, so the result.gene returned is the last value.gene of the values array.
To distinguish the value that will override the value existing in the collection, you can add a flag.
res = db.Pos.mapReduce(
function() {
emit({chr: this.chr, begin:this.begin, end:this.end},{gene:this.gene || ''});
},
function(key,values){
var result = {};
values.forEach(function(value){
if (value)
result.gene = value.gene;
});
},
{out: {reduce: 'joined'}}
);
res = db.Gene.mapReduce(
function() {
//Add a flag override here
emit({chr: this.chr, begin:this.begin, end:this.end},{gene:this.gene || '', override: true});
},
function(key,values){
var result = {};
values.forEach(function(value){
if (value.override)
result.gene = value.gene;
});
return result;
},
{out: {reduce: 'joined'}}
);
Hope it's clear :)

Pymongo count all documents where a field exists with map reduce

I'm trying to calculate all documents where a certain field exists (in this case it's "country" field) with Map+Reduce, and the only solution that worked for me is this:
mapper = Code("""
function () {
if (typeof this.country != 'undefined') {
var key = 1
emit(key, {count: 1})
};
};
""")
I'm not really interested in keys, just if field exists, so I just passed 1.
But I'm sure that's wrong.
reducer = Code("""
function (key, values) {
var sum = 0;
values.forEach(function (value) {
sum += value['count'];
});
return {count: sum};
};
""")
And then calling map_reduce:
results = dbHandle.cards.map_reduce(mapReduce.mapper, mapReduce.reducer, "resultsMR")
for doc in results.find():
print "Found %s documents." % int(doc.get('value').get('count'))
Also I'm thinking on how to get the amount of docs where their creation date is > than other date, should I use a "query" option in map_reduce function?
query = {"foundationDate":{"$gt":datetime.datetime(2012, 1, 1, 00, 00, 00)}}
Thank you :)

Per Chien-Wei Huang comment, why not use the in built functionality e.g.
db.somName.find({"country":{"$exists":True}}).count()
Else if you want some token map-reduce code you cold simply mimic the count functionality, via the group() function e.g.
db.somName.group(
{ cond: { "country" : {$exists : true} }
, key: { }
, initial: {count : 0}
, reduce: function(doc, out){ out.count++}
}
);
Note: If you also want a count by country value, then stick the following in the key:
, key { "country" : 1 }