I have an IndexedDB table that follows accepts following structured JSON as a row:
{
id : 1,
name : 'doc1',
createdDate : '2018-08-08'
}
I want to get count for each available date in the table. ie: groupby:date then count. Expected example output is in the format of:
{
'2018-08-08' : 5,
'2018-08-18' : 19,
...
}
Table contains large number of records. So, how can efficiently achieve this requirement using Dexie?
If you index createdDate,
const db = new Dexie("yourDB");
db.version(1).stores({
yourTable: "id, name, createdDate"
});
Then you could do the following:
const result = {}
await db.yourTable.orderBy('createdDate').eachKey(date => {
result[date] = (result[date] || 0) + 1;
});
Related
Building a script in google apps script.
I get values from an invoice data sheet with multiple lines per invoice so as to account for line items.
My progress so far has been to extract individual invoice numbers from the column (each invoice number occurs as many line items the individual invoice has).
The array todaysInvoices looks like this: [35033817, 35033818, 35033819, 35033820, 35033821]
Now, I need a way to create an object for each of these invoice numbers that has different properties (such as invoiceDate and customerName etc.). The initial invoice number as in the array should thereby be assigned as 'id' property to the new invoice object.
I need help to use objects in javascript.
If you require additional information, please let me know.
Below is a screenshot of a simplified version of my order sheet:
This is a clipping of my order sheet. Before and after the shown columns there are many more with more details but the hierarchies of information are already in the image
Below is the code I have so far:
const orderSheet = SpreadsheetApp.openById('SPREADSHEETID').getSheetByName('SHEETNAME');
const invoiceTemplate = DriveApp.getFileById('DOCUMENTID');
const tempFolder = DriveApp.getFolderById('FOLDERID');
const invoiceData = orderSheet.getRange(4,7, orderSheet.getLastRow() - 1, 57).getDisplayValues().filter(function (rows){ return rows[0] === 'INVOICED'});
const invDataRepo = SpreadsheetApp.openById('SPREADSHEETID2');
var timestamp = new Date();
function printBulkInvoices() {
logLineItems ();
var todaysInvoices = uniqueInvIDs ();
todaysInvoices.sort();
todaysInvoices.map(String);
//fetchInvData (todaysInvoices);
Logger.log (todaysInvoices)
}
function fetchInvData (invoiceIDs) {
let invoices = {
}
Logger.log(invoices)
invoiceIDs.forEach
}
function fetchLineItems (invoiceDataArray) {
}
// send array of todays unique invoice numbers (later all inv data?) to invdata sheet and log them
function logTodaysInvoices (invIDArr){
invIDArr.forEach
invDataRepo.getSheetByName('invdata').getRange(invDataRepo.getSheetByName('invdata').getLastRow()+1,1,invIDArr.length,1).setValue(invIDArr);
}
// return an array of unique invoice ids from todays invoice data
function uniqueInvIDs (){
let singleArray = invoiceData.map(row => row[5]);
let unique = [...new Set(singleArray)];
return unique;
}
//log incoicedata to invdatarepo-sheet 'lineitems'
function logLineItems (){
invDataRepo.getSheetByName('lineitems').getRange(invDataRepo.getSheetByName('lineitems').getLastRow()+1,2,invoiceData.length,invoiceData[0].length).setValues(invoiceData);
}
It's hard to say exactly what you need since we cannot see your Invoice Data Sheet.
But here's something that might give you a start:
let iobj = {idA:[]};
[35033817, 35033818, 35033819, 35033820, 35033821].forEach((id => {
if(!iobj.hasOwnProperty(id)) {
iobj[id]={date: invoiceDate, name: customName, items:[]};
iobj.idA.push(id);//I find it handy to have an array of object properties to loop through when I wish to reorganize the data after it's all collected
} else {
iobj[id].items.push({item info properties});//I am guessing here that you may wish to addition additional information about the items which are on the current invoice
}
});
Javascript Object
To follow up from your question:
Your loop to collect object data would start to look something like this:
function getInvoiceData() {
const ss = SpreadsheetApp.getActive();
const ish = ss.getSheetByName('Invoice Data');
const isr = 2;
const hA = ish.getRange(1, 1, 1, ish.getLastColumn()).getValues()[0];
let idx = {};//object return head index into row array based on header title which in this case I assume invoice number is labeled 'Invoicenumber'
hA.forEach((h, i) => {idx[h] = i});
const vs = ish.getRange(isr, 1, ish.getLastRow() - isr + 1, ish.getLastColumn()).getValues();
let iobj = { idA: [] };
vs.forEach(r => {
if (!iobj.hasOwnProperty(r[idx['invoicenumber']])) {
iobj[r[idx['invoicenumber']]] = { date: r[idx['invoicedate']], name: r[idx['customername']], items: [] };
iobj.idA.push(r[idx['invoicenumber']]);
} else {
iobj[r[idx['invoicenumber']]].items.push({ iteminfoproperties:'' });
}
});
}
if I have data in my db similar to this
/values:
value1: somevalue1
value2: somevalue2
specialValue1 : 1
specialValue2 : 2 //to be "removed" and rest values moved
specialValue3 : 3
specialValue4 : 4
how can I do "array splice" like operation, to get rid of specialValue2 and move the rest, so I get:
value1: somevalue1
value2: somevalue2
specialValue1 : 1
specialValue2 : 3
specialValue3 : 4
There is no splice-like operation in Firebase Realtime Database. You will have to read all the data, modify it in your application code, and write it back to the database.
In pure JavaScript the object manipulation would look like this:
var input = {
specialValue1 : 1,
specialValue2 : 2, //to be "removed" and rest values moved
specialValue3 : 3,
specialValue4 : 4
};
var keys = Object.keys(input);
var offset = 0;
var output = {};
keys.forEach((key, index) => {
if (input[key] != 2) {
output[keys[index-offset]] = input[key];
}
else {
offset = offset + 1;
}
});
console.log(output);
For instance I have some JSON data like below (The JSON data is just an example, I just want to give out some fake, make up and wrong format JSON as example)
cata :[{
name:test1,
data:['abc1, abc2' , 'abc3,abc4']
}
name:test2,
data:['abc5, abc6' , 'abc7,abc8']
}]
And indeed I need to render it to frontend, therefore I made a new object and try to push data into it
var name = "";
var key= [];
for(var i=0;i<2;i++){
name .push(cata[i].name)
key.push(cata[i].data.join(' + '));
}
var rehandle = {
name : name,
key : key
}
The above is just how i do it now, and which do no provide the desire result, i want to know how could i restore it so i can change the format from
['abc5, abc6' , 'abc7,abc8']
to
abc5+abc6 , abc7+abc8
UPDATE version of the question:
I think i better explain it step by step:
I have some raw data
I have a row of "data" in each set of data
(E.g:data:['abc1, abc2' , 'abc3,abc4'])
I want to change it's format to abc1+abc2 , abc3+abc4 and store it to another variable
I will pass the variable store abc1+abc2 , abc3+abc4 to an object
5.Render it one by one in a table
UPDATE 2
I have seen #pill's answer, am i able to render the data like
for(var i=0;i<cata.length;i++){
var trythis = trythis + '<td>'+name[i]+'</td>' + '<td>'+data[i]+'</td>'
}
To format your data from
['abc5, abc6' , 'abc7,abc8']
to
abc5+abc6 , abc7+abc8
you'd simply use
data.map(k => k.split(/,\s*/).join('+')).join(' , ')
or the ES5 version
data.map(function(k) {
return k.split(/,\s*/).join('+');
}).join(' , ');
For example...
var cata = [{"name":"test1","data":["abc1, abc2","abc3,abc4"]},{"name":"test2","data":["abc5, abc6","abc7,abc8"]}];
var rehandle = cata.reduce(function(o, d) {
o.name.push(d.name);
o.key.push(d.data.map(function(k) {
return k.split(/,\s*/).join('+');
}).join(' , '));
return o;
}, {
name: [],
key: []
});
console.log('rehandle:', rehandle);
Note that I had to fix up your data formatting
I'm new to Lodash and I'm trying to perform a complex sum with group by as SQL but I don't find any solution. I have tried to use/combine multiple Lodash functions without success.
My requirement is like this. I have a JSON response:
input =
[{"quantity":1067,"gross_revenue":4094.2,"date":"03","company":"Cat1","product":"Car"},
{"quantity":106,"gross_revenue":409,"date":"02","company":"Cat2","product":"Car"},
{"quantity":106,"gross_revenue":85,"date":"03","company":"Cat2","product":"House"},
{"quantity":106,"gross_revenue":100,"date":"02","company":"Cat3","product":"House"},
{"quantity":20,"gross_revenue":150,"date":"03","company":"Cat5","product":"Technology"},
{"quantity":40,"gross_revenue":100,"date":"01","company":"Cat5","product":"Technology"},
{"quantity":20,"gross_revenue":15,"date":"01","company":"Cat5","product":"Car"},
{"quantity":20,"gross_revenue":18,"date":"01","company":"Cat5","product":"House"},
{"quantity":20,"gross_revenue":2,"date":"01","company":"Cat2","product":"House"},
{"quantity":20,"gross_revenue":25,"date":"01","company":"Cat3","product":"House"}]
I need to generate a result as below to populate the series for a HighChart:
[{ name: 'Car', data: [15, 409, 4094.2] },
{ name: 'House', data:[45, 100, 85] },
{ name: 'Techonology', data:[100, null, 150] }]
Those values are the result from:
Make a group by using Product with the tag name
Based on following procedure, generate an array with the tag data
2.1 Sum the gross revenue based on Product and date (all existing dates)
2.2 Include a null value if there doesn't exist gross revenue for any existing day
2.3 Sort the results for gross revenue based on date, ascending order
Is this possible? Or is there another solution for this?
Thanks.
Here's one way to do it - certainly not the only solution...
var input = [
{"quantity":1067,"gross_revenue":4094.2,"date":"03","company":"Cat1","product":"Car"},
{"quantity":106,"gross_revenue":409,"date":"02","company":"Cat2","product":"Car"},
{"quantity":106,"gross_revenue":85,"date":"03","company":"Cat2","product":"House"},
{"quantity":106,"gross_revenue":100,"date":"02","company":"Cat3","product":"House"},
{"quantity":20,"gross_revenue":150,"date":"03","company":"Cat5","product":"Technology"},
{"quantity":40,"gross_revenue":100,"date":"01","company":"Cat5","product":"Technology"},
{"quantity":20,"gross_revenue":15,"date":"01","company":"Cat5","product":"Car"},
{"quantity":20,"gross_revenue":18,"date":"01","company":"Cat5","product":"House"},
{"quantity":20,"gross_revenue":2,"date":"01","company":"Cat2","product":"House"},
{"quantity":20,"gross_revenue":25,"date":"01","company":"Cat3","product":"House"}
];
var result = [];
var groupedByProduct = _.groupBy(input, "product");
// get the set of unique dates
var dates = _.uniq(_.map(input, 'date'));
// for each product, perform the aggregation
_.forEach(groupedByProduct, function(value, key) {
// initialize the data array for each date
data = [];
for (var i = 0; i < dates.length; i++) {
data.push(null);
}
// aggregate gross_revenue by date
_.forEachRight(_.groupBy(groupedByProduct[key], "date"), function(dateValue, dateKey) {
// use the date as an array index
data[parseInt(dateKey) - 1] = _.sumBy(dateValue, function(o) {
return o.gross_revenue
});
});
// push into the result array
result.push({"name": key, "data": data});
});
document.getElementById("result").innerHTML = JSON.stringify(result);
<script src="https://cdn.jsdelivr.net/lodash/4.11.1/lodash.min.js"></script>
<pre id="result"></pre>
I'm trying to calculate all documents where a certain field exists (in this case it's "country" field) with Map+Reduce, and the only solution that worked for me is this:
mapper = Code("""
function () {
if (typeof this.country != 'undefined') {
var key = 1
emit(key, {count: 1})
};
};
""")
I'm not really interested in keys, just if field exists, so I just passed 1.
But I'm sure that's wrong.
reducer = Code("""
function (key, values) {
var sum = 0;
values.forEach(function (value) {
sum += value['count'];
});
return {count: sum};
};
""")
And then calling map_reduce:
results = dbHandle.cards.map_reduce(mapReduce.mapper, mapReduce.reducer, "resultsMR")
for doc in results.find():
print "Found %s documents." % int(doc.get('value').get('count'))
Also I'm thinking on how to get the amount of docs where their creation date is > than other date, should I use a "query" option in map_reduce function?
query = {"foundationDate":{"$gt":datetime.datetime(2012, 1, 1, 00, 00, 00)}}
Thank you :)
Per Chien-Wei Huang comment, why not use the in built functionality e.g.
db.somName.find({"country":{"$exists":True}}).count()
Else if you want some token map-reduce code you cold simply mimic the count functionality, via the group() function e.g.
db.somName.group(
{ cond: { "country" : {$exists : true} }
, key: { }
, initial: {count : 0}
, reduce: function(doc, out){ out.count++}
}
);
Note: If you also want a count by country value, then stick the following in the key:
, key { "country" : 1 }