Related
So the data set looks like this:
screenshot of the data structure
{
"YearWeekISO": "2020-W53",
"FirstDose": 0,
"FirstDoseRefused": "",
"SecondDose": 0,
"DoseAdditional1": 0,
"DoseAdditional2": 0,
"UnknownDose": 0,
"NumberDosesReceived": 0,
"NumberDosesExported": 0,
"Region": "AT",
"Population": "8901064",
"ReportingCountry": "AT",
"TargetGroup": "ALL",
"Vaccine": "JANSS",
"Denominator": 7388778
}, {
"YearWeekISO": "2020-W53",
"FirstDose": 0,
"FirstDoseRefused": "",
"SecondDose": 0,
"DoseAdditional1": 0,
"DoseAdditional2": 0,
"UnknownDose": 8,
"NumberDosesReceived": 0,
"NumberDosesExported": 0,
"Region": "AT",
"Population": "8901064",
"ReportingCountry": "AT",
"TargetGroup": "ALL",
"Vaccine": "UNK",
"Denominator": 7388778
},
link to the data set
The query parameters will look like :
GET /vaccine-summary?c=AT&dateFrom=2020-W10&dateTo=2020-W53&range=5
where
c, country code to get report for
dateFrom, yyyy-Www, eg. 2020-W10 (Including)
dateTo, yyyy-Www, eg, 2020-W20 (Excluding)
rangeSize, number, eg, the period for which to calculate metrics
After applying the aggregation, you should have a transformed data set that looks like :
{
"summary": [{
"weekStart": "2020-W10",
"weekEnd": "2020-W15",
"NumberDosesReceived": 1000
},
{
"weekStart": "2020-W15",
"weekEnd": "2020-W20"
"NumberDosesReceived": 2000
}, …
till end of range(dateTo)
]
}
}
Notice how the weekStart incremental from 2020-W10 to 2020-W15, similar with weekEnd.
NumberDosesReceived is the sum of NumberDosesReceived fileld within that range
So was able to come up with a working solution using a mongo aggregate method called bucket, but one of the problem is that if you want an aggregation of like week 1 - week 20 in chunks of 5, i.e, 1-5 (1 included, 5 excluded), 5- 10,10-15 and 15-20, you will have to give it an array like; boundaries: [1,5,10,15,20] as part of the argument and from the question, i have to create a JS function to return an array of numbers between start week and end week with the range given also. Written in typescript, the return array from this question would look like : [2020-W01,2020-W05,2020-W10,2020-W15,2020-W20], Also there are certain edge cases you have to account for since all the parameters are dynamic, like if the week spans more than one year, also, the fact that mongo to the best of my knowledge don't have date format like "2020-W10" makes it a bit more complex
export function customLoop(
startWeek: number,
endWeek: number,
rangeNum: number,
year: number
): returnData {
const boundaryRange: string[] = [];
let skip = 0;
for (let i = startWeek; i <= endWeek; i += rangeNum) {
const currentNum: string = i < 10 ? `0${i}` : `${i}`;
const result = `${year}-W${currentNum}`;
boundaryRange.push(result);
//if all the weeks in a year, Check where the last loop stops to determine skip
if (endWeek === 53 && i + rangeNum > 53) {
skip = i + rangeNum - 53 - 1;
}
}
return {
skip,
theRange: boundaryRange,
};
}
After this i opened my mongo compass on local to construct and chain aggregate methods and function to satisfy the task given:
const result = await VaccinationModel.aggregate([
{
$match: {
ReportingCountry: c,
},
},
{
$bucket: {
groupBy: "$YearWeekISO",
boundaries: [...boundaryRange],
default: "others",
output: {
NumberDosesReceived: {
$sum: "$NumberDosesReceived",
},
},
},
},
{
$addFields: {
range: rangeNum,
},
},
{
$addFields: {
weekStart: "$_id",
weekEnd: {
$function: {
body: "function(id,range) {\n const arr = id.split('-')\n const year = arr[0];\n let week;\n let result=''\n if(arr[1]){\n week = arr[1].slice(1);\n result=`${year}-W${Number(week) + range}`\n\n }\n\n return result\n }",
args: ["$_id", "$range"],
lang: "js",
},
},
},
},
{
$unset: ["_id", "range"],
},
{
$match: {
weekEnd: {
$ne: "",
},
},
},
{
$sort: {
weekStart: 1,
},
},
]);
In that aggregation:
Match the country code.
I basically called the bucket aggregation with the array of boundaries, then summing the results of each chunk/range using its NumberDosesReceived field while naming it NumberDosesReceived.
since i needed extra two fields to complete the number of fields to return, namely weekStart and weekEnd that isnt in the dataset, the weekStart is the _id field from the bucket aggregation, to get the weekEnd, i added the range as a field.
If for instance the current mongo iteration is 2020-W5, which would be the in the _id, that means the weekend would be 5 + range = 10, so i used the mongo function method to extract that passing _id and range as argument.
Used the unset method to remove the _id and range field as it wouldn't be part of the return data.
Get this new weekEnd field excluding empty ones.
sort using it.
here is the link to the repo: link
It should work
const YearWeekISO = { $toDate: "$YearWeekISO" };
{
$project: {
fiveWeekperiod: {
$subtract: [
{ $week: YearWeekISO },
{ $mod: [{ $week: YearWeekISO }, 5] },
],
},
date: YearWeekISO,
NumberDosesReceived: 1,
},
},
{
$group: {
_id: {
year: { $year: "$date" },
fiveWeek: "$fiveWeekperiod",
},
weekStart: { $min: "$date" },
weekEnd: { $max: "$date" },
NumberDosesReceived: { $sum: "$NumberDosesReceived" },
},
},
{
$project: {
_id: 0,
weekStart: {
$dateToString: {
date: "$weekStart",
format: "%G-W%V",
},
},
weekEnd: {
$dateToString: {
date: {
$dateAdd: {
startDate: "$weekEnd",
unit: "week",
amount: 1,
},
},
format: "%G-W%V",
},
},
NumberDosesReceived: 1,
},
}
my Test config file is not able to find the spec file. i already have spec files created. previously i ran those scripts also.. but for more framework update i updated the paths of the files.. that makes spec files error and even i gave the full path also it is not able to find the spec file.
Spec File
var utilityInit,page2//browser2;
page1=new facebook(firstBrowser);
module.exports=function(){
this.Given(/^Open the browser and Load the URL$/,async function(){
await firstBrowser.get(properties.get("url1"));
browser.logger.info("Title of the window is :"+await browser.getTitle());
//screenshots.takesScreenshot("filename");
});
this.When(/^User entered the text in the search box$/,async function(){
firstBrowser.sleep(3000);
await page1.email().sendKeys(testData.Login.CM[0].Username);
browser.sleep(3000);
await page1.password().sendKeys(testData.Login.CM[0].Password);
});
this.Then(/^click on login button$/,async function(){
browser.sleep(3000);
await facebook.submit().click();
});
this.Then(/^User tried to open in new browser instance$/,async function(){
browser2=await openNewBrowser.newBrowserInit(firstBrowser);
utilityInit=new utility(browser2);
utilityInit.ignoreSync(properties.get("url2"));
browser2.manage().window().maximize();
console.log(await utilityInit.title()+" title");
browser2.sleep(5000);
});
this.When(/^User entered the text in the email field$/,async function(){
page2=new facebook(browser2);
console.log(await page2.title()+" browser2");
await page2.search().sendKeys("testing");
browser2.sleep(3000);
page1=new facebook(firstBrowser);
console.log(await page1.title()+" browser1")
await page1.email().sendKeys(testData.Login.CM[0].Username);
screenshots.takeScreenshot("newScreenshot");
firstBrowser.sleep(5000);
})
}
Config file
const log4js = require('log4js');
var fs=require('fs');
global.screenshots = require('protractor-take-screenshots-on-demand');
global.browser2;
var propertiesReader=require('properties-reader');
exports.config = {
//seleniumAddress: 'http://localhost:4444/wd/hub',
directConnect:true,
framework: 'custom',
// path relative to the current config file
frameworkPath: require.resolve('protractor-cucumber-framework'),
capabilities: {
'browserName': 'chrome',
metadata: {
browser: {
name: 'chrome',
version: '79'
},
device: 'MacBook Pro 15',
platform: {
name: 'OSX',
version: '10.12.6'
},
disableLog:true,
durationInMS:true,
openReportInBrowser:true
}
},
ignoreUncaughtExceptions:false,
// Spec patterns are relative to this directory.
specs: [
'../Proc/src/test/java/com/proc/features/test.feature'
],
beforeLaunch:function(){
if (fs.existsSync('./logs/ExecutionLog.log')) {
fs.unlink('./logs/ExecutionLog.log')
}
log4js.configure({
appenders: {
out: { type: 'console' },
info:{ type: 'dateFile', filename: '../Reports/logs/info', "pattern":"-dd.log",alwaysIncludePattern:false},
"console" : {
"type": "console",
"category": "console"
},
"file" : {
"category": "test-file-appender",
"type": "file",
"filename": "../Reports/logs/log_file.log",
"maxLogSize": 10240,
// "backups": 3,
// "pattern": "%d{dd/MM hh:mm} %-5p %m"
}
},
categories: {
"info" :{"appenders": ["console"], "level": "info"},
"default" :{"appenders": ["console", "file"], "level": "DEBUG"},
//"file" : {"appenders": ["file"], "level": "DEBUG"}
}
});
},
cucumberOpts: {
require:['../src/test/resources/com.proc.utility/timeOutConfig.js','../src/test/java/com/proc/TestCases/spec.js'],
tags: false,
profile: false,
format:'json:../Reports/jsonResult/results.json',
'no-source': true
},
onPrepare: function () {
const logDefault = log4js.getLogger('default');
const logInfo=log4js.getLogger('info');
screenshots.browserNameJoiner = ' - '; //this is the default
//folder of screenshot
screenshots.screenShotDirectory = '../Screenshots';
global.openNewBrowser=require("../src/test/resources/com.proc.utility/newBrowserinstance.js")
global.testData=require("../TestData/testData.json");
browser.logger = log4js.getLogger('protractorLog4js');
global.firstBrowser=browser;
global.properties=propertiesReader("../TestData/propertyConfig.properties");
browser.waitForAngularEnabled(false);
browser.manage().window().maximize();
global.facebook=require("../src/test/java/com/proc/pages/fbPageObjects.js");
global.utility=require("../src/test/resources/com.proc.utility/testFile.js");
},
plugins: [{
package: '../Proc/node_modules/protractor-multiple-cucumber-html-reporter-plugin',
options:{
// read the options part for more options
automaticallyGenerateReport: true,
removeExistingJsonReportFile: true,
reportPath:"../Reports/HtmlReports",
reportName:"test.html"
},
customData: {
title: 'Run info',
data: [
{label: 'Project', value: 'Framework Setup'},
{label: 'Release', value: '1.2.3'},
{label: 'Cycle', value: 'Test Cycle'}
]
},
}]
};
Log
H:\workspace\Proc\Configuration>protractor testConfig.js
[23:38:00] I/launcher - Running 1 instances of WebDriver
[23:38:00] I/direct - Using ChromeDriver directly...
DevTools listening on ws://127.0.0.1:51680/devtools/browser/e9688f83-2047-4535-91ba-bf2100fe6016
0 scenarios
0 steps
0m00.000s
can some one please help me to clear this issue.. this is related to my project framework POC..
To clarify some other comments I have seen being made by others here, you do not have to give the exact file path. You can do
specs: [
'../Proc/src/test/java/com/proc/features/']
This path does not look correct to me ^ are you sure that is the correct path? also you have a javascript project but your file structure is a java structure src/test/java why?
it's a simple thing that makes my framework works..
npm install cucumber#1.3.3 --save-dev
This because of the Spec path
specs: [
'../Proc/src/test/java/com/proc/features/test.feature' //Provide a complete path to your test file along with file extension(.ts)
],
Hope it helps you
I'm getting an error on this elastic search for terms. The error message is
"[parsing_exception] [terms] unknown token [START_ARRAY] after [activeIds], with { line=1 & col=63 }"
Active Ids is an array of unique ids. It sort of looks like
const activeIds = [ '157621a1-d892-4f4b-80ca-14feddb837a0',
'd04c5c93-a22c-48c3-a3b0-c79a61bdd923',
'296d40d9-f316-4560-bbc9-001d6f46858b',
'2f8c6c37-588d-4d24-9e69-34b6dd7366c2',
'ba0508dd-0e76-4be8-8b6e-9e938ab4abed',
'ab076ed9-1dd5-4987-8842-15f1b995bc0d',
'ea6b0cff-a64f-4ce3-844e-b36d9f161e6f' ]
let items = await es.search({
"index": table,
"body": {
"from": 0, "size": 25,
"query": {
"terms" : {
"growerId" : {
activeIds
}
},
"bool": {
"must_not": [
{ "match":
{
"active": false
}
},
],
"must": [
{ "query_string" :
{
"query": searchQuery,
"fields": ["item_name"]
}
}
],
}
}
}
})
Appreciate the help!
Edit: Answering this question- "What's the expected result? Can you elaborate and share some sample data? – Nishant Saini 15 hours ago"
I'll try to elaborate a bit.
1) Overall I'm trying to retrieve items that belong to active users. There are 2 tables: user and items. So I'm initially running an ES that returns all the users that contain { active: true } from the user table
2) Running that ES returns an array of ids which I'm calling activeIds. The array looks like what I've already displayed in my example. So this works so far (let me know if you want to see the code for that, but if I'm getting an expected result then I don't think we need that now)
3) Now I want to search through the items table, and retrieve only the items that contain one of the active ids. So an item should look like:
4) expected result is retrieve an array of objects that match the growerId with one of the activeIds. So if I do a search query for "flowers", a single expected result should look like:
[ { _index: 'items-dev',
_type: 'items-dev_type',
_id: 'itemId=fc68dadf-21c8-43c2-98d2-cf574f71f06d',
_score: 11.397207,
_source:
{ itemId: 'fc68dadf-21c8-43c2-98d2-cf574f71f06d',
'#SequenceNumber': '522268700000000025760905838',
item_name: 'Flowers',
grower_name: 'Uhs',
image: '630b5d6e-566f-4d55-9d31-6421eb2cff87.jpg',
dev: true,
growerId: 'd04c5c93-a22c-48c3-a3b0-c79a61bdd923',
sold_out: true,
'#timestamp': '2018-12-20T16:09:38.742599',
quantity_type: 'Pounds',
active: true,
pending_inventory: 4,
initial_quantity: 5,
price: 10,
item_description: 'Field of flowers' } },
So here the growerId matches activeIds[1]
But if I do a search for "invisible", which is created by a an inactive user, I get:
[ { _index: 'items-dev',
_type: 'items-dev_type',
_id: 'itemId=15200473-93e1-477c-a1a7-0b67831f5351',
_score: 1,
_source:
{ itemId: '15200473-93e1-477c-a1a7-0b67831f5351',
'#SequenceNumber': '518241400000000004028805117',
item_name: 'Invisible too',
grower_name: 'Field of Greens',
image: '7f37d364-e768-451d-997f-8bb759343300.jpg',
dev: true,
growerId: 'f25040f4-3b8c-4306-9eb5-8b6c9ac58634',
sold_out: false,
'#timestamp': '2018-12-19T20:47:16.128934',
quantity_type: 'Pounds',
pending_inventory: 5,
initial_quantity: 5,
price: 122,
item_description: 'Add' } },
Now that growerId does not match any of the ids in activeIds.
5) Using the code you helped with, it's returning 0 items.
Let me know if you need more detail. I've been working on this for a bit too long :\
Terms query accept array of terms so the terms query should be defined as below:
"terms": {
"growerId": activeIds
}
You might face other errors as well after making the above correction. So below is full query which might help you:
{
"from": 0,
"size": 25,
"query": {
"bool": {
"must_not": [
{
"match": {
"active": false
}
}
],
"must": [
{
"query_string": {
"query": searchQuery,
"fields": [
"item_name"
]
}
},
{
"terms": {
"growerId": activeIds
}
}
]
}
}
}
I am creating google slides with the help of javascript API. I need to set up the page size But its not working. Below is my code .
var size = {
width:{
magnitude: 612,
unit: "PT"
},
height: {
magnitude: 792,
unit: 'PT'
}};
gapi.client.slides.presentations.create({
title: "Some title",
pageSize: size
}).then((response) => {
var presentationId = response.result.presentationId;
})
Use this Method: presentations.create
Creates a new presentation using the title given in the request. Other
fields in the request are ignored. Returns the created presentation.
Here is the request body:
{
"presentationId": string,
"pageSize": {
object(Size)
},
"slides": [
{
object(Page)
}
],
"title": string,
"masters": [
{
object(Page)
}
],
"layouts": [
{
object(Page)
}
],
"locale": string,
"revisionId": string,
"notesMaster": {
object(Page)
},
}
To check the description of pageSize, you can refer to the fields table. It was specified as the size of pages in the presentation.
Also, you may want to check this javascript Browser Quickstart to help you further.
i have a runningPnL field and description field as well as a date field.
I want to do:
if the description is "Breakup" then multipliedPnL=2*runningPnL
otherwise multipliedPnL=runningPnL
my code is:
db.getCollection('user0').aggregate(
[
{
$project:
{
thedate: 1,
realPnL:
{
$cond: { if: { $gte: [ "$description", "Breakup" ] }, then: {$multiply:[$runningPnL,2]}, else: $runningPnL}
}
}
}
]
)
Theres a reference error : runningPnL not defined.
WHats wrong with my code?
thx a lot in advance for your help!
Can you try $mul? What is your mongodb version?
Reference: https://docs.mongodb.com/manual/reference/operator/update/mul/
An example for the documentation is like this
db.products.update(
{ _id: 1 },
{ $mul: { price: 1.25 } }
)