We Keep Coding

Filter the populated data and then paginate in Mongodb

Hello I am trying to populate the data and then trying to paginate that data.
Here is the example
Schema A (Users)
{
name: 'Demo',
postId: 'someObjectId',
}
Schema B (Posts)
{
id: 'someObjectId',
postName: 'Post 1',
date: 'date of creation'
}
Here is my code
const users = UserModel.find({
name: 'Demo'
}).populate({
path: 'postId',
select: 'date',
match: {'postId.date' : {$lt: 'today'}}
}).page(pagination).limit(20)
Not getting the result needed. Can someone point out what's wrong?
NOTE: I have just given the overview. Please don't take it as real code. I know I haven't written what we would write in javascript

A populate have following things:
Post.find({})
.populate([
// here array is for our memory.
// because may need to populate multiple things
{
path: 'user',
select: 'name',
model:'User',
options: {
sort:{ },
skip: 5,
limit : 10
},
match:{
// filter result in case of multiple result in populate
// may not useful in this case
}
}
]);
.exec((err, results)=>{
console.log(err, results)
});

Getting decimal from (10:10:00) while trying to load an xlsx excel file with a JavaScript library

I'm using read-excel-file library to load an excel file with the next structure,
The problem is when the code read this value,
The returned value is the next,
And here is my code,
let ExcelLoader = window.readXlsxFile;
let input = document.getElementById('input-file');
//Task Created by Responsible person Status Created on Closed on Deadline Tags
const schema = {
'Task': {
prop: 'task',
type: String
},
'Created by': {
prop: 'createdBy',
type: String
},
'Responsible person': {
prop: 'responsiblePerson',
type: String
},
'Status': {
prop: 'status',
type: String
},
'Created on': {
prop: 'createdOn',
type: String
},
'Closed on': {
prop: 'closedOn',
type: String
},
'Deadline': {
prop: 'deadline',
type: String
},
'Tags': {
prop: 'tags',
type: String
}
};
input.addEventListener('change', () => {
console.log('The Change listener actioned!');
ExcelLoader(input.files[0], { schema })
.then(({rows, errors}) => {
console.log(rows);
//console.log(errors);
})
});
I'm doing this little excersice client side and I was trying to test other types of values in the schema paramether also I was looking for a converter from decimal to date and decimal to datetime with no success,
Can anybody knows what's going on?
I'm open to use other client side libraries...

As Tim Williams says, in Excel Dates and Times are stored as numbers.

Dynamically return config value (node.js)

I'm building a content middleware which gather contents from our external publishers. The publishers will share their contents either in rss or json and the key/value field would be different from each other. To make thing easier, I created a config file where I can pre-defined the key/value and the feed type. The problem is, how can I dynamically return this config value based on publishers name.
Example: To get Publisher #1 feed type, I just can use config.articles.rojak_daily.url_feed
my config file /config/index.js
module.exports = {
batch:100,
mysql: {
database: process.env.database,
host: process.env.host,
username: process.env.username,
password: process.env.password
},
articles:{
rojak_daily:{ // Publisher 1
url: 'xxx',
url_feed: 'rss',
id: null,
Name: 'title',
Description: 'description',
Link: 'link',
DatePublishFrom: 'pubDate',
LandscapeImage: 's3image',
SiteName: 'Rojak Daily',
SiteLogo: null
},
rojak_weekly:{ // publisher 2
url: 'xxx',
url_feed: 'json',
id: null,
Name: 'Name',
Description: 'Desc',
Link: 'link',
DatePublishFrom: 'pubDate',
LandscapeImage: 's3image',
SiteName: 'Rojak Weekly',
SiteLogo: null
}
}
}
my main application script
const config = require('#config'); // export from config file
class Main {
constructor(){
this.publishers = ['rojak_daily','rojak_weekly'];
}
// Main process
async startExport(){
try{
for(let publisher of this.publishers){
const feedType = await this.getFeedType(publisher)
const result = (feedType == 'rss')? await this.rss.start(publisher): await this.json.start(publisher)
return result
}
}catch(err){
console.log("Error occured: ", err)
}
}
// Get feed type from config
async getFeedType(publisher){
return await config.articles.rojak_daily.url_feed;
// this only return publisher 1 url feed.
// my concern is to dynamically passing variable
// into this config file (example: config.articles.<publisher>.url_feed)
}
}
module.exports = Main

async getFeedType(publisher){
return await config.articles[publisher].url_feed;
}
You can access properties of objects by variable

You could either loop over the articles by using Object.entries(articles) or Object.values(articles) in conjunction with Array.prototype.forEach(), or since you already have the name of the publisher you could access that entry with config.articles[publisher].url_feed, like so:
const config = {
articles: {
rojak_daily: { // Publisher 1
url: 'xxx',
url_feed: 'rss',
id: null,
Name: 'title',
Description: 'description',
Link: 'link',
DatePublishFrom: 'pubDate',
LandscapeImage: 's3image',
SiteName: 'Rojak Daily',
SiteLogo: null
},
rojak_weekly: { // publisher 2
url: 'xxx',
url_feed: 'json',
id: null,
Name: 'Name',
Description: 'Desc',
Link: 'link',
DatePublishFrom: 'pubDate',
LandscapeImage: 's3image',
SiteName: 'Rojak Weekly',
SiteLogo: null
}
}
}
const publishers = ['rojak_daily', 'rojak_weekly']
function getFeedType(publisher) {
return config.articles[publisher].url_feed;
}
publishers.forEach(publisher => console.log(getFeedType(publisher)));

Upload Intent function Dialogflow V2

I am trying to develop an API to upload the intent to Dialogflow V2. I have tried below snippet, which it is not working however if trying to communicate with Dialogflow it does work (detect intent)and does get a reply from the Dialogflow for queries.
PERMISSION
I AM & ADMIN > SERVICE ACCOUNTS > DIALOGFLOW ADMIN
ERROR
Error: 7 PERMISSION_DENIED: IAM permission 'dialogflow.entityTypes.create' on 'projects/dexter-47332/agent' denied.
BLOGS/ REFERENCES
Dialogflow easy way for authorization
https://github.com/dialogflow/dialogflow-nodejs-client-v2/blob/master/samples/resource.js#L26
https://www.npmjs.com/package/dialogflow
https://developers.google.com/apis-explorer/
https://cloud.google.com/docs/authentication/production
//------- keys.json (test 1)
{
"type": "service_account",
"project_id": "mybot",
"private_key_id": "123456asd",
"private_key": "YOURKEY",
"client_email": "yourID#mybot.iam.gserviceaccount.com",
"client_id": "098091234",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/yourID%40mybot.iam.gserviceaccount.com"
}
//--------------------- ** (test 2) ** ---------
let privateKey = 'key';
let clientEmail = "email";
let config = {
credentials: {
private_key: privateKey,
client_email: clientEmail
}
}
function createEntityTypes(projectId) {
// [START dialogflow_create_entity]
// Imports the Dialogflow library
const dialogflow = require('dialogflow');
// ******** Instantiates clients (Test 1)********
const entityTypesClient = new dialogflow.EntityTypesClient({
'keyFilename': './keys.json'
});
const intentsClient = new dialogflow.IntentsClient({
'keyFilename': './keys.json'
});
// ******** Instantiates clients (Test 2)********
const entityTypesClient = new dialogflow.EntityTypesClient(config);
const intentsClient = new dialogflow.IntentsClient(config);
// The path to the agent the created entity type belongs to.
const agentPath = intentsClient.projectAgentPath(projectId);
const promises = [];
// Create an entity type named "size", with possible values of small, medium
// and large and some synonyms.
const sizeRequest = {
parent: agentPath,
entityType: {
displayName: 'test',
kind: 'KIND_MAP',
autoExpansionMode: 'AUTO_EXPANSION_MODE_UNSPECIFIED',
entities: [{
value: 'small',
synonyms: ['small', 'petit']
},
{
value: 'medium',
synonyms: ['medium']
},
{
value: 'large',
synonyms: ['large', 'big']
},
],
},
};
promises.push(
entityTypesClient
.createEntityType(sizeRequest)
.then(responses => {
console.log('Created size entity type:');
logEntityType(responses[0]);
})
.catch(err => {
console.error('Failed to create size entity type ----->:', err);
})
);
}
createEntityTypes(projectId);

You can use JWT(JSON Web Tokens) for authenticating with service accounts like in this example
const serviceAccount = { }; // JSON key contents {"type": "service_account",...
const serviceAccountAuth = new google.auth.JWT({
email: serviceAccount.client_email,
key: serviceAccount.private_key,
scopes: 'https://www.googleapis.com/auth/calendar'
});
For more OAuth2.0 scopes for Google APIs you can see the full list here.

I encountered the same error. I corrected it by deleting the current service account and creating a new one and selected the "owner" option for the role.

The associated service-account has to have the role "Dialogflow API Admin" to be able to create intents and entities.

I think you must provide a name parameter there in the sizeRequest and make it equal to an empty string.
Take a look at the code snippet.
let request = {
parent: `projects/${PROJECID}/agent`,
entityType: {
name: '',
autoExpansionMode: 'AUTO_EXPANSION_MODE_DEFAULT',
displayName: 'size_type',
enableFuzzyExtraction: false,
entities: [
{
value: 'Big',
synonyms: ['big', 'large', 'huge']
},
{
value: 'Medium',
synonyms: ['medium', 'not big']
}
],
kind: 'KIND_MAP'
},
languageCode: 'en'
};
Please let me know if this helps.

Keystone JS - Update / seeding / with Relationship

I'm fairly new to KeystoneJS and struggle with pre-populating a database through seeds / updates. I have no problem with independent properties but struggle with properties with relationships.
I have for example a Location Model that includes photos.
var Location = new keystone.List('Location', {
sortable: true,
autokey: {
path: 'slug',
from: 'name',
unique: true
}
});
Location.add({
name: {
type: Types.Text,
required: true,
initial: true
},
photos: {
type: Types.Relationship,
ref: 'Photo',
many: true
}
}
and the Photo Model is as such:
var Photo = new keystone.List('Photo', {
autokey: {
path: 'slug',
from: 'title',
unique: true
}
});
Photo.add({
title: {
type: Types.Text,
initial: true,
index: true
},
image: {
type: Types.CloudinaryImage,
required: true,
initial: false
}
});
Photo.relationship({
ref: 'Location',
path: 'photos',
refPath: 'photos'
});
Within the update folder I'm trying to seed the database with pre-loaded data. Both Location and Photo models get populated individually but I am failing to pre-populate the relationship within both within the Admin UI and lacks knowledge on how to solve. I did quite some research, tried different things such as using __ref and _ids but couldn't make it work. I could not find the answer within the KeystoneJS documentation either. Maybe there is something obvious that I'm actually missing.
exports.create = {
Location: [
{
name: 'London',
photos: [
// <-- how to do it here?
]
},
{
name: 'New York',
photos: [
// <-- how to do it here?
]
}
]
};
Would anyone know the right way to pre-populate KeystoneJs database relationships? Thank you very much.

I managed to solve by mapping through the photos and replace by the actual photo found by title. Here is how I did in case it can help someone else:
exports = module.exports = function (next) {
Promise.all([
{
name: 'London',
photos: ['london_1', 'london_2']
},
{
name: 'New York',
photos: ['new_york_1', 'new_york_2']
}
].map(function (location) {
var _photos = location.photos || [];
location.photos = [];
return Promise.all([
_photos.map(function (title) {
return Photo.model.findOne({ title: title })
.then(function (photo) {
location.photos.push(photo);
});
})
])
.then(function () {
new Location.model(location).save();
});
}))
.then(function () {
next();
})
.catch(next);
};