We Keep Coding

Bulk patch operation #azure/cosmos javascript sdk

I have a container of cost guides in my Azure Cosmos DB (using the core SQL API). Each cost guide has an array of materials. I need to add a material to this array in every document in the container. Is this possible with javascript in a single transaction? I am familiar with partially updating documents individually using the patch operation but I would prefer to do it all at once if possible. I'm using the #azure/cosmos version 3.15 package
This is how I update individual documents in my function app:
const CosmosClient = require('#azure/cosmos').CosmosClient;
const config = require('../config/config');
const { endpoint, key, databaseId } = config;
const client = new CosmosClient({ endpoint, key });
const database = client.database(databaseId);
module.exports = async function (context, req) {
const containerId = req.query.containerId;
const container = database.container(containerId);
const id = req.query.id;
const updates = req.body;
const querySpec = {
query: `SELECT * from c where c.id = "${id}"`
}
const { resources: items } = await container.items
.query(querySpec)
.fetchAll()
const patchOp = [];
// loop through updates object
Object.keys(updates).map(key => {
patchOp.push({
op: 'replace',
path: `/${key}`,
value: updates[key]
})
})
const { resource: patchSource } = await container.item(items[0].id, items[0].id).patch(patchOp);
}

Technically, till now no such type of single transaction using Java Script is available. There are other possibilities like using .NET which are used for similar requirements.
Other languages like JAVA and PYTHON are having partial implementation. Terraform can also help a bit in partial implementation.
https://learn.microsoft.com/en-us/azure/cosmos-db/sql/sql-api-sdk-bulk-executor-dot-net

The closest I have seen is using the bulk or batch operation on items within a container. For example:
const operations = [
{
operationType: "Create",
resourceBody: { id: "doc1", name: "sample", key: "A" }
},
{
operationType: "Upsert",
partitionKey: 'A',
resourceBody: { id: "doc2", name: "other", key: "A" }
}
];
await database.container.items.batch(operations);
Link to azure documentation: https://learn.microsoft.com/en-us/javascript/api/#azure/cosmos/items?view=azure-node-latest##azure-cosmos-items-batch

javascript/nodejs - how to manipulate Json Object by language

I am using nodejs for the server.
Currently I have a Json in my project folder.
name.json
{
"name_English": "Apple",
"name_German": "Apfel",
"name_French": "Pomme"
}
When I send request to server, it returns:
GET http://localhost:3030/name
{
"name_English": "Apple",
"name_German": "Apfel",
"name_French": "Pomme"
}
But I found it is not convenient for frontend development.
Is there any way to do something like below?
GET http://localhost:3030/name?lang=en
{
"name": "Apple"
}
GET http://localhost:3030/name?lang=fr
{
"name": "Apfel"
}
Edit 1
The code of getting the Json in Feathers.js
name.class.js
const nameLists = require('../name.json')
exports.Test = class Test {
constructor (options) {
this.options = options || {};
}
async find (params) {
return nameLists
}
};
Edit 2
Is it possible to make name.json like this?
{
"name": ${name}
}
Edit 3
The reason I want to achieve above because I have to return whole Json file.
For the internationalization library, it seems needed to handle outside the JSON and I don't know what is the best practise to do so.

Here's a full demonstration with just express. (Hope that's ok.)
const express = require('express');
const app = express();
const port = 3030;
const nameLists = require('./name.json');
const unabbreviateLanguage = {
en: 'English',
de: 'German',
fr: 'French'
}
function filterByLanguage(obj, abbr) {
let language = unabbreviateLanguage[abbr];
let suffix = '_' + language;
let result = {};
for (let key in obj) {
if (key.endsWith(suffix)) {
let newkey = key.slice(0, -suffix.length);
result[newkey] = obj[key];
}
}
return result;
}
app.get('/name', (req, res) => {
res.json(filterByLanguage(nameLists, req.query.lang));
});
app.listen(port);
e.g.:
curl http://localhost:3030/name?lang=de
output:
{"name":"Apfel"}
The idea is to iterate over the keys of the input object and prepare an output object that only has keys that match the language suffix (then strip off that suffix). You'll likely want to either have a mapping of en -> English, or just use the key names that match the parameter e.g., rename name_English to name_en.

In FeathersJS, the params object of find will store the query string of the passed in URL. So if you call http://localhost:3030/name?lang=en, the params object will be :-
{
query: {
lang: 'en'
}
}
You can then use this information to determine which result from the JSON to return.
https://docs.feathersjs.com/guides/basics/services.html#service-methods

Your question appears to be two parts: handling queries, and handling the internationalization.
Handling the query:
Feathers presents queries through the context object at the following location:
context.params.query
Handling Internationalization:
There are many solid packages available for handling internationalization:
https://openbase.com/categories/js/best-nodejs-internationalization-libraries?orderBy=RECOMMENDED&

Relation not found error in multi-row insert with postgresql and pg-promise

I'm developing a nodejs project with a postgresql-db.
I've created a schema for the db with some tables and now I've created a file with some prepared data (multilple rows) to insert. For that I followed the instructions of this entry: Multi-row insert with pg-promise.
The problem is, that I get following error: { error: relation "tenantx.ObjectGroup" does not exist ... but it does. I can also see that from my api, when I want to read the table content, it returns an empty object.
Can the "schema.table"-declaration be the reason?
The code for the prepared data looks like this:
'use strict';
const pgp = require('pg-promise')({
// Initialization Options
});
const csObjectGroup = new pgp.helpers.ColumnSet(['objectgroup', 'description'], {
table: 'tenantx.ObjectGroup'
});
// data input values:
const valuesObjectGroup = [{
objectgroup: 'bla',
description: 'bla'
}, {
objectgroup: 'blu',
description: 'blu'
}, {
objectgroup: 'bla',
description: 'bla'
}];
module.exports = {
csObjectGroup: csObjectGroup,
valuesObjectGroup: valuesObjectGroup
}
And the code which is called from api to setup content:
// insert prepared data into tables
function initializeData(pTenantId, dbUri) {
var dbPostgres = dbUri;
const datafile = require("./../Data/data_" + pTenantId);
var statements = [];
var valuesObjectGroup = datafile.valuesObjectGroup;
var csObjectGroup = datafile.csObjectGroup;
statements.push(valuesObjectGroup, csObjectGroup);
var i, query;
for (i = 0; i < statements.length - 1; i += 2) {
query = pgp.helpers.insert(statements[i], statements[i + 1]);
dbPostgres.none(query)
.then(data => {
console.log("+++++ Data successfully initialized.");
})
.catch(err => {
console.log("----- Data could not be initialized.");
console.log(err);
});
}
return true;
}
I've created the table from this:
const createObjectGroupTable =
'CREATE TABLE IF NOT EXISTS tenantx.ObjectGroup \
( \
id serial, \
objectgroup varchar(50), \
description varchar(100), \
PRIMARY KEY (id) \
)';

If you look at the reported error with more attention:
{ error: relation "tenantx.ObjectGroup" does not exist ...
That's not "tenantx"."ObjectGroup", that's just one table name "tenantx.ObjectGroup".
And the problem originates from the way you declare the table:
const csObjectGroup = new pgp.helpers.ColumnSet(['objectgroup', 'description'], {
table: 'tenantx.ObjectGroup'
});
instead of providing schema + table names, you specify it all as a table name, and so it is escaped accordingly.
The correct way to specify schema + table can be done in one of the following ways:
table: {schema: 'tenantx', table: 'ObjectGroup'}
or:
table: new pgp.helpers.TableName('ObjectGroup', 'tenantx')
or:
table: new pgp.helpers.TableName({table: 'ObjectGroup', schema: 'tenantx'})
See API: TableName.

Mongoose infinite loop on Model creation

Background
I have Mongoose Schema about Surveys, that needs to check if the Survey belongs to a set of countries that is in another collection.
Code
To check this, I have a surveySchema, a countrySchema, and a file where I create the models and connect to the DB.
To perform the check that a survey belongs to a valid country, I am using Mongoose async validators in surveySchema like the following:
surveySchema.js:
"use strict";
const mongoose = require("mongoose");
const surveySchema = {
subject: { type: String, required: true },
country: {
type: String,
validate: {
validator: {
isAsync: true,
validator: async function(val, callback) {
const {
Country
} = require("./models.js").getModels();
const countriesNum = await Country.find({"isoCodes.alpha2": val}).count();
callback(countriesNum === 1);
}
},
message: "The country {VALUE} is not available in the DB at the moment."
}
}
};
module.exports = new mongoose.Schema(surveySchema);
module.exports.surveySchema = surveySchema;
countrySchema.js:
"use strict";
const mongoose = require("mongoose");
const countrySchema = {
name: { type: String, required: true },
isoCodes:{
alpha2: { type: String, required: true }
}
}
};
module.exports = new mongoose.Schema(countrySchema);
module.exports.countrySchema = countrySchema;
models.js:
"use strict";
const mongoose = require("mongoose");
const fs = require("fs");
const DB_CONFIG = "./config/dbConfig.json";
/**
* Module responsible for initializing the Models. Should be a Singleton.
*/
module.exports = (function() {
let models;
const initialize = () => {
//Connect to DB
const {
dbConnectionURL
} = JSON.parse(fs.readFileSync(DB_CONFIG, "utf8"));
mongoose.connect(dbConnectionURL);
mongoose.Promise = global.Promise;
//Build Models Object
models = {
Country: mongoose.model('Country', require("./countrySchema.js")),
Survey: mongoose.model('Survey', require("./surveySchema.js"))
};
};
const getModels = () => {
if (models === undefined)
initialize();
return models;
};
return Object.freeze({
getModels
});
}());
The idea here is that I am using the models.js file in other places as well. Because this file is also responsible for connecting to the DB, I decided to make it a Singleton. This way, I should only connect once, and all further requests will always return the same Models, which would be ideal.
Problem
The problem here is that I have a circular dependency that results in:
RangeError: Maximum call stack size exceeded at exports.isMongooseObject (/home/ubuntu/workspace/server/node_modules/mongoose/lib/utils.js:537:12)
...
The flow of code leading to this error is:
Code runs getModels()`
getModels() checks that models is undefined and runs initialize()
initialize() tries to create the models.
When creating the Survey model Survey: mongoose.model('Survey', require("./surveySchema.js")) it runs into the validator function, which again requires models.js
Infinite loop begins
Questions
Is there any other way to check if a Survey's country is part of the county's collection without making a async validation?
How can I structure/change my code so this doesn't happen?

As said in the comments, I think you are a bit confused about how you are using your models.js module. I think this is what is happening:
You are exporting a single function from models.js:
models.js
module.exports = function() { ... };
Therefore, when you require it, you just get that single function:
surveySchema.js
const models = require("./models.js");
models is now a function. Which means every time you call it, you run through the code in models.js and create a new variable let models;, and also new functions initialize() and getModels().
You could move the let models out of the anonymous function into the global scope which would probably fix it, but for my money you only want to run the anonymous function in models.js once, so I would invoke it immediately and set the exports of the module to its result:
models.js
module.exports = (function() {
// codez here
return Object.freeze({
getModels
});
})(); // immediately invoke the function.
Use it:
// models is now the frozen object returned
const { Survey } = models.getModels();
As for options to validation, there's no reason why you can't add your own middleware validation code if normal async validation doesn't do it for you using serial or parallel mechanisms as described in the docs.
Update after comments
The second problem as you point out is that during first execution of getModels() -> initialize() you call require('./surveySchema.js'), but this calls getModels() which is still in the process of being called and hasn't yet initialized models, so initialize() is re-entered.
I think what you're trying to achieve is fine (survey schema depends on customer model), because you can still draw an object graph without any circular dependencies, and it's just the way you've implemented it that you've ended up with one. The simplest way to deal with this I think is actually to keep the circular reference, but defer the point at which you call getModels() in surveySchema.js:
"use strict";
const mongoose = require("mongoose");
const models = require("./models.js");
const surveySchema = {
subject: { type: String, required: true },
country: {
type: String,
validate: {
validator: {
isAsync: true,
validator: async function(val, callback) {
// !! moved from the global scope to here, where we actually use it
const {
Country
} = models.getModels();
const countries = await Country.find({"isoCodes.alpha2": val});
callback(countries.length === 1);
}
},
message: "The country {VALUE} is not available in the DB at the moment."
}
}
};
module.exports = new mongoose.Schema(surveySchema);
module.exports.surveySchema = surveySchema;
A neater and probably more extensible way of approaching it, though, might be to separate out the connection code from the models code, since it's a different concept altogether.
Update #2 after more comments
The infinite stack you're seeing is because you have not used the API correctly. You have:
const surveySchema = {
country: {
validate: {
validator: {
isAsync: true,
validator: async function(val, callback) {...}
},
},
...
}
};
You should have:
const surveySchema = {
country: {
validate: {
isAsync: true,
validator: async function(val, callback) {...}
},
...
}
};
As per the docs.

mongoose save vs insert vs create

What are different ways to insert a document(record) into MongoDB using Mongoose?
My current attempt:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var notificationsSchema = mongoose.Schema({
"datetime" : {
type: Date,
default: Date.now
},
"ownerId":{
type:String
},
"customerId" : {
type:String
},
"title" : {
type:String
},
"message" : {
type:String
}
});
var notifications = module.exports = mongoose.model('notifications', notificationsSchema);
module.exports.saveNotification = function(notificationObj, callback){
//notifications.insert(notificationObj); won't work
//notifications.save(notificationObj); won't work
notifications.create(notificationObj); //work but created duplicated document
}
Any idea why insert and save doesn't work in my case? I tried create, it inserted 2 document instead of 1. That's strange.

The .save() is an instance method of the model, while the .create() is called directly from the Model as a method call, being static in nature, and takes the object as a first parameter.
var mongoose = require('mongoose');
var notificationSchema = mongoose.Schema({
"datetime" : {
type: Date,
default: Date.now
},
"ownerId":{
type:String
},
"customerId" : {
type:String
},
"title" : {
type:String
},
"message" : {
type:String
}
});
var Notification = mongoose.model('Notification', notificationsSchema);
function saveNotification1(data) {
var notification = new Notification(data);
notification.save(function (err) {
if (err) return handleError(err);
// saved!
})
}
function saveNotification2(data) {
Notification.create(data, function (err, small) {
if (err) return handleError(err);
// saved!
})
}
Export whatever functions you would want outside.
More at the Mongoose Docs, or consider reading the reference of the Model prototype in Mongoose.

You can either use save() or create().
save() can only be used on a new document of the model while create() can be used on the model. Below, I have given a simple example.
Tour Model
const mongoose = require("mongoose");
const tourSchema = new mongoose.Schema({
name: {
type: String,
required: [true, "A tour must have a name"],
unique: true,
},
rating: {
type: Number,
default:3.0,
},
price: {
type: Number,
required: [true, "A tour must have a price"],
},
});
const Tour = mongoose.model("Tour", tourSchema);
module.exports = Tour;
Tour Controller
const Tour = require('../models/tourModel');
exports.createTour = async (req, res) => {
// method 1
const newTour = await Tour.create(req.body);
// method 2
const newTour = new Tour(req.body);
await newTour.save();
}
Make sure to use either Method 1 or Method 2.

I'm quoting Mongoose's Constructing Documents documentation:
const Tank = mongoose.model('Tank', yourSchema);
const small = new Tank({ size: 'small' });
small.save(function (err) {
if (err) return handleError(err);
// saved!
});
// or
Tank.create({ size: 'small' }, function (err, small) {
if (err) return handleError(err);
// saved!
});
// or, for inserting large batches of documents
Tank.insertMany([{ size: 'small' }], function(err) {
});

TLDR: Use Create (save is expert-mode)
The main difference between using the create and save methods in Mongoose is that create is a convenience method that automatically calls new Model() and save() for you, while save is a method that is called on a Mongoose document instance.
When you call the create method on a Mongoose model, it creates a new instance of the model, sets the properties, and then saves the document to the database. This method is useful when you want to create a new document and insert it into the database in one step. This makes the creation an atomic transaction. Therefore, the save method leaves the potential to create inefficiencies/inconsistencies in your code.
On the other hand, the save method is called on an instance of a Mongoose document, after you have made changes to it. This method will validate the document and save the changes to the database.
Another difference is that create method can insert multiple documents at once, by passing an array of documents as parameter, while save is intended to be used on a single document.
So, if you want to create a new instance of a model and save it to the database in one step, you can use the create method. If you have an existing instance of a model that you want to save to the database, you should use the save method.
Also, if you have any validation or pre-save hook in your content schema, this will be triggered when using the create method.