Bulk patch operation #azure/cosmos javascript sdk - javascript

I have a container of cost guides in my Azure Cosmos DB (using the core SQL API). Each cost guide has an array of materials. I need to add a material to this array in every document in the container. Is this possible with javascript in a single transaction? I am familiar with partially updating documents individually using the patch operation but I would prefer to do it all at once if possible. I'm using the #azure/cosmos version 3.15 package
This is how I update individual documents in my function app:
const CosmosClient = require('#azure/cosmos').CosmosClient;
const config = require('../config/config');
const { endpoint, key, databaseId } = config;
const client = new CosmosClient({ endpoint, key });
const database = client.database(databaseId);
module.exports = async function (context, req) {
const containerId = req.query.containerId;
const container = database.container(containerId);
const id = req.query.id;
const updates = req.body;
const querySpec = {
query: `SELECT * from c where c.id = "${id}"`
}
const { resources: items } = await container.items
.query(querySpec)
.fetchAll()
const patchOp = [];
// loop through updates object
Object.keys(updates).map(key => {
patchOp.push({
op: 'replace',
path: `/${key}`,
value: updates[key]
})
})
const { resource: patchSource } = await container.item(items[0].id, items[0].id).patch(patchOp);
}

Technically, till now no such type of single transaction using Java Script is available. There are other possibilities like using .NET which are used for similar requirements.
Other languages like JAVA and PYTHON are having partial implementation. Terraform can also help a bit in partial implementation.
https://learn.microsoft.com/en-us/azure/cosmos-db/sql/sql-api-sdk-bulk-executor-dot-net

The closest I have seen is using the bulk or batch operation on items within a container. For example:
const operations = [
{
operationType: "Create",
resourceBody: { id: "doc1", name: "sample", key: "A" }
},
{
operationType: "Upsert",
partitionKey: 'A',
resourceBody: { id: "doc2", name: "other", key: "A" }
}
];
await database.container.items.batch(operations);
Link to azure documentation: https://learn.microsoft.com/en-us/javascript/api/#azure/cosmos/items?view=azure-node-latest##azure-cosmos-items-batch

Related

How to set Opentelemetry - NodeJS - Auto instrumentation service name?

I have the following config set in tracing.js (1:1 from the documentation) and I can't figure out how to set the service name. Right now in Datadog all I see is "unknown_service".
const opentelemetry = require("#opentelemetry/sdk-node");
const { getNodeAutoInstrumentations } = require("#opentelemetry/auto-instrumentations-node");
const { OTLPTraceExporter } = require("#opentelemetry/exporter-trace-otlp-http");
const { diag, DiagConsoleLogger, DiagLogLevel } = require('#opentelemetry/api');
//How can I set custom service name?
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);
const sdk = new opentelemetry.NodeSDK({
traceExporter: new OTLPTraceExporter({
url: process.env.OPENTELEMETRY_URL,
headers: {},
}),
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
There are a couple of ways to do that:
First adding it to your code:
const opentelemetry = require("#opentelemetry/sdk-node");
const { getNodeAutoInstrumentations } = require("#opentelemetry/auto-instrumentations-node");
const { OTLPTraceExporter } = require("#opentelemetry/exporter-trace-otlp-http");
const { Resource } = require('#opentelemetry/resources');
const { SemanticResourceAttributes } = require('#opentelemetry/semantic-conventions');
const { diag, DiagConsoleLogger, DiagLogLevel } = require('#opentelemetry/api');
//How can I set custom service name?
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);
const sdk = new opentelemetry.NodeSDK({
traceExporter: new OTLPTraceExporter({
url: process.env.OPENTELEMETRY_URL,
headers: {},
}),
instrumentations: [getNodeAutoInstrumentations()],
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: '<service-name>',
}),
});
sdk.start();
Another way is using the environment variable OTEL_SERVICE_NAME as recommended by #Jan Garaj:
OTEL_SERVICE_NAME=<service-name>
At last, you can also use the environment variable OTEL_RESOURCE_ATTRIBUTES:
OTEL_RESOURCE_ATTRIBUTES=service.name=<service-name>
Those are the official OpenTelemetry ways of naming a service, IDK if they work with the backend you are using.
That works with OSS tools like Jager and Zipkin.
Try to set general-purpose environment variable OTEL_SERVICE_NAME.
Doc: https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/

javascript/nodejs - how to manipulate Json Object by language

I am using nodejs for the server.
Currently I have a Json in my project folder.
name.json
{
"name_English": "Apple",
"name_German": "Apfel",
"name_French": "Pomme"
}
When I send request to server, it returns:
GET http://localhost:3030/name
{
"name_English": "Apple",
"name_German": "Apfel",
"name_French": "Pomme"
}
But I found it is not convenient for frontend development.
Is there any way to do something like below?
GET http://localhost:3030/name?lang=en
{
"name": "Apple"
}
GET http://localhost:3030/name?lang=fr
{
"name": "Apfel"
}
Edit 1
The code of getting the Json in Feathers.js
name.class.js
const nameLists = require('../name.json')
exports.Test = class Test {
constructor (options) {
this.options = options || {};
}
async find (params) {
return nameLists
}
};
Edit 2
Is it possible to make name.json like this?
{
"name": ${name}
}
Edit 3
The reason I want to achieve above because I have to return whole Json file.
For the internationalization library, it seems needed to handle outside the JSON and I don't know what is the best practise to do so.
Here's a full demonstration with just express. (Hope that's ok.)
const express = require('express');
const app = express();
const port = 3030;
const nameLists = require('./name.json');
const unabbreviateLanguage = {
en: 'English',
de: 'German',
fr: 'French'
}
function filterByLanguage(obj, abbr) {
let language = unabbreviateLanguage[abbr];
let suffix = '_' + language;
let result = {};
for (let key in obj) {
if (key.endsWith(suffix)) {
let newkey = key.slice(0, -suffix.length);
result[newkey] = obj[key];
}
}
return result;
}
app.get('/name', (req, res) => {
res.json(filterByLanguage(nameLists, req.query.lang));
});
app.listen(port);
e.g.:
curl http://localhost:3030/name?lang=de
output:
{"name":"Apfel"}
The idea is to iterate over the keys of the input object and prepare an output object that only has keys that match the language suffix (then strip off that suffix). You'll likely want to either have a mapping of en -> English, or just use the key names that match the parameter e.g., rename name_English to name_en.
In FeathersJS, the params object of find will store the query string of the passed in URL. So if you call http://localhost:3030/name?lang=en, the params object will be :-
{
query: {
lang: 'en'
}
}
You can then use this information to determine which result from the JSON to return.
https://docs.feathersjs.com/guides/basics/services.html#service-methods
Your question appears to be two parts: handling queries, and handling the internationalization.
Handling the query:
Feathers presents queries through the context object at the following location:
context.params.query
Handling Internationalization:
There are many solid packages available for handling internationalization:
https://openbase.com/categories/js/best-nodejs-internationalization-libraries?orderBy=RECOMMENDED&

Stuck in generating the right LEFT join queries with count using the Sequelize models in the nodejs app

I'm using Sequelize in my NodeJS app with Postgres as database. Following is my correct SQL query which gives me right set of data while running them on database server :-
SELECT
"expertises"."uuid" AS "id",
"expertises"."display_name" AS "name",
"expertises"."modified_by" AS "modifiedBy",
(
SELECT COUNT(expertise_uuid)
FROM expertise_endorsements
WHERE expertise_uuid = "expertises"."uuid"
) AS "users"
FROM "expertises" AS "expertises"
LEFT JOIN "expertise_endorsements" AS "expertise_endorsements"
ON "expertise_endorsements"."expertise_uuid" = "expertises"."uuid"
LIMIT '16' OFFSET '1';
However when I run the app, the ORM is generating the wrong SQL query without any count (as shown below):-
SELECT
"expertises".*,
"expertise_endorsements"."id" AS "expertise_endorsements.id",
"expertise_endorsements"."expertise_uuid" AS "expertise_endorsements.expertise_uuid"
FROM
(
SELECT
"expertises"."id",
"expertises"."uuid" AS "id",
"expertises"."display_name" AS "name",
"expertises"."modified_by" AS "modifiedBy",
"expertises"."uuid"
FROM
"expertises" AS "expertises"
ORDER BY
"expertises"."name" DESC LIMIT '16' OFFSET '1'
)
AS "expertises"
LEFT OUTER JOIN
"expertise_endorsements" AS "expertise_endorsements"
ON "expertises"."uuid" = "expertise_endorsements"."expertise_uuid"
ORDER BY
"expertises"."name" DESC;
This is my model association :-
const consts = require("../services/static/constants");
const Expertises = require("./models/expertises.model");
const ExpertisesEndorsees = require("./models/expertises_endorsees.model");
const ExpertisesEndorsements = require("./models/expertise_endorsements.model");
exports.setAssociations = (db, Sequelize, type) => {
const expertisesModel = Expertises(db, Sequelize);
const expertisesEndorseesModel = ExpertisesEndorsees(db, Sequelize);
const expertisesEndorsementsModel = ExpertisesEndorsements(db, Sequelize);
expertisesModel.hasMany(expertisesEndorsementsModel, { sourceKey: consts.uuidField, foreignKey: consts.expertiseUuid, as: consts.modelAliases.EXPERTISE_ENDORSEMENTS });
expertisesEndorsementsModel.belongsTo(expertisesModel, { foreignKey: consts.uuidField, as: consts.modelAliases.EXPERTISE_ENDORSEMENTS });
return { expertisesModel, expertisesEndorseesModel, expertisesEndorsementsModel };
};
My logic is below. I'm stuck in understanding and applying the count properly using the Sequelize:-
const models = dbUtils.setAssociations(db, consts.getExpertiseFlow);
const includesData = [
{
model: models.expertisesEndorsementsModel,
attributes: [consts.expertiseUuid],
// attributes: [
// sequelize.fn("COUNT", sequelize.col(`${consts.modelAliases.EXPERTISE_ENDORSEMENTS}.${consts.get_endorsee_uuid}`)), "users"
// ],
as: consts.modelAliases.EXPERTISE_ENDORSEMENTS,
required: false
}
];
let {count: expertiseCount, rows: expertises} = JSON.parse(JSON.stringify(
await manageExpertise.getSortedExpertises(models, consts.get_expertises, pageOffset, pageLimit, orderBy, includesData, false)));
Method:-
exports.getSortedExpertises = (models, attributes, offset, limit, orderBy, includes, raw=true) => {
return models.expertisesModel
.findAndCountAll({
attributes,
include: includes,
offset: offset,
limit: limit,
//order: orderBy,
raw
}).then(data => {
return data;
})
.catch(err => {
throw err;
});
};
Would really appreciate if you can assist me in resolving this issue and let me know the root cause. Thank you

Working gremlin javascript example

There is a new version out, but the documentation is somewhat lacking a working example.
Github Ticket: https://github.com/jbmusso/gremlin-javascript/issues/109
I've been trying to get an example to work. Any help appreciated:
gremlin-server: 3.3.2 with config gremlin-server-modern.yaml
npm gremlin lib: 3.3.2
import gremlin from 'gremlin';
import DriverRemoteConnection from 'gremlin/lib/driver/driver-remote-connection';
import { Graph } from 'gremlin/lib/structure/graph';
const graph = new Graph()
const g = graph.traversal().withRemote(new DriverRemoteConnection('ws://localhost:8182/gremlin', { mimeType: 'application/vnd.gremlin-v3.0+json' }));
const fetchById = async (id) => {
const result = await g.V(id).toList()
console.log(result);
}
const addUser = async (name) => {
const newVertex = await g.addV().property('name','marko').property('name','marko a. rodriguez').next()
console.log(newVertex)
}
addUser()
fetchById(0)
Current Output:
[]
{ value: null, done: true }
UPDATE
Gremlin JavaScript now supports GraphSON3 and the latest Gremlin Server.
Working example:
const gremlin = require('gremlin');
const Graph = gremlin.structure.Graph;
const DriverRemoteConnection = gremlin.driver.DriverRemoteConnection;
Obtain a traversal source (g):
const graph = new Graph();
const connection = new DriverRemoteConnection('ws://localhost:8182/gremlin');
const g = graph.traversal().withRemote(connection);
Once you have a traversal source (g), reuse it across your application to create traversals, for example:
// Get the friends' names
const friends = await g.V().has("name","Matt")
.out("knows").values("name").toList();
See more information on the documentation: https://tinkerpop.apache.org/docs/current/reference/#gremlin-javascript
ORIGINAL ANSWER
Gremlin JavaScript doesn't support GraphSON3 serialization, which is the default in TinkerPop 3.3+. This causes your response to not be properly parsed.
I've filed a ticket to support GraphSON3 in the JavaScript GLV: https://issues.apache.org/jira/browse/TINKERPOP-1943
In the meantime, as a workaround, you can add GraphSON2 serializers to the server by adding the following line to your yaml, below serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV2d0] }}
Regarding the read property 'reader' of undefined issue. I falled back to version gremlin#3.3.4 and it works fine.

Ampersand fetch children/collection with parent

What is the appropriate way to load a child model property of an Ampersand model when doing a fetch on the collection?
I have a Task object:
{
"id": 1
"projectId": 2 // child entity id
"name": "This task for project 1"
}
And I have created an Ampersand model:
var AmpModel = require('ampersand-model');
var Project = require('../project');
module.exports = AmpModel.extend({
props: {
id: ['number'],
projectId: ['number'],
name: ['string']
},
children: {
project: Project
}
});
And a collection:
var AmpCollection = require('ampersand-rest-collection');
var Task = require('./task');
module.exports = AmpCollection.extend({
model: Task,
url: '/api/task'
});
My API is very simple, as the app only stores input in memory:
var _ = require('lodash');
var store = require('../data'); // this is a file with some init data
function get(id) {
return _.findWhere(store.tasks.data, { id: parseInt(id + '', 10) });
}
exports.list = function (req, res) {
res.send(store.tasks.data);
};
exports.add = function (req, res) {
var item = req.body;
item.id = store.tasks.id++;
store.tasks.data.push(item);
res.status(201).send(item);
};
exports.get = function (req, res) {
var found = get(req.params.id);
res.status(found ? 200 : 404);
res.send(found);
};
In Entity Framework I would have included the child entity when retrieving. The projectId field would map to the projects table and the framework would take care of it for me.
After reading the doco and some help in the JavaScript chat room, I realise I have been looking at this architecture the wrong way.
My API should read the tasks data source and use the retrieved project ID to load the project data. The project object would be assigned to the projects property on the task and the whole task object returned. Ampersand will then take over and assemble my model as expected.

Categories