what is the best way to update many records with different data ?
I'm doing it like this
const updateBody = JSON.parse(req.body);
try {
for (let object of updateBody) {
await prisma.comissions.upsert({
where: {
producer: object.producer,
},
update: {
rate: object.rate,
},
create: object,
});
}
I'm being able to update it, but it's taking a really long time to do so. I'm aware of transaction, but i'm not sure how to use it.
In Prisma transaction query is used in two ways.
Sequential operations: Pass an array of Prisma Client queries to be executed sequentially inside of a transaction.
Interactive transactions: Pass a function that can contain user code including Prisma Client queries, non-Prisma code, and other control flow to be executed in a transaction.
In our case we should use the interactive transaction, Because it contain user code, To use the callback function in the Prisma transaction, we need to add a preview feature to the Prisma.schema file
generator client {
provider = "prisma-client-js"
previewFeatures = ["interactiveTransactions"]
}
prisma.$transaction(async(prisma) => {
try {
for (let object of updateBody) {
await prisma.comissions.upsert({
where: {
producer: object.producer,
},
update: {
rate: object.rate,
},
create: object,
});
}
});
Related
I'm running a Node.js server, connecting to a MongoDB database with mongoose.
Inside my controller, I have several methods that make operations to the database. One of them is this one:
async findMultiple(req, res) {
const [baseSkillsArray] = Array(req.body);
try {
// if there is not baseSkillsArray, skip
if (!baseSkillsArray) {
return res.status(200).send([]);
}
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({
_id: { $in: [baseSkillsArray.baseSkillArray] } //
});
console.log('test ' + allBaseSkills);
res.status(200).send(allBaseSkills);
} catch (error) {
console.error(error.message);
res.status(500).send('Server error find BaseSkills');
}
}
However, this returns me nothing. I did some debugging and I found the reason is the find id $in the array. So I tried hard coding a value, like '2', for instance.
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({ _id: { $in: ['2'] } });
No success. So I went to MongoDB Atlas, where my DB is stored. I tried filtering using the same line of code in my collections.
{ _id: { $in: ['2'] } }
Surprisingly, it returns my document as I wanted!
The issue is that I need to make it work with mongoose. Any ideas? Is this a known bug?
There is nothing wrong with the query, nor a bug regarding $in.
In fact, what's wrong is the actual collection name. I manually created a collection in MongoDB Atlas, called "baseSkills". However, mongoose by default transforms your collection name into lowercase and adds an "s" if your collection's name is not in the plural.
So every time I started my server, I noticed that there was a new collection called "baseskills". I assumed it was a bug and deleted it. Only after making this post that I realized the collection was there again.
So I exported the documents to this collection and my query was working fine.
FYI, there is a way to enforce the collection's name in mongoose. When you declare you model, add a second parameter to the Schema function called "collection". Here is an example:
const BaseSkillSchema = new mongoose.Schema({
_id: {
type: String,
required: true
}, ...
}, { collection: 'baseSkills' })
That's it! Sorry for the mess and thank you for your help!
you want to query over mongo db object ids. So you should create a new ObjectId to do that.
import {Types} from 'mongoose';
{ _id: { $in: [new Types.Object("2")] } }
Or if you have 2 ids one generated and one custom created as id then you can query without creating a new object.
{ id: { $in: ['2'] } }
I have this query in my code which allows me to build a tag cloud for this blog front page
tagCloud:allContentfulBlogPost {
group(field: tags, limit: 8) {
fieldValue
}
}
It's passing data that I map in my component using {data.tagCloud.group.map(tag => (...))};. The code works nicely, but it won't be limited by the filter I'm passing above in the group(fields: tags, limit: 8) in my query. It renders all the tags and not only the first eight.
I've unsuccessfully tried the skip filter as well for the sake of seeing if it works.
Is this the proper way to limit the count to my mapping component in Gatsby?
The Contentful source plugin doesn't define arguments on any of the nodes it creates, unfortunately. Instead you would need to create these yourself. The easiest way to do that is through the createResolvers API.
Here's a similar example from a project of mine:
// in gatsby-node.js
exports.createResolvers = ({ createResolvers }) => {
createResolvers({
SourceArticleCollection: {
// Add articles from the selected section(s)
articles: {
type: ["SourceArticle"],
args: {
// here's where the `limit` argument is added
limit: {
type: "Int",
},
},
resolve: async (source, args, context, info) => {
// this function just needs to return the data for the field;
// in this case, I'm able to fetch a list of the top-level
// entries that match a particular condition, but in your case
// you might want to instead use the existing data in your
// `source` and just slice it in JS.
const articles = await context.nodeModel.runQuery({
query: {
filter: {
category: {
section: {
id: {
in: source.sections.map((s) => s._ref),
},
},
},
},
},
type: "SourceArticle",
})
return (articles || []).slice(0, args.limit || source.limit || 20)
},
},
},
})
}
Because resolvers run as part of the data-fetching routines that support the GraphQL API, this will run server-side at build-time and only the truncated/prepared data will be sent down to the client at request time.
I have a Documents in a Collection that have a field that is an Array (foo). This is an Array of other subdocuments. I want to set the same field (bar) for each subdocument in each document to the same value. This value comes from a checkbox.
So..my client-side code is something like
'click #checkAll'(e, template) {
const target = e.target;
const checked = $(target).prop('checked');
//Call Server Method to update list of Docs
const docIds = getIds();
Meteor.call('updateAllSubDocs', docIds, checked);
}
I tried using https://docs.mongodb.com/manual/reference/operator/update/positional-all/#positional-update-all
And came up with the following for my Server helper method.
'updateAllSubDocs'(ids, checked) {
Items.update({ _id: { $in: ids } }, { $set: { "foo.$[].bar": bar } },
{ multi: true }, function (err, result) {
if (err) {
throw new Meteor.Error('error updating');
}
});
}
But that throws an error 'foo.$[].bar is not allowed by the Schema'. Any ideas?
I'm using SimpleSchema for both the parent and subdocument
Thanks!
Try passing an option to bypass Simple Schema. It might be lacking support for this (somewhat) newer Mongo feature.
bypassCollection2
Example:
Items.update({ _id: { $in: ids } }, { $set: { "foo.$[].bar": bar } },
{ multi: true, bypassCollection2: true }, function (err, result) {
if (err) {
throw new Meteor.Error('error updating');
}
});
Old answer:
Since you say you need to make a unique update for each document it sounds like bulk updating is the way to go in this case. Here's an example of how to do this in Meteor.
if (docsToUpdate.length < 1) return
const bulk = MyCollection.rawCollection().initializeUnorderedBulkOp()
for (const myDoc of docsToUpdate) {
bulk.find({ _id: myDoc._id }).updateOne({ $set: update })
}
Promise.await(bulk.execute()) // or use regular await if you want...
Note we exit the function early if there's no docs because bulk.execute() throws an exception if there's no operations to process.
If your data have different data in the $set for each entry on array, I think you need a loop in server side.
Mongo has Bulk operations, but I don't know if you can call them using Collection.rawCollection().XXXXX
I've used rawCollection() to access aggregate and it works fine to me. Maybe work with bulk operations.
I aggregated some data and published it, but I'm not sure how/where to access the subscribed data. Would I be able to access WeeklyOrders client collection (which is defined as client-only collection i.e WeeklyOrders = new Mongo.Collection(null);)?
Also, I see "self = this;" being used in several examples online and I just used it here, but not sure why. Appreciate anyone explaining that as well.
Here is publish method:
Meteor.publish('customerOrdersByWeek', function(customerId) {
check(customerId, String);
var self = this;
var pipeline = [
{ $match: {customer_id: customerId} },
{ $group: {
_id : { week: { $week: "$_created_at" }, year: { $year: "$_created_at" } },
weekly_order_value: { $sum: "$order_value" }
}
},
{ $project: { week: "$_id.week", year: "$_id:year" } },
{ $limit: 2 }
];
var result = Orders.aggregate(pipeline);
result.forEach(function(wo) {
self.added('WeeklyOrders', objectToHash(wo._id), {year: wo.year, week: wo.week, order_value: wo.weekly_order_value});
});
self.ready();
});
Here is the route:
Router.route('/customers/:_id', {
name: 'customerOrdersByWeek',
waitOn: function() {
return [
Meteor.subscribe('customerOrdersByWeek', this.params._id)
];
},
data: function() { return Customers.findOne(this.params._id); }
});
Here is my template helper:
Template.customerOrdersByWeek.helpers({
ordersByWeek: function() {
return WeeklyOrders.find({});
}
});
You want var self = this (note the var!) so that call to self.added works. See this question for more details. Alternatively you can use the new es6 arrow functions (again see the linked question).
There may be more than one issue where, but in your call to added you are giving a random id. This presents two problems:
If you subscribe N times, you will get N of the same document sent to the client (each with a different id). See this question for more details.
You can't match the document by id on the client.
On the client, you are doing a Customers.findOne(this.params._id) where this.params._id is, I assume, a customer id... but your WeeklyOrders have random ids. Give this a try:
self.added('WeeklyOrders', customerId, {...});
updated answer
You'll need to add a client-only collection as a sort-of mailbox for your publisher to send WeeklyOrders to:
client/collections/weekly-orders.js
WeeklyOrders = new Meteor.Collection('WeeklyOrders');
Also, because you could have multiple docs for the same user, you'll probably need to:
Forget what I said earlier and just use a random id, but never subscribe more that once. This is an easy solution but somewhat brittle.
Use a compound index (combine the customer id + week, or whatever is necessary to make them unique).
Using (2) and adding a customerId field so you can find the docs on the client, results in something like this:
result.forEach(function (wo) {
var id = customerId + wo.year + wo.week;
self.added('WeeklyOrders', id, {
customerId: customerId,
year: wo.year,
week: wo.week,
order_value: wo.weekly_order_value,
});
});
Now on the client you can find all of the WeeklyOrders by customerId via WeeklyOrders.find({customerId: someCustomerId}).
Also note, that instead of using pub/sub you could also just do all of this in a method call. Both are no-reactive. The pub/sub gets you collection semantics (the ability to call find etc.), but it adds the additional complexity of having to deal with ids.
Sorry if I'm not getting the terminology right. Here's what I have currently my MongoDB user docs db.users:
"liked" : [
"EBMKgrD4DjZxkxvfY",
"WJzAEF5EKB5aaHWC7",
"beNdpXhYLnKygD3yd",
"RHP3hngma9bhXJQ2g",
"vN7uZ2d6FSfzYJLmm",
"NaqAsFmMmnhqNbqbG",
"EqWEY3qkeJYQscuZJ",
"6wsrFW5pFdnQfoWMs",
"W4NmGXyha8kpnJ2bD",
"8x5NWZiwGq5NWDRZX",
"Qu8CSXveQxdYbyoTa",
"yLLccTvcnZ3D3phAs",
"Kk36iXMHwxXNmgufj",
"dRzdeFAK28aKg3gEX",
"27etCj4zbrKhFWzGS",
"Hk2YpqgwRM4QCgsLv",
"BJwYWumwkc8XhMMYn",
"5CeN95hYZNK5uzR9o"
],
And I am trying to migrate them to a new key that also captures the time that a user liked the post
"liked_time" : [
{
"postId" : "5CeN95hYZNK5uzR9o",
"likedAt" : ISODate("2015-09-23T08:05:51.957Z")
}
],
I am wondering if it might be possible to simply do this within the MongoDB Shell with a command that iterates over each user doc and then iterates over the liked array and then updates and $push the new postId and time.
Or would it be better to do this in JavaScript. I am using Meteor.
I almost got it working for individual users. But want to know if I could do all users at once.
var user = Meteor.users.findOne({username:"atestuser"});
var userLiked = user.liked;
userLiked.forEach(function(entry) {
Meteor.users.update({ username: "atestuser" },
{ $push: { liked_times: { postId: entry, likedAt: new Date() }}});
console.log(entry);
});
Still a bit of a newbie to MongoDB obviously......
Here is something i made real quick you should run this on the server side just put it into a file e.g. "migrate.js" in root meteor and run the meteor app
if (Meteor.isServer) {
Meteor.startup(function () {
var users = Meteor.users.find().fetch();
users.forEach(function (doc) {
liked.forEach(function (postId) {
Meteor.users.update(doc._id, { $push: { liked_times: { postId: postId, likedAt: new Date() } } });
});
});
console.log('finished migrating');
});
}
p.s I didn't test it
If this is a one time migration i would do something like this in a one time js script.
Get all users
Iterate over each user
Get all likes
Iterate over them, get likedAt
var liked_times = _.collect(likes, function (likeId) {
return {
'postId' : likeId,
'likedAt': // get post liked time from like id.
}
});
Insert the above in the collection of choice.
Note:
The above example makes use of lodash
I would rather just save likedAt as a timestamp.