So the document contains an array of objects, each object containing it's own array. So how would I go about updating one of the elements in the array that's inside the object which is inside another array. I've read some things with $. But I don't understand completely how to use it to call a position. I know the position of the element. But I can't just say $[] because the position is defined in a variable and not a string...
I've tried doing a simple
db.collection.findOne({...}, (err, data) => {...});
and then changing the arrays in the objects in the array in there with a simple:
data.arr[x].type[y] = z; data.save().catch(err => {console.log(err)});
But it doesn't save the new values I set for for the element of the array.
Sample structure after proposed solution from #Tom Slabbaert:
Data.findOne({
userID: 'CMA'
}, (err, doc) => {
if(err) {console.log(err)}
if(doc) {
for(var i = 0; i<CMA.stockMarket.length; i++) {
if(CMA.stockMarket[i].name == data.userID) {
for(var z = 0; z<CMA.stockMarket[i].userStock.length; z++) {
if(z == company) {
var updateAmount = CMA.stockMarket[i].userStock[z]+args[1]
var updateKey = `stockMarket.${i}.userStock.${z}`
Data.updateOne({userID: 'CMA'}, {'$set': {[updateKey]: updateAmount}})
}
}
}
}
}
});
-------------------------EDIT-------------------------
So I tried changing some things around in the data base to see if that would fix the problem I was having. I modified the updated code that was provided by #Tom Slabbaert. But nothing seems to work for some reason :/ Here's what I have so far, at this point I hope it's just a syntax error somewhere. Cause this is really frustrating at this point. Note that I'm still using the for loops here to find if the info exists. And if not, push that info into the database. This might only be temporary until I find a better way / if there is a better way.
for(var i = 0; i<CMA.userStocks.length; i++) {
if(CMA.userStocks[i].name == data.userID) {
for(var z = 0; z<CMA.userStocks[i].shares.length; z++) {
//console.log(CMA.userStocks[i].shares[z].companyName)
if(CMA.userStocks[i].shares[z].companyName == args[0]) {
var updateKey = `CMA.userStocks.$[elem1].shares.$[elem2].amount`
Data.updateOne(
{userID: 'CMA'},
{
"$inc": {
[updateKey]: args[1]
}
},
{
arrayFilters: [
{
"elem1.name": data.userID,
"elem2.companyName": args[0]
}
]
}
)
purchaseComplete(); return;
}
}
CMA.userStocks[i].shares.push({companyName: args[0], amount: parseInt(args[1])})
CMA.save().catch(err => {console.log(err)});
purchaseComplete(); return;
}
}
CMA.userStocks.push({name: data.userID, shares: [{companyName: args[0], amount: parseInt(args[1])}]});
CMA.save().catch(err => {console.log(err)});
purchaseComplete(); return;
The data I'm trying to find and change is structured like the following:
And what I'm trying to change in the end is the 'amount' (which is an integer)
_id: (Not relavent in this question)
userID: 'CMA'
stockMarket: [...] (Not relavent in this question)
userStocks: [
Object: (position 0 in userStocks array)
name: 'string' (equal to data.userID in the code)
shares: [
Object: (position 0 in shares array)
companyName: 'string' (this is args[0] in the code)
amount: integer
]
]
You can just prepare the "key" ahead of time. like so:
const updateKey = `arr.${x}.type.${y}`
db.collection.updateOne(
{...},
{
"$set": {
[updateKey]: z
}
})
Mongo Playground
Using Mongo's positional operators ($ and $[]) are usually required when you don't know the position in the array and want to use a condition to update the element.
------ EDIT-----
After given your sample code you just have a minor syntax error:
var updateKey = `stockMarket.${i}.userStock.${z}`
Should just be:
var updateKey = `CMA.stockMarket.${i}.userStock.${z}`
However After seeing your code I recommend you execute the following solution which uses a single update with arrayFilters, it just cleans up the code quite a bit:
const updateKey = `CMA.stockMarket.$[elem1].userStock.${company}`;
db.collection.update(
{userID: 'CMA'},
{
"$inc": {
[updateKey]: args[1]
}
},
{
arrayFilters: [
{
"elem1.name": data.userID
}
]
})
Mongo Playground
Well I found something that worked. Apparently it didn't save the db.collection.updateMany unless I made a .then() function on the end? I have no idea why, but it's the same with an aggregate I made. (It basically does the same as a Data.findOne and save it too, but it isn't limited by the parallel save error)
Solution I found with aggregation:
<collection field> = <new data for collection field>
Data.aggregate([
{
$match: { //This is used to create a filter
['<insert field>']: <insert filter>
}
}, {
$addFields: { //This is used to update existing data, or create a new field containing the data if the field isn't found
['<collection field>']: <new data for collection field>
}
}, {
$merge: { //This is used to merge the new data / document with the rest of the collection. Thus having the same effect as a standard save
into: {
db: '<insert database name>',
coll: '<insert collection name>'
}
}
}
]).then(() => {
//After it's done, do something here. Or do nothing at all it doesn't matter as long as the .then() statement remains. I found that not having this part will break the code and make it not save / work for some reason.
}); return;
Solution I found with db.collection.updateMany
db.collection.updateMany(
{<insert field>: filter}, {$set: {'<insert field>': <new data>}}
).then(() => {
//This .then() statment in my case was needed for the updateMany function to work correctly. It wouldn't save data without it for some reason, it does not need to contain any actual info in this part. As long as it's here.
});
With this new info I could simply access and change the data that I was trying to before using the previous instructions provided by #Tom Slabbaert and my new method of actually making it save the changes made into the document.
I've decided on splitting my environments keeping them in .js files in an environment folder and keep all the sensitive information in .env file (use a third-party module 'DOTENV')
That's what I've come up with but I understand that it's not the best practice and there are a lot of things which should have been implemented in a completely different way but I just lack experience and practice.
At first, I tried to use as more " for loop " as it's possible because as far as I know, it's the fastest way to loop through an object, but in some cases, it was much easier to with "map or filter".
It doesn't look nice to assign data by returning a Promise. Maybe there is a way to get data without a Promise?
I would appreciate any suggestions on how the code can be improved and good practices, your experience.
And I am not sure if I used logging right and error handling. That's a completely new thing for me at the moment, but I used "try catch" to catch them and simply logged them on the console and put into a file.
code:
import { readdirSync } from 'fs';
import path from "path";
import { logger } from '../src/utils/logging';
import { merge } from "lodash";
// FIXME: Function returns a Promise with the data.
// It's not comfortable and seem a bad practice - too much code for a simple task,
// and deal with a promise what may outcome in decreasing perfomance
// ( The simplest code, the fastest code )
export let env = getEnvironment().then(
res => { return res },
err => logger.error(err)
);
// TODO: Rewrite this function into a class Environment to keep it organized and implement ES6 standart
async function getEnvironment() {
const mode = process.env.NODE_ENV || 'development';
const rootPath = process.cwd();
const folder = 'environment';
const loadEnvironments = () => {
// Getting the list of available environments in the "environment" folder,
// at the same time excluding index.js file
const list = readdirSync(path.join(rootPath, folder)).filter(file => !/(?=^(index.js))/i.test(file));
const parameters = {};
// Loading the files found in the folder,
// merging them with the help of a "lodash" library
// just to get one common Object with all possible parameters from all found environments
const loaded = list.map(fileName => {
let name = fileName.split('.')[0];
let loadedFile = require(path.join(rootPath, folder, fileName));
const file = loadedFile[name];
merge(parameters, { ...file });
return loadedFile;
});
// Picking the currect mode out of already loaded ones
const current = { ...loaded.filter(file => file[mode]).map(file => file[mode])[0] };
// Returning an object with all parameters
return {
parameters,
current
}
};
const environments = loadEnvironments();
const environment = {} = looping(environments.parameters, environments.current);
function looping(obj, values) {
const collection = {};
for (const key in obj) {
if (obj.hasOwnProperty(key)) {
if (typeof obj[key] !== 'object') {
try {
if (values.hasOwnProperty(key)) {
// By a recursive function run through all parameters,
// transforming the keys to uppercased,
// assigning value to 'obj' (file containing all the parameters)
// from the current mode
collection[key.toUpperCase()] = values[key];
} else {
// if there is no such a key in the current mode,
// 'null' is assigned
collection[key.toUpperCase()] = null;
}
} catch (e) {
logger.error(` Missing parameter "${key.toUpperCase()}" in ${mode} mode!!!`);
}
} else {
// Recursing through the object and the nested objects
collection[key.toUpperCase()] = looping(obj[key], values[key]);
}
}
}
return collection;
}
// When parameters are ready,
// the current mode is assigned
environment["MODE"] = mode;
return environment;
}
I've a module which is using parsing ( a parsing functionality), other modules should query this parser values.
my question is
how should I build it (design aspects ) ?
which method should init the parser (the first method that call it
to get specific value)
This is sample code which return two object from the parser but I dont think that this is the right way to do that since maybe I'll need to provide additional properties
the is the module parse
parse = function (data) {
var ymlObj = ymlParser.parse(data);
return {
web: ymlObj.process_types.web,
con: ymlObj.con
}
};
If I understood you right you can just make simple module with getters and setter.
(parse.js)
var ymlObj = {};
function Parse() {}
Parse.prototype.setData = function (data) {
ymlObj = data;
}
Parse.prototype.getWeb = function () {
return ymlObj.process_types.web;
}
Parse.prototype.getCon = function () {
return ymlObj.con;
}
module.exports = new Parse();
(parseUser.js)
var parse = require('./parse.js');
function ParseUser() { }
ParseUser.prototype.useParse = function () {
console.log(parse.getCon());
}
module.exports = new ParseUser();
(app.js)
var parse = require('./parse.js');
var parseUser = require('parseUser.js');
parse.setData({ ... });
parseUser.useParse();
You still have to do basics like handle exceptions but hope this helps you understand the basic structure.
What comes to init it really depends when you want to initialize (fetch?) your data and where does that data come from. You can set timestamp to indicate how old your data is and make decision if you still rely on it or fetch newer data. Or you can register callbacks from your user modules to deal with new data every time its fetched.
So its up to you how you design your module. ;)
I'm running a script on an apache webserver on a linux box. Based on the parameter I want to change the name of variable(or set it)
The idea is that humDev(lines 11 and 14) is named humDev21 for example. Where devId is the number 21 in this example.
My script looks like this:
function getHumDev(devId){
$.ajax({
async: false,
url: "/url" + devId,
success: function(result) {
var array = result["Device_Num_" + devId].states;
function objectFindByKey(array, key, value) {
for (var i = 0; i < array.length; i++) {
if (array[i][key] === value) {
humDev = array[i].value;
}
}
return humDev;
};
objectFindByKey(array, 'service', 'some');
}
});
};
If Im looking in the wrong direction, please do let me know. Maybe its bad practice what Im trying. The reason I want to have the object a unique name is because this function is called several times by another function, based on the content of an array. But when I have the humDev object named without the number suffix to make it unique, the content of the object is getting mixed up between the different calls.
I may be off base but I am making some assumptions based on what I understand of what you are trying to do.
First, you need to understand how to do file I/O in node.js. So lets start there:
var pathToFile, //set with file path string
fs = require('fs'), //require the file i/o module API
bunchOfHumDevs = {},
fileContents; //we'll cache those here for repeated use
fs.readFile(pathToFile, function(err, result) {
if (err) {
throw new Error(); //or however you want to handle errors
} else {
fileContents = JSON.parse(result); //assumes data stored as JSON
}
});
function getHumDev(devId) {
//first make sure we have fileContents, if not try again in 500ms
if (!fileContents) {
setTimeout(function() {
getHumDev(devId);
}, 500);
} else {
var array = fileContents["Device_Num_" + devId].states,
i = array.length,
//if 'service' and 'some' are variable, make them params of
//getHumDev()
while (i--) {
if (array[i]['service'] === 'some') {
//store uniquely named humDev entry
bunchOfHumDevs['humDev' + devId.toString()] = array[i].value;
break; //exit loop once a match is found
}
}
}
return null;
}
getHumDev(21);
assuming a match is found for the devId 21, bunchOfHumdevs will now have a property 'humDev21' that is the object (value?) in question. Also, the fileContents are now cached in the program so you don't have to reopen it every time you call the function.
I'm using mongoose to insert some data into mongodb. The code looks like:
var mongoose = require('mongoose');
mongoose.connect('mongo://localhost/test');
var conn = mongoose.connection;
// insert users
conn.collection('users').insert([{/*user1*/},{/*user2*/}], function(err, docs) {
var user1 = docs[0], user2 = docs[1];
// insert channels
conn.collection('channels').insert([{userId:user1._id},{userId:user2._id}], function(err, docs) {
var channel1 = docs[0], channel2 = docs[1];
// insert articles
conn.collection('articles').insert([{userId:user1._id,channelId:channel1._id},{}], function(err, docs) {
var article1 = docs[0], article2 = docs[1];
}
});
};
You can see there are a lot of nested callbacks there, so I'm trying to use q to refactor it.
I hope the code will look like:
Q.fcall(step1)
.then(step2)
.then(step3)
.then(step4)
.then(function (value4) {
// Do something with value4
}, function (error) {
// Handle any error from step1 through step4
})
.end();
But I don't know how to do it.
You'll want to use Q.nfcall, documented in the README and the Wiki. All Mongoose methods are Node-style. I'll also use .spread instead of manually destructuring .then.
var mongoose = require('mongoose');
mongoose.connect('mongo://localhost/test');
var conn = mongoose.connection;
var users = conn.collection('users');
var channels = conn.collection('channels');
var articles = conn.collection('articles');
function getInsertedArticles() {
return Q.nfcall(users.insert.bind(users), [{/*user1*/},{/*user2*/}]).spread(function (user1, user2) {
return Q.nfcall(channels.insert.bind(channels), [{userId:user1._id},{userId:user2._id}]).spread(function (channel1, channel2) {
return Q.nfcall(articles.insert.bind(articles), [{userId:user1._id,channelId:channel1._id},{}]);
});
})
}
getInsertedArticles()
.spread(function (article1, article2) {
// you only get here if all three of the above steps succeeded
})
.fail(function (error) {
// you get here if any of the above three steps failed
}
);
In practice, you will rarely want to use .spread, since you usually are inserting an array that you don't know the size of. In that case the code can look more like this (here I also illustrate Q.nbind).
To compare with the original one is not quite fair, because your original has no error handling. A corrected Node-style version of the original would be like so:
var mongoose = require('mongoose');
mongoose.connect('mongo://localhost/test');
var conn = mongoose.connection;
function getInsertedArticles(cb) {
// insert users
conn.collection('users').insert([{/*user1*/},{/*user2*/}], function(err, docs) {
if (err) {
cb(err);
return;
}
var user1 = docs[0], user2 = docs[1];
// insert channels
conn.collection('channels').insert([{userId:user1._id},{userId:user2._id}], function(err, docs) {
if (err) {
cb(err);
return;
}
var channel1 = docs[0], channel2 = docs[1];
// insert articles
conn.collection('articles').insert([{userId:user1._id,channelId:channel1._id},{}], function(err, docs) {
if (err) {
cb(err);
return;
}
var article1 = docs[0], article2 = docs[1];
cb(null, [article1, article2]);
}
});
};
}
getInsertedArticles(function (err, articles) {
if (err) {
// you get here if any of the three steps failed.
// `articles` is `undefined`.
} else {
// you get here if all three succeeded.
// `err` is null.
}
});
With alternative deferred promise implementation, you may do it as following:
var mongoose = require('mongoose');
mongoose.connect('mongo://localhost/test');
var conn = mongoose.connection;
// Setup 'pinsert', promise version of 'insert' method
var promisify = require('deferred').promisify
mongoose.Collection.prototype.pinsert = promisify(mongoose.Collection.prototype.insert);
var user1, user2;
// insert users
conn.collection('users').pinsert([{/*user1*/},{/*user2*/}])
// insert channels
.then(function (users) {
user1 = users[0]; user2 = users[1];
return conn.collection('channels').pinsert([{userId:user1._id},{userId:user2._id}]);
})
// insert articles
.match(function (channel1, channel2) {
return conn.collection('articles').pinsert([{userId:user1._id,channelId:channel1._id},{}]);
})
.done(function (articles) {
// Do something with articles
}, function (err) {
// Handle any error that might have occurred on the way
});
Considering Model.save instead of Collection.insert (quite the same in our case).
You don't need to use Q, you can wrap yourself the save method and return directly a Mongoose Promise.
First create an utility method to wrap the save function, that's not very clean but something like:
//Utility function (put it in a better place)
var saveInPromise = function (model) {
var promise = new mongoose.Promise();
model.save(function (err, result) {
promise.resolve(err, result);
});
return promise;
}
Then you can use it instead of save to chain your promises
var User = mongoose.model('User');
var Channel = mongoose.model('Channel');
var Article = mongoose.model('Article');
//Step 1
var user = new User({data: 'value'});
saveInPromise(user).then(function () {
//Step 2
var channel = new Channel({user: user.id})
return saveInPromise(channel);
}).then(function (channel) {
//Step 3
var article = new Article({channel: channel.id})
return saveInPromise(article);
}, function (err) {
//A single place to handle your errors
});
I guess that's the kind of simplicity we are looking for.. right? Of course the utility function can be implemented with better integration with Mongoose.
Let me know what you think about that.
By the way there is an issue about that exact problem in the Mongoose Github:
Add 'promise' return value to model save operation
I hope it's gonna be solved soon. I think it takes some times because they are thinking of switching from mpromise to Q: See here and then here.
Two years later, this question just popped up in my RSS client ...
Things have moved on somewhat since May 2012 and we might choose to solve this one in a different way now. More specifically, the Javascript community has become "reduce-aware" since the decision to include Array.prototype.reduce (and other Array methods) in ECMAScript5. Array.prototype.reduce was always (and still is) available as a polyfill but was little appreciated by many of us at that time. Those who were running ahead of the curve may demur on this point, of course.
The problem posed in the question appears to be formulaic, with rules as follows :
The objects in the array passed as the first param to conn.collection(table).insert() build as follows (where N corresponds to the object's index in an array):
[ {}, ... ]
[ {userId:userN._id}, ... ]
[ {userId:userN._id, channelId:channelN._id}, ... ]
table names (in order) are : users, channels, articles.
the corresopnding object properties are : user, channel, article (ie the table names without the pluralizing 's').
A general pattern from this article by Taoofcode) for making asynchronous call in series is :
function workMyCollection(arr) {
return arr.reduce(function(promise, item) {
return promise.then(function(result) {
return doSomethingAsyncWithResult(item, result);
});
}, q());
}
With quite light adaptation, this pattern can be made to orchestrate the required sequencing :
function cascadeInsert(tables, n) {
/*
/* tables: array of unpluralisd table names
/* n: number of users to insert.
/* returns promise of completion|error
*/
var ids = []; // this outer array is available to the inner functions (to be read and written to).
for(var i=0; i<n; i++) { ids.push({}); } //initialize the ids array with n plain objects.
return tables.reduce(function (promise, t) {
return promise.then(function (docs) {
for(var i=0; i<ids.length; i++) {
if(!docs[i]) throw (new Error(t + ": returned documents list does not match the request"));//or simply `continue;` to be error tolerant (if acceptable server-side).
ids[i][t+'Id'] = docs[i]._id; //progressively add properties to the `ids` objects
}
return insert(ids, t + 's');
});
}, Q());
}
Lastly, here's the promise-returning worker function, insert() :
function insert(ids, t) {
/*
/* ids: array of plain objects with properties as defined by the rules
/* t: table name.
/* returns promise of docs
*/
var dfrd = Q.defer();
conn.collection(t).insert(ids, function(err, docs) {
(err) ? dfrd.reject(err) : dfrd.resolve(docs);
});
return dfrd.promise;
}
Thus, you can specify as parameters passed to cascadeInsert, the actual table/property names and the number of users to insert.
cascadeInsert( ['user', 'channel', 'article'], 2 ).then(function () {
// you get here if everything was successful
}).catch(function (err) {
// you get here if anything failed
});
This works nicely because the tables in the question all have regular plurals (user => users, channel => channels). If any of them was irregular (eg stimulus => stimuli, child => children), then we would need to rethink - (and probably implement a lookup hash). In any case, the adaptation would be fairly trivial.
Today we have mongoose-q as well. A plugin to mongoose that gives you stuff like execQ and saveQ which return Q promises.