We Keep Coding

Unable to refer to method `db.start Session ()` in mongoose, mongodb

When I try to refer to db.startSession () I have an error db.startSession () is not a function. In console.log I put db and I put logged variable below. Why can't I refer to startSession()?
connection
module.exports.db = mongoose
.createConnection(process.env.DATABASE, {
useNewUrlParser: true,
useFindAndModify: false,
useUnifiedTopology: true,
useCreateIndex: true
}
);
controller
const db = require('./connection');
const session = db.startSession();
session.startTransaction();
logged variable db
{
db: NativeConnection {
base: Mongoose {
connections: [Array],
models: [Object],
modelSchemas: [Object],
options: [Object],
_pluralize: [Function: pluralize],
plugins: [Array]
},
collections: {},
models: {},
config: { autoIndex: true },
replica: false,
options: null,
otherDbs: [],
relatedDbs: {},
states: [Object: null prototype] {
'0': 'disconnected',
'1': 'connected',
'2': 'connecting',
'3': 'disconnecting',
'99': 'uninitialized',
disconnected: 0,
connected: 1,
connecting: 2,
disconnecting: 3,
uninitialized: 99
},
_readyState: 1,
_closeCalled: false,
_hasOpened: true,
_listening: false,
_connectionOptions: {
useNewUrlParser: true,
useFindAndModify: false,
useUnifiedTopology: true,
useCreateIndex: true,
promiseLibrary: [Function: Promise]
},
client: MongoClient {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
s: [Object],
topology: [ReplSet],
[Symbol(kCapture)]: false
},
name: null,
'$initialConnection': Promise { [Circular] },
db: Db {
_events: [Object: null prototype],
_eventsCount: 3,
_maxListeners: undefined,
s: [Object],
serverConfig: [Getter],
bufferMaxEntries: [Getter],
databaseName: [Getter],
[Symbol(kCapture)]: false
}
}
}

you can do like this for using transaction in mongoose:
Note:for using transaction you need to replica mongodb, so you should Convert a Standalone to a Replica Set, and need to add retryWrites: false as argument in mongoose.createConnection()
connect to mongoose :
await mongoose.connect(`mongodb://localhost/namedb`, {
useNewUrlParser: true,
useUnifiedTopology: true,
useCreateIndex: true,
retryWrites: false
});
const mongoose = require("mongoose");
const sess = await mongoose.startSession();
sess.startTransaction();
await teacher.save({ session: sess }); // a query
student.messeges.push(); // do somthing
await student.save({ session: sess }); //// a query
await sess.commitTransaction();
for using replica set in localhost you should stop mongod service and run this code
mongod --port 27017 --dbpath C:\data\rs1 --replSet m101 --bind_ip localhost
C:\data\rs1 data stored path
rs.initiate()

mongoDB: getMongo() on a db instance returns error

I'm trying to do a sql transaction. for this I need to start a session which a possible only with a mongo dB connection instance.
In my app, I can reach only the DB instance. and I need to be able to get from it the mongodb connection instance. As I understand, getMongo() function should do exactly that.
The problem is the getMongo() is not acting as expected.
when i run this following code:
const db = await this.dbClient;
console.log(db,"db");
console.log(db.getMongo(),"getMongo");
const session = db.getMongo().startSession();
I get this following console.log and errors:
Db {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
s: {
dbCache: {},
children: [],
topology: NativeTopology {
_events: [Object: null prototype],
_eventsCount: 35,
_maxListeners: Infinity,
s: [Object],
[Symbol(kCapture)]: false,
[Symbol(waitQueue)]: [Denque]
},
options: {
retryWrites: true,
readPreference: [ReadPreference],
promiseLibrary: [Function: Promise]
},
logger: Logger { className: 'Db' },
bson: BSON {},
readPreference: ReadPreference { mode: 'primary', tags: undefined },
bufferMaxEntries: -1,
parentDb: null,
pkFactory: undefined,
nativeParser: undefined,
promiseLibrary: [Function: Promise],
noListener: false,
readConcern: undefined,
writeConcern: undefined,
namespace: MongoDBNamespace { db: 'MyDB', collection: undefined }
},
serverConfig: [Getter],
bufferMaxEntries: [Getter],
databaseName: [Getter],
[Symbol(kCapture)]: false
} db
{"Error msg: TypeError: db.getMongo is not a
function\n...","level":"error"}
{"session is not defined....","level":"error"}
Anybody knows what it the problem?
My mongo version is 4.2.6.
Many thanks

Failing to convert (some) eps to png with ImageMagick (Firebase Cloud Function)

I'm having an issue with some EPS conversions using ImageMagick in Firebase Cloud Functions. The ones that does work, work every time - and the ones that don't, always fails, and I don't understand why..
I'm getting this error in the logs:
{ ChildProcessError: `convert -colorspace sRGB -density 300 input.eps -resize 400x400 output.png` failed with code 1
at ChildProcess.<anonymous> (/user_code/node_modules/child-process-promise/lib/index.js:132:23)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:920:16)
at Socket.<anonymous> (internal/child_process.js:351:11)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at Pipe._handle.close [as _onclose] (net.js:509:12)
name: 'ChildProcessError',
code: 1,
childProcess:
ChildProcess {
domain: null,
_events: { error: [Function: t], close: [Function] },
_eventsCount: 2,
_maxListeners: undefined,
_closesNeeded: 3,
_closesGot: 3,
connected: false,
signalCode: null,
exitCode: 1,
killed: false,
spawnfile: 'convert',
_handle: null,
spawnargs:
[ 'convert',
'-colorspace',
'sRGB',
'-density',
'300',
'input.eps',
'-resize',
'400x400',
'output.png' ],
pid: 12,
stdin:
Socket {
connecting: false,
_hadError: false,
_handle: null,
_parent: null,
_host: null,
_readableState: [Object],
readable: false,
domain: null,
_events: [Object],
_eventsCount: 2,
_maxListeners: undefined,
_writableState: [Object],
writable: false,
allowHalfOpen: false,
destroyed: true,
_bytesDispatched: 0,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: null,
_server: null,
write: [Function: writeAfterFIN],
_idleNext: null,
_idlePrev: null,
_idleTimeout: -1 },
stdout:
Socket {
connecting: false,
_hadError: false,
_handle: null,
_parent: null,
_host: null,
_readableState: [Object],
readable: false,
domain: null,
_events: [Object],
_eventsCount: 3,
_maxListeners: undefined,
_writableState: [Object],
writable: false,
allowHalfOpen: false,
destroyed: true,
_bytesDispatched: 0,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: null,
_server: null,
_idleNext: null,
_idlePrev: null,
_idleTimeout: -1,
write: [Function: writeAfterFIN] },
stderr:
Socket {
connecting: false,
_hadError: false,
_handle: null,
_parent: null,
_host: null,
_readableState: [Object],
readable: false,
domain: null,
_events: [Object],
_eventsCount: 3,
_maxListeners: undefined,
_writableState: [Object],
writable: false,
allowHalfOpen: false,
destroyed: true,
_bytesDispatched: 0,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: null,
_server: null,
write: [Function: writeAfterFIN],
read: [Function],
_consuming: true,
_idleNext: null,
_idlePrev: null,
_idleTimeout: -1 },
stdio: [ [Object], [Object], [Object] ] },
stdout: undefined,
stderr: undefined }
My cloud function looks like this:
/**
* When an image is uploaded in the Storage bucket We generate a thumbnail automatically using
* ImageMagick.
* After the thumbnail has been generated and uploaded to Cloud Storage,
* we write the public URL to Firestore.
*/
exports.generateThumbnail = functions.storage.object().onFinalize((object) => {
// File and directory paths.
const filePath = object.name;
const contentType = object.contentType; // This is the image MIME type
const fileDir = path.dirname(filePath);
const fileName = path.basename(filePath);
const thumbFilePath = path.normalize(path.join(fileDir, `${THUMB_PREFIX}${fileName}.png`));
const tempLocalFile = path.join(os.tmpdir(), filePath);
const tempLocalDir = path.dirname(tempLocalFile);
const tempLocalThumbFile = path.join(os.tmpdir(), thumbFilePath);
// Don't run when deleting files
if (object.resourceState === 'not_exists') {
console.log('Deleted file');
return null;
}
// Check if it's a eps, ps or ai file before proceeding
if (contentType != "application/postscript") {
console.log("We don't need to transform this file");
return null;
}
// Exit if the image is already a thumbnail.
if (fileName.startsWith(THUMB_PREFIX)) {
console.log('Already a Thumbnail.');
return null;
}
// Cloud Storage files.
const bucket = gcs.bucket(object.bucket);
const file = bucket.file(filePath);
const thumbFile = bucket.file(thumbFilePath);
const metadata = {
contentType: 'image/png',
// To enable Client-side caching you can set the Cache-Control headers here. Uncomment below.
// 'Cache-Control': 'public,max-age=3600',
};
// Create the temp directory where the storage file will be downloaded.
return mkdirp(tempLocalDir).then(() => {
// Download file from bucket.
return file.download({ destination: tempLocalFile });
}).then(() => {
console.log("The file has been downloaded to", tempLocalFile);
// Generate a thumbnail using ImageMagick.
return spawn('convert', ['-colorspace', 'sRGB', '-density', '300', tempLocalFile, '-resize', '400x400', tempLocalThumbFile]);
}).then(() => {
console.log('Thumbnail created at', tempLocalThumbFile);
// Uploading the Thumbnail.
return bucket.upload(tempLocalThumbFile, { destination: thumbFilePath, metadata: metadata });
}).then(() => {
console.log('Thumbnail uploaded to Storage at', thumbFilePath);
// Once the image has been uploaded delete the local files to free up disk space.
fs.unlinkSync(tempLocalFile);
fs.unlinkSync(tempLocalThumbFile);
// Get the Signed URLs for the thumbnail and original image.
const config = {
action: 'read',
expires: '03-01-2500',
};
return Promise.all([
thumbFile.getSignedUrl(config)
]);
}).then((results) => {
const thumbResult = results[0];
const thumbFileUrl = thumbResult[0];
console.log('Got Signed URLs: ' + results);
console.log('Thumb result:' + thumbResult);
console.log('Thumb file URL: ' + thumbFileUrl);
// Add the URLs to the Database
return admin.firestore().doc(fileDir).update({ thumb_src: thumbFileUrl, processed: true }).then(() => {
// write is complete here
console.log("Thumbnail URLs saved to database.");
});
}).catch((error) => {
console.error(error);
});
});
Let me know if I need to provide more information, but like I said - the code works for some EPS files. First I thought that the problem could be the file size of the input files, and that the conversion didn't wait for the download to finish - But I'm not sure if that's the case or not.

Can't retrieve data from mongoDB with Node.js

I'm pretty new in Node.js and I have this project. Basically I have some data sits in a mongoDB collection("data") and I'm trying to get that data and display it on the browser.
Here is what I've got so far;
var MongoClient = require('mongodb').MongoClient
,format = require('util').format;
var sys = require ("sys");
my_http = require("http");
my_http.createServer(function(request, response){
sys.puts("Touched !!");
response.writeHeader(200, {"Content-Type": "text/plain"});
response.write(extractData()).toString();
response.end();
}).listen(8080);
sys.puts("Server is running on 8080"); // Server kicks in...np
function extractData(){
MongoClient.connect('mongodb://127.0.0.1:27017/mongoDB', function(err, db){
if (err){
console.log("Can't Connect to DB !!");
}else {
console.log("Connected to DB !!"); // connects to DB, np
db.data.find({}, function(err, data){ // .find is the problem
if (err || !data) console.log("No Data Found");
else data.forEach(function (data){
console.log(data);
});
}).toArray();
}
});
}
And after I run "node server.js" and refresh the already open localhost:8080, I get this;
Server is running on 8080
Touched !!
Touched !!
Connected to DB !!
d:\Projects\SCRIPTS\mdp.scripts.testing-tools\jsFinderWmongoDB\node_modules\mongodb\lib\mongodb\mongo_client.js:475
throw err
^
TypeError: Cannot call method 'find' of undefined
at d:\Projects\SCRIPTS\mdp.scripts.testing-tools\jsFinderWmongoDB\server.js:21:21
at d:\Projects\SCRIPTS\mdp.scripts.testing-tools\jsFinderWmongoDB\node_modules\mongodb\lib\mongodb\mongo_client.js:4
72:11
at process._tickCallback (node.js:415:13)
Don't understand why there is a problem with .find() and of course can't display any data...
Any ideas?
Edit:
Well, we are certainly getting somewhere. I've made some changes.
Current code:
function extractData(){
MongoClient.connect('mongodb://127.0.0.1:27017/mongoDB', function(err, db){
if (err){
console.log("Can't Connect to DB !!");
}else {
sys.puts("Connected to DB !!"); // connects to DB, np
db.collection('data').find({}, function(err, data){
if (err || !data) console.log("No Data Found");
//else db.collection('data').forEach(function (data){
// console.log(data);
//});
});//.toArray();
}
});
}
Browser response is "undefined"
I'm guessing "extractData" function is NOT returning something legit. Therefore the collection set "data" is returning empty.
And yes I've checked one more time, I have data in the dataset.

You need to set a collection to use before you can do anything with it.
var collection = db.collection('data');
collection.find({},function(err,data){
console.log(data);
});
Would be how you do it.
Update ** This was how I did my first mongoDB stuff using express
Might help you.
var mongo = require('mongodb');
var monk = require('monk');
var db = monk('localhost:27017/nodetest1');
app.get('/userlist', function(req, res) {
var db = req.db;
var collection = db.get('usercollection');
collection.find({},{},function(e,docs){
res.render('userlist', {
"userlist" : docs
});
});
});

I get the details about the stored data when do console.log(data);
Here is a glimpse of it
Server is running on 8080
Touched !!
Connected to DB !!
{ db:
{ domain: null,
_events: {},
_maxListeners: 10,
databaseName: 'mongoDB',
serverConfig:
{ domain: null,
_events: {},
_maxListeners: 10,
auth: [Getter],
_callBackStore: [Object],
_commandsStore: [Object],
_dbStore: [Object],
host: '127.0.0.1',
port: 27017,
options: [Object],
internalMaster: true,
connected: true,
poolSize: 5,
disableDriverBSONSizeCheck: false,
_used: true,
replicasetInstance: null,
emitOpen: false,
ssl: false,
sslValidate: false,
sslCA: null,
sslCert: undefined,
sslKey: undefined,
sslPass: undefined,
serverCapabilities: [Object],
name: '127.0.0.1:27017',
socketOptions: [Object],
logger: [Object],
eventHandlers: [Object],
_serverState: 'connected',
_state: [Object],
recordQueryStats: false,
socketTimeoutMS: [Getter/Setter],
_readPreference: [Object],
db: [Circular],
dbInstances: [Object],
connectionPool: [Object],
isMasterDoc: [Object] },
options:
{ read_preference_tags: null,
read_preference: 'primary',
url: 'mongodb://127.0.0.1:27017/mongoDB',
native_parser: true,
readPreference: [Object],
safe: false,
w: 1 },
_applicationClosed: false,
slaveOk: false,
bufferMaxEntries: -1,
native_parser: true,
bsonLib:
{ BSON: [Object],
Long: [Object],
ObjectID: [Object],
DBRef: [Object],
Code: [Object],
Timestamp: [Object],
Binary: [Object],
Double: [Object],
MaxKey: [Object],
MinKey: [Object],
Symbol: [Object] },
bson: { promoteLongs: true },
bson_deserializer:
{ Code: [Object],
Symbol: [Object],
BSON: [Object],
DBRef: [Object],
Binary: [Object],
ObjectID: [Object],
Long: [Object],
Timestamp: [Object],
Double: [Object],
MinKey: [Object],
MaxKey: [Object],
promoteLongs: true },
bson_serializer:
{ Code: [Object],
Symbol: [Object],
BSON: [Object],
DBRef: [Object],
Binary: [Object],
ObjectID: [Object],
Long: [Object],
Timestamp: [Object],
Double: [Object],
MinKey: [Object],
MaxKey: [Object],
promoteLongs: true },
_state: 'connected',
pkFactory:
{ [Function: ObjectID]
index: 16043018,
createPk: [Function: createPk],
createFromTime: [Function: createFromTime],
createFromHexString: [Function: createFromHexString],
isValid: [Function: isValid],
ObjectID: [Circular],
ObjectId: [Circular] },
forceServerObjectId: false,
safe: false,
notReplied: {},
isInitializing: true,
openCalled: true,
commands: [],
logger: { error: [Function], log: [Function], debug: [Function] },
tag: 1425061857066,
eventHandlers:
{ error: [],
parseError: [],
poolReady: [],
message: [],
close: [] },
serializeFunctions: false,
raw: false,
recordQueryStats: false,
retryMiliSeconds: 1000,
numberOfRetries: 60,
readPreference: { _type: 'ReadPreference', mode: 'primary', tags: undefined } },
collection:

long poll with node.js and express: how to cache res object into redis or other cache framework

I try to build a chat room webpage with node.js/express/redis on the server side.
Following this snips of code : A Message Wall With Long Poll Properties in Node.JS and Express , I succeed to make one node server running correctly. In this example, the res objects are saved in a list, no any transformation is needed.
But I want to run the node app with pm2 cluster mode(-i 4), so I have to save res object into some place that shared between 4 nodes.
I already used redis in my node project, for the express.session. So I want to cache res into redis.
But the problem occured: when I try to sting-lized res object with JSON.stringify(res), I got :
TypeError: Converting circular structure to JSON
My problem is:
how could I save one res object for later use across node cluster, with redis or something else.
Appreciate.
I use util.inspect to print out my container of res (chatroom_id:res) object:
{ '0': null,
'1390640136999':
{ domain: null,
_events:
{ finish: [Object],
header: [Function],
close: [Function: logRequest] },
_maxListeners: 10,
output: [],
outputEncodings: [],
writable: true,
_last: false,
chunkedEncoding: false,
shouldKeepAlive: false,
useChunkedEncodingByDefault: false,
sendDate: true,
_headerSent: false,
_header: '',
_hasBody: true,
_trailer: '',
finished: false,
_hangupClose: false,
socket:
{ _connecting: false,
_handle: [Object],
_readableState: [Object],
readable: true,
domain: null,
_events: [Object],
_maxListeners: 10,
_writableState: [Object],
writable: true,
allowHalfOpen: true,
onend: [Function],
destroyed: false,
errorEmitted: false,
bytesRead: 894,
_bytesDispatched: 0,
_pendingData: null,
_pendingEncoding: '',
server: [Object],
_idleTimeout: 120000,
_idleNext: [Object],
_idlePrev: [Object],
_idleStart: 1390640145289,
parser: [Object],
ondata: [Function],
_paused: false,
_httpMessage: [Circular],
_peername: [Object] },
connection:
{ _connecting: false,
_handle: [Object],
_readableState: [Object],
readable: true,
domain: null,
_events: [Object],
_maxListeners: 10,
_writableState: [Object],
writable: true,
allowHalfOpen: true,
onend: [Function],
destroyed: false,
errorEmitted: false,
bytesRead: 894,
_bytesDispatched: 0,
_pendingData: null,
_pendingEncoding: '',
server: [Object],
_idleTimeout: 120000,
_idleNext: [Object],
_idlePrev: [Object],
_idleStart: 1390640145289,
parser: [Object],
ondata: [Function],
_paused: false,
_httpMessage: [Circular],
_peername: [Object] },
_headers: { 'x-powered-by': 'Express' },
_headerNames: { 'x-powered-by': 'X-Powered-By' },
req:
{ _readableState: [Object],
readable: true,
domain: null,
_events: {},
_maxListeners: 10,
socket: [Object],
connection: [Object],
httpVersion: '1.0',
complete: true,
headers: [Object],
trailers: {},
_pendings: [],
_pendingIndex: 0,
url: '/robot/chat/query/99/1390640136999/270125/',
method: 'GET',
statusCode: null,
client: [Object],
_consuming: false,
_dumped: false,
httpVersionMajor: 1,
httpVersionMinor: 0,
upgrade: false,
originalUrl: '/robot/chat/query/99/1390640136999/270125/',
_parsedUrl: [Object],
query: {},
res: [Circular],
next: [Function: next],
secret: undefined,
cookies: [Object],
signedCookies: {},
sessionStore: [Object],
sessionID: '4PACUldyCHhT8NgdGY1yz9Pk',
session: [Object],
_startTime: Sat Jan 25 2014 16:55:45 GMT+0800 (CST),
_remoteAddress: '127.0.0.1',
body: {},
originalMethod: 'GET',
_route_index: 2,
route: [Object],
params: [Object] },
locals: [Function: locals],
end: [Function],
student_id: '99',
channel_id: '1390640136999',
last_msg_id: '270125' } }
There are three [Circular].
My pseudocode:
/*
* query from http client
* url: /chat/query/:student_id/:channel_id/:last_msg_id/
*/
exports.query = function(req, res){
// if find some new msg
// return them as json, immediately
// else
// set participator info into res object
// read the res_list from redis
// put this res into res_list
// write back res_list into redis
};
/*
* notification from other web server: one new msg been created
* url: /chat/notify/:new_msg_id/
*/
exports.notify = function(req, res){
// get new_msg from database by id
// read the res_list from redis
// for old_res in res_list
// if this old_res is releated with the new_msg (participator)
// old_res.sent(json_content)
// remove this old_res from res_list
// write back res_list into redis
};
how could i implement those 'read-and-write-back' part?

The short answer: it's not possible to put the res object into a cache and obtain it again from another process, in any meaningful way.
If you think about how long poll works, each HTTP client maintains an open connection to the server, waiting (a long time) for something to be sent back. In other words, when you come to push out new messages, you're just sending data down an already open connection. If you cache your res into redis, what would happen to the connection?
It may not matter though, as long as you have a way to pass the content between your 4 backend processes, they can each update their own set of res connections. New connections would still be load balanced.