We Keep Coding

Sequelize converts passed date object to local date

When I execute a query that that has a data passed as replacement, the date is not set as UTC-Date in the query.
My code:
let startInterval = moment('2020-12-09').toDate();
db.query(`
SELECT kv.kpiId
FROM kpiValues kv
WHERE kv.insertDate >= :startInterval
`, {
type: QueryTypes.SELECT,
replacements: {
startInterval: startInterval,
}
}).catch(next)
Printing the startInterval variable results in 2020-12-08T23:00:00.000Z.
When I output the query it shows
SELECT kv.kpiId
FROM kpiValues kv
WHERE kv.insertDate >= '2020-12-09 00:00:00.000'
The column is a sequelize Date column (SQL Datetime). The value in the query is my local time - but the database should only use UTC values - so I would expect it to use 2020-12-08T23:00:00.000 in the query. What can I do?
The sequelize connection:
const sequelize = new Sequelize(dbName, dbUser, dbPassword, {
host: Settings.dbUrl,
port: dbPort,
dialect: 'mariadb',
dialectOptions: {},
timezone: '+00:00',
pool: {
max: 70,
min: 5,
acquire: 30000
},
define: {
timestamps: false,
freezeTableName: true
},
logging: true // Remove property when all statements that are executed should be printed
});

As there is no solution for this problem with mariaDB as DBMS using sequelize the following workaround works:
Just manually pass the replacement variable as UTC-String:
dateVariable.toISOString()

How to get current date/time using database timezone in sequelize

I need to get the current time, according to the database timezone (not my local timezone, and not default UTC timezone), using Sequelize. Is there a Sequelize method to do this?
My database is in Eastern time, and when I query the db directly SELECT CURRENT_TIMESTAMP; it returns the date/time in Eastern time (which is correct).
But when I query via Sequelize in Node const [[{time}]] = await db.sequelize.query('SELECT CURRENT_TIMESTAMP AS time'); it returns the date/time in UTC.
Two problems:
1 - I would prefer using a Sequelize method instead of a raw query.
2 - This still doesn't get me the result I want. The time needs to be Eastern.
This is my DB setup:
const sequelize = new Sequelize(dbUrl, {
dialectOptions: {
useUTC: false // for reading from database
},
timezone: '-04:00', // for writing to database
define: {
charset: 'utf8'
}
})
As mentioned above, when I query using the above queries, the date is always returned in UTC, which I did not expect, given I said useUTC: false. How do I get it in Eastern time (the database timezone)?

I'm not aware of a sequelize method like getCurrentDate().
The UTC conversion problem seems to bite everyone (myself included). Here are some details. Not sure if dialectOptions: {useUTC: false }, has any function at all - just adding the typeCast method solved the problem for me.
dialectOptions: {
typeCast: function (field, next) { // for reading from database
if (field.type === 'DATETIME') {
return field.string()
}
return next()
},
The result can be used for a new js Date object:
const sql = 'select current_timestamp';
my_app.my_DB.query(sql, {raw: true, type: Sequelize.QueryTypes.SELECT})
.then(data => {
console.log(data[0].current_timestamp);
let d1 = new Date(data[0].current_timestamp);
});
This works fine for me - but make sure to test thoroughly!

Add useUTC property in your dialectOptions like this
dialectOptions: {
encrypt: false ,
options: {
useUTC: false, // for reading from database
},
},

Knex returns DATETIME fields applying offset of my machine timezone

I have following row saved in timezone Europe/Madrid at database:
dateA = '2019-03-26 15:00:00'
dateB = '2019-03-26 14:00:00'
When selecting the entry:
let entry = this.query().findById(id)
I get following values (console.log(entry)):
{
dateA: 2019-03-26T06:00:00.000Z,
dateB: 2019-03-26T05:00:00.000Z
}
What is applying this convertion?
My enviroment:
Im using knex, objection and moment
Moment is configured with moment.tz.setDefault('Europe/Madrid')
My local machine timezone is set to UTC+09:00 (Chita) for testing
I tried:
Setting a connection timezone on knex creation:
const connection = {
user: dbCreds.username,
password: dbCreds.password,
host: hostname,
database: dbCreds.database,
timezone: '-1:00'
}
const pool = knex({
client: 'mysql2',
connection,
pool: {
min: 2,
max: 30
}
})
I still get the same result

Finally found out whats going on.
Knex, when retrieving the info of the datetime field checks the timezone of the database. You can check it using:
SELECT ##global.time_zone, ##session.time_zone;
This returns SYSTEM SYSTEM in my case. So its using my machine timezone. To test it, I changed it to UTC.
So now, when reading the following value:
dateA = '2019-03-26 15:00:00'
dateB = '2019-03-26 14:00:00'
Knex assumes its in UTC, so the value in UTC is returned:
{
dateA: 2019-03-26T15:00:00.000Z,
dateB: 2019-03-26T14:00:00.000Z
}
Seems not possible to configure MySQL to use 'Europe/Madrid' as default timezone.

Node JS - Mysql : invisible characters

To simplify, I save a JSON in my Mysql database, the JSON has this form:
{
"idAd": "",
"titleAd": "F2 47m² 1340 CC",
"dateAd": "2017-11-26",
"priseAd": 1340,
}
The field "titleAd" can contain special characters, including some weird and invisible characters :p
So when I make an extraction of this field the invisible character turns into "?"
I know it's a basic and simple encoding problem but I can't find the solution.
{
"idAd": "",
"titleAd": "F2 47m² 1340? CC",
"dateAd": "2017-11-26",
"priseAd": 1340,
}
For information I have specified the type of encoding in connection to the database
const mysql = require('mysql');
const db = mysql.createConnection({
host: 'localhost',
user: 'root',
database: 'dababase',
charset: "utf8_general_ci"
});
Thanks for your help.

Getting MongoDb document to expire at a certain time using mongoose [duplicate]

Below is the command that can be used via the mongo terminal to set an expiry time for collections (a TTL):
db.log.events.ensureIndex( { "status": 1 }, { expireAfterSeconds: 3600 } )
How do I do this from my code in Node.js using mongoose?

In Mongoose, you create a TTL index on a Date field via the expires property in the schema definition of that field:
// expire docs 3600 seconds after createdAt
new Schema({ createdAt: { type: Date, expires: 3600 }});
Note that:
MongoDB's data expiration task runs once a minute, so an expired doc might persist up to a minute past its expiration.
This feature requires MongoDB 2.2 or later.
It's up to you to set createdAt to the current time when creating docs, or add a default to do it for you as suggested here.
{ createdAt: { type: Date, expires: 3600, default: Date.now }}

this code is working for me.
may it help
let currentSchema = mongoose.Schema({
id: String,
name: String,
packageId: Number,
age: Number
}, {timestamps: true});
currentSchema.index({createdAt: 1},{expireAfterSeconds: 3600});

Providing a string to expires also works nicely with Mongoose if you do not want to deal with the expire time calculation and improve the overall readability of the schema.
For example here we are setting the expires to 2m (2 minutes) and mongoose would convert to 120 seconds for us:
var TestSchema = new mongoose.Schema({
name: String,
createdAt: { type: Date, expires: '2m', default: Date.now }
});
Mongoose would create an index in the background and auto set the expireAfterSeconds to in this case 120 seconds (specified by the 2m).
It is important to note that the TTL process runs once every 60 seconds so it is not perfectly on time always.

If you are working with Mongodb Atlas Replica Sets - try:
import * as mongoose from 'mongoose';
let currentSchema = new mongoose.Schema({
createdAt: { type: Date, expires: 10000, default: Date.now },
id: String,
name: String,
packageId: Number,
age: Number
});
currentSchema.index({"lastModifiedDate": 1 },{ expireAfterSeconds: 10000 });

new Scehma({
expireAt: {
type: Date,
expires: 11,
default: Date.now
}
)}
This is the solution that worked for me according to this in the current Mongoose docs.

There is a npm library - 'mongoose-ttl'.:
var schema = new Schema({..});
schema.plugin(ttl, { ttl: 5000 });
you can see all the options of this library:
https://www.npmjs.com/package/mongoose-ttl

const Schema = new mongoose.Schema({id: {
type: Number},
createdAt: {
type: Date, expires: '4h', index: true,
default: Date.now}});
You need to add index: true while creating you schema

9/2022 Working Solution using Mongoose 6.5.4
None of the answers here worked for me, but I was able to finally get it working using the latest version of Mongoose currently available, 6.5.4.
Say our Schema looks like this:
const MySchema = new mongoose.Schema({
id: { type: Number },
myCustomTTLField: { type: Date }
});
myCustomTTLField is the field you want to index and have control the expiration. To achieve this, we add the following under our schema definition:
MySchema.path('myCustomTTLField').index({ expires: 60 });
The argument in MySchema.path is the name of the field you want to index for TTL. The expires option should be the number of seconds that will elapse from the Date represented in myCustomTTLField before the document is deleted. In the example above, the document will be deleted 60 seconds after whatever date is saved in myCustomTTLField. The full example:
const MySchema = new mongoose.Schema({
id: { type: Number },
myCustomTTLField: { type: Date }
});
MySchema.path('myCustomTTLField').index({ expires: 60 });
Please let me know if this works for you, I hope this helps. Mongoose TTL has been a thorn in my side for a long time, as their docs are notoriously tough to navigate. I found this solution via a small example buried in the docs here.
IMPORTANT NOTE:
TTL is not guaranteed to happen at exactly the time specified by your date + expiration seconds. This is due to how MongoDB's background delete process works. It runs every 60 seconds, so you may theoretically wait up to 60 seconds past expected TTL before seeing your document deleted. More info on that from the MongoDB docs.

FWIW I could only get the expires feature to work on a field called expiresAt. Here's my interface, and schema for implementing this in Typescript.
import { model, Schema, Types } from 'mongoose';
export interface ISession {
sessionId: string;
userId: Types.ObjectId;
role: string;
expiresAt?: Date;
}
const sessionSchema = new Schema<ISession>({
sessionId: { type: String, required: true, indexes: { unique: true} },
userId: { type: Schema.Types.ObjectId, required: true, ref: 'users'},
role: { type: String, required: true, enum: [ 'ADMIN', 'BASIC_USER' ]},
expiresAt: { type: Date, expires: '1h', default: Date.now }
}, { versionKey: false });
Reading the Mongoose documentation it seems like all the other proposed solutions should work too. I don't know why they were not for me. You can read the official Mongoose docs on expiresAt here.