We Keep Coding

In sails, how to handle "nulls" in a JSON field in a database?

Well say I have a database that has a schema like:
CREATE TABLE public.mytable
(
datacolumn json,
id serial NOT NULL,
PRIMARY KEY (id)
)
Clearly datacolumn can be null: null is even an important value of this.
Now in sails I have a datamodel defined as:
moduel.exports = {
attributes: {
datacolumn: {
type: 'ref',
columnType: 'json',
allowNull: true,
},
},
id: {
type: 'number',
},
}
This should work right? - Well it doesn't sails refuses to lift:
Failed to lift app: userError: The attribute datacolumn on the mytable model contains invalid properties. The allowNull flag may not be used on attributes with type json or type ref.
I could of course just remove the line allowNull: but that would give lots of warnings that null is not a valid value (directly logged, and giving like 10 lines per warning). And I doubt it will correctly insert null values would it?
So how to handle above database schema?

Edit readonly javascript objects

I have a read-only object that is returned by GraphQL (vue-apollo) query, the result which is read-only looks something like this:
result: {
id: 'yh383hjjf',
regulations: [{ title: 'Test', approved: false}]
})
I want to bind this to a form and be able to edit/update the values in the regulations array and save it back to the database.
at the moment when I try to edit I get the error below:
Uncaught TypeError: "title" is read-only
I tried cloning the result returned by the database using object.assign
//target template
const regulatoryApprovals = {
id: null,
regulations: [{ title: null, approved: null}]
})
regulatoryApprovals = Object.assign(regulatoryApprovals, result, {
regulations: Object.assign(regulatoryApprovals.regulations, result.regulations)
})
but this didn't work.
Does anyone know how I can properly clone the result?

regulatoryApprovals= Object.assign(regulatoryApprovals, ... indicates the problem because regulatoryApprovals is modified with Object.assign, so it would need no assignment.
Read-only regulatoryApprovals object needs to be cloned. regulations is an array and won't be merged correctly with Object.assign, unless it's known that array elements need to be replaced. It should be:
regulatoryApprovals = {
...regulatoryApprovals,
...result,
regulations: [...regulatoryApprovals.regulations, result.regulations]
}
Where { ...regulatoryApprovals, ... } is a shortcut for Object.assign({}, regulatoryApprovals, ...).

Printing Object doesnt contain all keys

Im completely lost. This is some test code I use to print a specific key of an object, then im printing the entire object.
console.log(docs[0].mc_ign);
console.log(docs[0]);
Now this is the output I see on the console:
The__TxT
{
id: 0,
status: 1,
testing: false,
_id: 5dbc17eb20b3a8594d569570,
timestamp: 2019-11-01T11:32:59.380Z,
mc_uuid: 'dac89e44d1024f3b810478ed62d209a1',
discord_id: '653029505457586176',
email_address: 'gut97930#eveav.com',
country: 'Germany',
birth_month: 3,
birth_year: 1943,
about_me: 'about me text',
motivation: 'motivation text',
build_images: '',
publish_about_me: true,
publish_age: false,
publish_country: true,
__v: 0
}
Where is the mc_ign key?
The object itself comes from mongoose, the missing key is added by me after the fact:
docs[i].mc_ign = mc_ign;
I tried logging the entire object before and after I add the key and assign the value. They are both the same.
What am I missing? Why can I read the value out, but cant see it?

It is mongoose document object. To achieve what you want do the following.
docs[0] = docs[0].toObject();
docs[0].mc_ign = "stuff";
console.log(docs[0])
.toObject() convert it to plain JS object.

Mongoose saving empty array error "TypeError: Cannot read property '1' of null"

I have a schema that is defined like so:
const userSchema = new Schema({
...
surveys: [surveyKeySchema],
...
})
Where surveyKeySchema is actually a subdocument scheme defined like so:
const surveyKeySchema = new Schema({
slug: {
type: String,
required: 'Please supply a slug',
unique: true,
lowercase: true,
trim: true
},
name: {
type: String,
required: 'Please supply a name',
trim: true
},
responseCount: {
type: Number,
default: 0
}
})
Now whenever I try to modify anything on the user except for this array, everything goes fine. When instantiating the user, it is also totally fine. I can also call await user.save() in my code right before I empty the array.
It's also fine when I remove any subdocument from the survey as long as there is at least 1 element remaining.
However, when I try to remove the final subdocument using:
await user.surveys.id(sid).remove()
await user.save()
I get an error on the .save() which is just TypeError: Cannot read property '1' of null. I'm confused and can't find anything about this online, I assume it must be requiring at least one subdocument to be present? Is there any way to remove this, or if my assumption is wrong how would I go about resolving this?
Thanks in advance! And my apologies if I'm missing something obvious!
EDIT:
I found that mongoose's mongo error handler was actually throwing this in a regex it was using to parse the error message. Hacking this around to return the raw error message:
E11000 duplicate key error index: db.users.$surveys.slug_1 dup key: { : null }
As per this question I tried adding sparse: true but this didn't work.

For anyone else having this issue, here's what I did:
In node_modules/mongoose-mongodb-errors/lib/plugin.js on line 19, add a simple console.error(err.message) so you can actually get the output and not the regex handler error.
Because when you save an empty array of subdocuments in Mongoose it is equivalent to having the subschema set to values of null, this means that when Mongoose evaluates the indices of your subdocument collection it will evaluate it as having a value of null for each property. If you're indexing with a property (i.e. one of the properties in your subdocument schema has unique: true on it) then this is a violation as a null value cannot be unique, at least not in Mongo world. To get around this you can add sparse: true.
Any documents in the existing collection and the existing collection itself will create an issue with now having a changed index. You need to drop the index for this to work. I dropped the entire collection because I didn't need it anyways.
Here's my updated schema:
const surveyKeySchema = new Schema({
slug: {
type: String,
required: 'Please supply a slug',
unique: true,
lowercase: true,
sparse: true,
trim: true
},
name: {
type: String,
required: 'Please supply a name',
trim: true
},
responseCount: {
type: Number,
default: 0
}
})

Mongoose: how to prevent null values being added to array of ObjectId

Title says it all. Below is a schema property definition that I thought would work, but doesn't do the job. I figured required: true on the type object should work?
users: {
type: [
{
type: Schema.Types.ObjectId,
ref: 'User',
required: true
}
],
default: []
}
the following query allows a null value to be added to the array:
MyModel.findByIdAndUpdate(id, {$addToSet: {users: ['']}})

aaaaaaaand, of course as soon as I post to SO, i find the answer for myself. need to set option runValidators: true on update...
This is the real reason I post to SO, its like always carrying an umbrella, its guaranteed not to rain.