I have a (long) json schema and I already validate the json objects in my code based on that schema.
My problem/use-case is to translate/restructure an incoming json object to a new json object that satisfies my schema.
I realized however, that it isn't that easy to do, especially if you look for an "adaptive" solution.
Adaptive meaning that the code doesn't care about changes of the schema or the incoming object.
So my first approach was to create a class that contains a mapping of the incoming object to a valid object.
This means however that I have to write this map by hand and update it if I change the schema.
So is there a solution where I can insert any schema, any non-valid (based on schema) json object and get a valid object out?
Or at-least a way to generate a valid json object template, where you can insert the data of the incoming invalid json object?
This way I could at least be independent from schema changes.
PS: I tried to solve this problem in an abstract way, but if needed I can provide schema and the invalid object.
PPS: My environment is node.js.
Edit: E.g.:
Schema
{
...//other Schema stuff (using draft-07)
"tokens": {
"$id": "/token",
"title": "Token",
"description": "A representation of a word in a Message with all the nlp Analysis results and infos",
"type": "array",
"minItems": 1,
"items": {
"$ref": "#/definitions/tokens/definitions/token"
},
"token": {
"$id": "#token",
"type": "object",
"required": [
"id",
"text"
],
"properties": {
"id": {
"description": "global (level:conversation) unique identifier to identify a token (cumulative)",
"type": "number",
"exclusiveMinimum": 0,
"uniqueItems": true
},
"isText": {
"description": "Describes if the token is a real semantic word or a whitespace or similar",
"type": "boolean"
},
"text": {
"description": "the actual content of the token",
"type": "string",
"pattern": "^(.+)$"
},
"word_index": {
"description": "Index of a word in a text",
"type": "number",
"exclusiveMinimum": 0,
"uniqueItems": true
},
"punctuation": {
"description": "The punctuation that needs to be rendered before/after that token. TODO: finish it",
"type": "string"
},
"characterOffsetBegin": {
"type": "integer",
"description": "The inclusive character index in the sentence where this token begins"
},
"characterOffsetEnd": {
"type": "integer",
"description": "The exclusive character index in the sentence where this token ends"
},
"lemma": {
"type": "string",
"pattern": "^(.+)$"
},
"pos": {
"type": "string",
"description": "part of speech tag",
"pattern": "^(.+)$"
},
}
},
"text_label": {
//This is where it gets difficult
"description": "Label of the text for the frontend. ",
"type": "object",
"required": [
"name",
"begin",
"end"
],
"additionalProperties": false,
"properties": {
"name": {
"description": "Name of the label",
"type": "string"
},
"begin": {
"description": "Token index where the label starts",
"type": "number"
},
"end": {
"description": "Token indes where the label ends",
"type": "number"
}
}
}
}
}
Object to translate:
{
"text": "Hello, my name is Somebody. ",
"sentences": [
{
"index": 0,
"tokens": [
{
"index": 1,
"word": "Hello",
"originalText": "Hello",
"characterOffsetBegin": 0,
"characterOffsetEnd": 5,
"before": "",
"after": "",
"pos": "UH",
"lemma": "hello"
},
{
"index": 2,
"word": ",",
"originalText": ",",
"characterOffsetBegin": 5,
"characterOffsetEnd": 6,
"before": "",
"after": " ",
"pos": ",",
"lemma": ","
},
{
"index": 3,
"word": "my",
"originalText": "my",
"characterOffsetBegin": 7,
"characterOffsetEnd": 9,
"before": " ",
"after": " ",
"pos": "PRP$",
"lemma": "my"
},
{
"index": 4,
"word": "name",
"originalText": "name",
"characterOffsetBegin": 10,
"characterOffsetEnd": 14,
"before": " ",
"after": " ",
"pos": "NN",
"lemma": "name"
},
{
"index": 5,
"word": "is",
"originalText": "is",
"characterOffsetBegin": 15,
"characterOffsetEnd": 17,
"before": " ",
"after": " ",
"pos": "VBZ",
"lemma": "be"
},
{
"index": 6,
"word": "Somebody",
"originalText": "Somebody",
"characterOffsetBegin": 18,
"characterOffsetEnd": 26,
"before": " ",
"after": "",
"pos": "NN",
"lemma": "somebody"
},
{
"index": 7,
"word": ".",
"originalText": ".",
"characterOffsetBegin": 26,
"characterOffsetEnd": 27,
"before": "",
"after": "",
"pos": ".",
"lemma": "."
}
],
"basicDependencies": []
}
]
}
Note: that this is corenlp analysis result that can change based on the annotator used.
Edit2: adding example of a correct object:
{ ...other parts of the object
tokens:[
{ id:1, text: 'Hello', isText: true, word_index: 0, characterOffsetBegin:0, characterOffsetEnd: 5, lemma: 'hello', pos:'UH', ...},
{...},
{...}
],
text_label:{
name: 'joy', begin: 0, end: 3
}
}
or another text_label:
text_label:{
name: 'PERSON', begin: 4, end: 5
}
Edit: Difference:
As you can see, some changes a suttle like index->word_index, some are additional information like id, but some need a bit more like the text_label.
corenlp. It could also happen that I need to unify 2 objects together. which is the case of the 'anger' text_label but isnt with the PERSON text_label.
So Im looking for some kind of json Object/Schema manager where translations, validations get easy and dont need hard coded mappings or the generation of that mappings is easy.
Related
I have a json file.
[
{
"line": 1,
"elements": [
{
"start_timestamp": "2022-10-17T20:07:41.706Z",
"steps": [
{
"result": {
"duration": 12216552000,
"status": "passed"
},
"line": 5,
"name": "m0e",
"match": {
"location": "seleniumgluecode.book.user_is_on_homepagee()"
},
"keyword": "Given "
},
{
"result": {
"duration": 2074982200,
"status": "passed"
},
"line": 6,
"name": "m1e1",
"match": {
"location": "seleniumgluecode.book.user_enters_Usernamee()"
},
"keyword": "When "
}
],
"tags": [
{
"name": "#Smokee"
}
]
},
{
"start_timestamp": "2022-10-17T20:08:12.284Z",
"steps": [
{
"result": {
"duration": 12090584100,
"status": "passed"
},
"line": 12,
"name": "m0e2",
"match": {
"location": "seleniumgluecode.book2.user_is_on_homepageee()"
},
"keyword": "Given "
}
],
"tags": [
{
"name": "#Smokee"
}
]
}
],
"name": "Login Featuree",
"description": " Verify if user is able to Login in to the sitee",
"id": "login-featuree",
"keyword": "Feature",
"uri": "file:src/test/java/features/tribe/squad1/kitab.feature",
"tags": []
},
{
"line": 1,
"elements": [
{
"start_timestamp": "2022-10-17T20:08:34.480Z",
"steps": [
{
"result": {
"duration": 11366098500,
"status": "passed"
},
"line": 5,
"name": "m0e",
"match": {
"location": "seleniumgluecode.book.user_is_on_homepagee()"
},
"keyword": "Given "
}
],
"tags": [
{
"name": "#Smokee"
}
]
}
],
"name": "Login Featureefghfgh",
"description": " Verify if user is able to Login in to the sitee",
"id": "login-featureefghfgh",
"keyword": "Feature",
"uri": "file:src/test/java/features/tribe1/squad2/kitab2.feature",
"tags": []
},
{
"line": 19,
"elements": [
{
"start_timestamp": "2022-10-17T20:09:40.836Z",
"steps": [
{
"result": {
"duration": 12761711100,
"status": "passed"
},
"line": 23,
"name": "m0e",
"match": {
"location": "seleniumgluecode.book.user_is_on_homepagee()"
},
"keyword": "Given "
}
],
"tags": [
{
"name": "#Smokee"
}
]
}
],
"name": "X Feature",
"description": " Verify if user is able to Login in to the sitee",
"id": "login-featuree",
"keyword": "Feature",
"uri": "file:src/test/java/features/tribe2/test.feature",
"tags": []
}
]
I am getting url addresses in this array
document.addEventListener("DOMContentLoaded", () => {
var i = report.length;
var array = [];
for(x = 0; x < i; x++){
array.push(report[x].uri.split("/"));
}
console.log(array2);
});
This return me :
0:
(7) ['file:src', 'test', 'java', 'features', 'tribe1', 'squad1', 'kitab.feature']
1:
(7) ['file:src', 'test', 'java', 'features', 'tribe1', 'squad2', 'kitab2.feature']
2:
(6) ['file:src', 'test', 'java', 'features', 'tribe2, kitab3.feature']
I don't need file:src, test, java, features. Deleting them in 3 arrays and getting a unique array like this:
0:
(3) ['tribe1', 'squad1', 'kitab.feature']
1:
(3) ['tribe1', 'squad2', 'kitab2.feature']
2:
(2) ['tribe2, kitab3.feature']
Finally, if there are 2 elements before the .feature, I need to create a new array by considering 1 as squad and 2 as tribe. Like this:
Diagram
[tribe1
squad1
elem
1
2
name
url
squad2
elem
1
2
name
url
tribe2
elem
1
2
name
url
]
How can I do that?. Thank you.
You should try destructing the array. An example is shown below:
[a,b,c,d, ...rest] = ['file:src', 'test', 'java', 'features', 'tribe1', 'squad1', 'kitab.feature']
console.log(rest);
You can transform or create a new array based on input array with this simple logic with the help of Array.map() along with String.split() and Array.splice() method.
Live Demo :
const arr = [
{
"line": 1,
"uri": "file:src/test/java/features/tribe/squad1/kitab.feature"
},
{
"line": 1,
"uri": "file:src/test/java/features/tribe1/squad2/kitab2.feature"
},
{
"line": 19,
"uri": "file:src/test/java/features/tribe2/test.feature"
}
];
const res = arr.map(({ uri}) => uri.split('/').splice(4));
console.log(res);
So I have a bunch of JSON data and it contains a few fields. for example:
[{
"id": "XXX",
"version": 1,
"head": {
"text": "Main title",
"sub": {
"value": "next"
},
"place": "secondary"
},
"body": [{
"id": "XXX1",
"info": "three little birds",
"extended": {
"spl": {
"text": "song",
"type": {
"value": "a"
}
}
}
},
{
"id": "XXX2",
"info": [
"how are you?"
],
"extended": {
"spl": {
"text": "just",
"non-type": {
"value": "abc"
}
}
}
}
]
}]
what I'm trying to do is kind of conversion table (from a different JSON file)
if a field has the value 'a' replace it with 'some other text..' etc.
I have a service for the JSON pipeline, so I guess this is the right place to do the replacement.
so for this example, I have the JSON above and in my conversion table I have the following terms:
next: forward,
song: music,
a: option1,
just: from
etc...
What you are looking for can be achieved with templates. Replace the variable sections with some specific markers that you can find and replace from some external tools such as perl or sed.
For example, you could have a template.json with something like this:
...
"type": {
"value": "##VALUE##"
}
...
Then when you need the actual JSON, you could pass this though an intermediate script that replaces these templates with actual data.
cat template.json | sed -e 's/##VALUE##/my_value/' > target.json
Alternatively, with Perl:
cat template.json | perl -pi -e 's:\#\#VALUE\#\#:my_value:' > target.json
The best way is to parse it, replace the text in the object, and then stringify it.
The next best way is to use a regular expression.
In this example, I catch exceptions if path cannot be indexed, and use ['type'] instead of .type so it will scale to indexing 'non-type' if you wish.
const data = `[{
"id": "XXX",
"version": 1,
"head": {
"text": "Main title",
"sub": {
"value": "next"
},
"place": "secondary"
},
"body": [{
"id": "XXX1",
"info": "three little birds",
"extended": {
"spl": {
"text": "song",
"type": {
"value": "a"
}
}
}
},
{
"id": "XXX2",
"info": [
"how are you?"
],
"extended": {
"spl": {
"text": "just",
"non-type": {
"value": "abc"
}
}
}
}
]
}]
`
const o = JSON.parse(data)
o[0].body.forEach(b => {
try {
if (b.extended.spl['type'].value === 'a') {
b.extended.spl['type'].value = 'CHANGED'
}
} catch (e) {}
})
const newData = JSON.stringify(o, null, 2)
console.log(newData)
A string replace approach will work if you know and can rely on your source conforming, such as the only "value" is inside "type"
const data = `[{
"id": "XXX",
"version": 1,
"head": {
"text": "Main title",
"sub": {
"value": "next"
},
"place": "secondary"
},
"body": [{
"id": "XXX1",
"info": "three little birds",
"extended": {
"spl": {
"text": "song",
"type": {
"value": "a"
}
}
}
},
{
"id": "XXX2",
"info": [
"how are you?"
],
"extended": {
"spl": {
"text": "just",
"non-type": {
"value": "abc"
}
}
}
}
]
}]
`
const newData = data.replace(/"value": "a"/g, '"value": "NEWVALUE"')
console.log(newData)
I've got a JSON Schema that looks like this:
{
"required": [
"instructions"
],
"properties": {
"instructions": {
"title": "Instructions",
"minItems": 3,
"type": "array",
"items": {
"required": [
"typeId",
"teamId",
"disciplineId"
],
"properties": {
"typeId": {
"minimum": 1,
"title": "Appointment Type",
"type": "integer"
},
"teamId": {
"minimum": 1,
"title": "Team",
"type": "integer"
},
"disciplineId": {
"minimum": 1,
"title": "Discipline",
"type": "integer"
},
"prefClinicianId": {
"title": "Pref. Clinician",
"anyOf": [
{
"type": "null"
},
{
"minimum": 1,
"type": "integer"
}
]
},
"prefTime": {
"title": "Pref. Time",
"anyOf": [
{
"type": "null"
},
{
"type": "integer"
}
]
},
"childRequired": {
"title": "Child Req'd",
"type": "boolean"
}
},
"type": "object"
}
}
},
"type": "object"
}
As you can see, I've added titles to all the properties. However, the error object I get back looks like:
[
{
"keyword": "minItems",
"dataPath": ".instructions",
"schemaPath": "#/properties/instructions/minItems",
"params": {
"limit": 3
},
"message": "should NOT have less than 3 items"
},
{
"keyword": "type",
"dataPath": ".instructions[0].typeId",
"schemaPath": "#/properties/instructions/items/properties/typeId/type",
"params": {
"type": "integer"
},
"message": "should be integer"
},
{
"keyword": "type",
"dataPath": ".instructions[0].teamId",
"schemaPath": "#/properties/instructions/items/properties/teamId/type",
"params": {
"type": "integer"
},
"message": "should be integer"
},
{
"keyword": "type",
"dataPath": ".instructions[0].disciplineId",
"schemaPath": "#/properties/instructions/items/properties/disciplineId/type",
"params": {
"type": "integer"
},
"message": "should be integer"
}
]
As you can see, the title is not in there. How can I get the titles with the errors?
Please note that this question is specific to AJV.
When you create your AJV object set the verbose option to true.
This will add a parentSchema property to the ajv error with the original schema. It will also add a schema property that contains the specific schema attribute that caused the validation failure.
Here's an example:
var ajv = new Ajv({
$data: true,
verbose: true
});
let schema = {
title: 'object title',
type: 'object',
properties: {
str: {
title: "A string property",
type: "string"
}
}
};
let data = {
str: 3
};
ajv.validate(schema, data)
console.log('ERRORS: ', this.ajv.errors)
<script src="https://cdnjs.cloudflare.com/ajax/libs/ajv/5.3.0/ajv.bundle.js"></script>
I am using Breeze.js in my project and the following query returns me all the group entities that I have in local cache although only one of them has group==group
breeze.EntityQuery.from('Groups').using(manager).where('group', '==', 'group').executeLocally();
Here my metadata definition:
{
"shortName": "Group",
"namespace": "CM.Models",
"baseTypeName": "Entity",
"autoGeneratedKeyType": "Identity",
"defaultResourceName": "Groups",
"dataProperties": [
{
"name": "groupID",
"dataType": "String",
"maxLength": 32,
"defaultValue": "",
"validators": [
{
"name": "maxLength",
"maxLength": 32
}
]
},
{
"name": "group",
"dataType": "String",
"maxLength": 32,
"defaultValue": "",
"validators": [
{
"name": "required"
},
{
"name": "maxLength",
"maxLength": 32
}
]
},
{
"name": "groupMembers",
"dataType": "String",
"isScalar": false,
"defaultValue": []
}
]
}
Is it a bug of Breeze.js?
For reference, I found the answer after digging into breeze.js code. It turns out that it is possible to escape the evaluation side. Therefore the query becomes:
breeze.EntityQuery.from('Groups').using(manager).where('group', '==', "'group'").executeLocally();
"'group'" insetad 'group'
I am new to javascript and to JSON, so please forgive me.
If I have the following JSON Object, how would I console.log() out the value of autonum?
{
"database": "testdb",
"table": "path",
"affectedColumns": [
{
"name": "autonum",
"charset": null,
"type": 8
},
{
"name": "TimeStamp",
"charset": null,
"type": 18,
"metadata": {
"decimals": 0
}
},
{
"name": "FilePath",
"charset": "latin1",
"type": 15,
"metadata": {
"max_length": 256
}
},
{
"name": "DirPath",
"charset": "latin1",
"type": 15,
"metadata": {
"max_length": 256
}
},
{
"name": "DirName",
"charset": "latin1",
"type": 15,
"metadata": {
"max_length": 256
}
},
{
"name": "EventName",
"charset": "latin1",
"type": 15,
"metadata": {
"max_length": 256
}
},
{
"name": "FileName",
"charset": "latin1",
"type": 15,
"metadata": {
"max_length": 256
}
},
{
"name": "FileExt",
"charset": "latin1",
"type": 15,
"metadata": {
"max_length": 10
}
},
{
"name": "FileSize",
"charset": null,
"type": 3
},
{
"name": "MainFlag",
"charset": null,
"type": 1
},
{
"name": "DeleteFlag",
"charset": null,
"type": 1
},
{
"name": "Status",
"charset": "latin1",
"type": 15,
"metadata": {
"max_length": 255
}
},
{
"name": "ProcessedFlag",
"charset": null,
"type": 1
}
],
"changedColumns": [],
"fields": {
"autonum": 121,
"TimeStamp": "2016-01-13T00:21:13.000Z",
"FilePath": "c:/1E0304F120151223030158001.mp4",
"DirPath": "c:\\",
"DirName": null,
"EventName": null,
"FileName": "1E0304F120151223030158001.mp4",
"FileExt": ".mp4",
"FileSize": 2218108,
"MainFlag": 0,
"DeleteFlag": 0,
"Status": null,
"ProcessedFlag": 0
}
}
Depending on which autonum you want, and assuming your variable storing the JSON is data, you'll want to do something as follows:
console.log(data.fields.autonum);
or
console.log(data.affectedColumns[0].name);
You'd use the following code, assuming the json object is called record:
console.log(record.fields.autonum);
You have two main ways of doing it. Both are correct ways. Lets say your object is named obj. Use console.log as following:
console.log(obj.fields.autonum)
console.log(obj['fields']['autonum'])
First case is easier as compare to second case.
Second case is safer as it will allow you to even take care of keys which have spaces e.g.
var my_other_object = {
'Santa Clara': 'USA',
'Toronto': 'Canada'
};
console.log(my_other_object['Santa Clara']) // Output will be 'USA'
For your understanding, in above object 'Santa Clara' and 'Toronto' are called 'keys' of my_other_object and 'USA', 'Canada' are called 'values' of those 'keys'.
So JSON object is essentially combination of key:value pairs.
P.S. Never apologize while asking a question all questions are valid but it is good to always search before asking. Still, people are always happy to help here. We all have went through same phases. :)
Lets say your object name var data, then data.fields.autonum would give us value 121
console.log(data.fields.autonum)
Just FYI:
As it is stated on MDN website:
some JavaScript is not JSON, and some JSON is not JavaScript
Just in order to improve understanding of vocabulary, 'JSON Object' is not a thing in our context ... it is
a syntax for serializing objects