Count Occurrences of Intrinsics Using Esprima - javascript

I am working on an application that allows user-submitted web-pages with embedded JavaScript code.
I would like to learn more about how my users are writing their JavaScript code, by building a map of common JavaScript intrinsics (built in objects) which shows the occurrence of each in user generated code.
Assume I have a JS file from a user's webpage:
Main.js
for (let i = 0; i < 10; i++) { myArrs.push(new Array()); }
let myObj = new Object();
I would like a script that can generate the following output:
Array: 10
Object: 1
I have attempted to do this using both regex and string traversal, but this does not account for the case where an item is used in an iteration. Simple string traversal would yield me with:
Array: 1
Object: 1
Which is not correct.
Esprima seems to provide a solution to this as it can perform syntactic analysis since it is a JS parser. I have attempted to use esprima.parseScript(input, config, delegate) to generate a tree and than traverse the tree but the output still does not take into account iterations.
Here is the output from my attempt at parsing this information using Esprima:
{
"type": "Program",
"body": [
{
"type": "ForStatement",
"init": {
"type": "VariableDeclaration",
"declarations": [
{
"type": "VariableDeclarator",
"id": {
"type": "Identifier",
"name": "i"
},
"init": {
"type": "Literal",
"value": 0,
"raw": "0"
}
}
],
"kind": "let"
},
"test": {
"type": "BinaryExpression",
"operator": "<",
"left": {
"type": "Identifier",
"name": "i"
},
"right": {
"type": "Literal",
"value": 10,
"raw": "10"
}
},
"update": {
"type": "UpdateExpression",
"operator": "++",
"argument": {
"type": "Identifier",
"name": "i"
},
"prefix": false
},
"body": {
"type": "BlockStatement",
"body": [
{
"type": "ExpressionStatement",
"expression": {
"type": "CallExpression",
"callee": {
"type": "MemberExpression",
"computed": false,
"object": {
"type": "Identifier",
"name": "myArrs"
},
"property": {
"type": "Identifier",
"name": "push"
}
},
"arguments": [
{
"type": "NewExpression",
"callee": {
"type": "Identifier",
"name": "Array"
},
"arguments": []
}
]
}
}
]
}
},
{
"type": "VariableDeclaration",
"declarations": [
{
"type": "VariableDeclarator",
"id": {
"type": "Identifier",
"name": "myObj"
},
"init": {
"type": "NewExpression",
"callee": {
"type": "Identifier",
"name": "Object"
},
"arguments": []
}
}
],
"kind": "let"
}
],
"sourceType": "script"
}
I was not able to find this answer on SO already - it seems like this is a useful problem to solve but requires a bit of knowledge in lexical analysis tools such as Esprima.

Related

How do I require one field to be (A and B), or ( A), or (B), or (A and C), or (C)

I'm having trouble coming up with a schema that can accommodate these scenarios:
This field can have A, and/or only one of B or C
This field can have only A
This field can have only one of B or one of C
A, B and C have the same schema.
Basically B and C can not exist simultaneously.
Example scenarios:
// Valid
{
"exampleItem": {
"A": [{
"key": "item A",
"description": "test desc"
}],
"B": [{
"key": "item B",
"description": "test desc"
}]
}
}
// Invalid
{
"exampleItem": {
"B": [{
"key": "item B",
"description": "test desc"
}],
"C": [{
"key": "item C",
"description": "test desc"
}]
}
}
What I've tried:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"exampleItemSchema": {
"type": "object",
"oneOf": [
{
"type": "object",
"properties": {
"A": {
"type": "array",
"items": {
"type": "object",
"required": [ "key", "description" ],
"properties": {
"key": {
"type": "string"
},
"description": {
"type": "string"
}
}
}
},
"B": {
"type": "array",
"items": {
"type": "object",
"required": [ "key", "description" ],
"properties": {
"key": {
"type": "string"
},
"description": {
"type": "string"
}
}
}
}
}
},
{
"type": "object",
"properties": {
"A": {
"type": "array",
"items": {
"type": "object",
"required": [ "key", "description" ],
"properties": {
"key": {
"type": "string"
},
"description": {
"type": "string"
}
}
}
},
"C": {
"type": "array",
"items": {
"type": "object",
"required": [ "key", "description" ],
"properties": {
"key": {
"type": "string"
},
"description": {
"type": "string"
}
}
}
}
}
}
]
}
}
}
I'm using oneOf right now, but I'm quite certain it's not what I want. I can't use anyOf because I don't want B and C to exist together, and allOf would assert against A, B and C.
To assert that a property can not exist, you can either do:
{
"not": {
"required": [
"property name"
]
}
}
or:
{
"properties": {
"property name": false
}
}
You can assert that a property must exist with the required keyword.
You can combine schemas together with allOf, anyOf, and oneOf.
Remember that subschemas are not just used to influence the overall result, but can be used as pre-conditions for another subschema to be evaluated, and that subschema will form part of the overall evaluation results. The keywords not, and if/then/else use preconditions to decide how they will evaluate. The conditions in a not and if are only used to determine a true/false result, and nothing else asserted in those subschemas will be used in the final result.
This should be all the tools in the toolbox that you need.
(For readability, I would keep all the properties keywords (that define what A, B and C should look like, should they exist and are permitted) at the top layer, and then use sub-assertions with if/then/else and not just to state which properties must or must not exist, and in what combinations.)

JSON schema definitions or oneOf in array of objects?

I need to express an array of different objects in a schema. The array, called contents, can contain any number of elements, but they must be one of two types of object: One type of object represents a piece of text, the other type of object represents an image.
So far, I've not been able to find a way to enforce the validation correctly. It seems (nested) required inside a oneOf doesn't work, so I tried using definitions but that doesn't seem to fit.
I've tried a few online validators, but they seem happy for me to add illegal values in item-2 value object. Its the value property that seems to be the biggest problem. Unfortunately, due to legacy issues, I'm stuck with this being an object in an array.
Is it possible to validate and enforce correct type/requirements for this object?
(this is the data, not a schema. Unfortunately, we also used the keyword type when we design the original json layout!)
{
"uuid":"780aa509-6b40-4cfe-9620-74a9659bfd59",
"contents":
[
{
"name":"item-1",
"label":"My editable text Label",
"value":"This text is editable",
"type":"text"
},
{
"name":"item-2",
"label":"My editable image label",
"index":0,
"type":"image",
"value":
[
{
"name":"1542293213356.png",
"rect":[0,0,286,286]
}
]
}
],
"version":"2.0"
}
Well, I think this is it, though the online validator doesn't seem 100% reliable. Editing the values isn't always invalidating the object.
{
"$schema": "http://json-schema.org/draft-06/schema#",
"type": "object",
"properties": {
"uuid": { "type": "string" },
"version": { "type": "string" },
"contents": {
"$ref": "#/definitions/contents"
},
},
"required": ["uuid", "version"],
"definitions": {
"image": {
"type": "object",
"properties": {
"name": { "type": "string" },
"label": { "type": "string" },
"type": { "enum": ["image", "text"] },
"value": { "type": "object" }
},
"required": ["name", "label", "type", "value"]
},
"text": {
"type": "object",
"properties": {
"name": { "type": "string" },
"label": { "type": "string" },
"type": { "enum": ["image", "text"] },
"value": { "type": "string" }
},
"required": ["name", "label", "type", "value"]
},
"contents": {
"type": "array",
"contains": {
"oneOf": [
{ "$ref": "#/definitions/image" },
{ "$ref": "#/definitions/text" }
]
},
},
},
}

Using anyOf to see objects name?

I have a schema here where I would like to have a drop down, to select an option, and from there - depending on the selection different options appear; all being nested within an array to have multiples of them.
I have noticed that when I am filling in with dummy data, the output json isnt storing the name of the selected option
so the data.json looks something like this:
{
"page1": [
{
"imageOptions": {
"imageHeightType": "vh",
"imageHeight": 50
},
"textboxArea": {
"headerText": "Header for selection1",
"headingTag": "h1",
"textBoxOpacity": 15
}
},
{
"content": "This is a complety different selection, yet there is no name to tell the difference between these two difference objects"
}
]
}
As you can see theres no object to wrap these two different items within the page1 array - Ideally would like something like:
{
"page1": [
{
// Title of object goes here from schema
"imageOptions": {
"imageHeightType": "vh",
"imageHeight": 50
},
"textboxArea": {
"headerText": "Header for selection1",
"headingTag": "h1",
"textBoxOpacity": 15
}
},
{
// Title of object goes here from schema
"content": "This is a completely different selection, yet there is no name to tell the difference between these two difference objects"
}
]
}
Is there a way to make this so? I have looked on the docs for AnyOf but not much luck. Quite new to React-JsonSchema-Forms.
Below is my current Schema:
{
"type": "object",
"properties": {
"page1": {
"type": "array",
"items": {
"type": "object",
"anyOf": [
{
"title": "Full Width Image",
"type": "object",
"properties": {
"imageOptions": {
"type": "object",
"title": "Image",
"properties": {
"image": {
"type": "string",
"title": "Image",
"format": "data-url"
},
"imageHeightType": {
"enum": [
"px",
"vh"
]
},
"imageHeight": {
"type": "number",
"title": "Image Height"
}
}
},
"textboxArea": {
"type": "object",
"title": "Textbox Area",
"properties": {
"headerText": {
"type": "string",
"title": "Heading Text"
},
"headingTag": {
"enum": [
"h1",
"h2",
"h3"
]
},
"imageText": {
"type": "string",
"title": "Body Text"
},
"textboxPosition": {
"title": "Textbox Position",
"enum": [
"Left",
"Center",
"Right"
]
},
"textboxColor": {
"title": "Color of Textbox Area",
"type": "string"
},
"textBoxOpacity": {
"title": "Textbox Opacity %",
"type": "integer",
"minimum": 0,
"maximum": 100,
"multipleOf": 5
}
}
}
}
},
{
"title": "Custom Block",
"type": "object",
"properties": {
"content": {
"type": "string"
}
}
}
]
}
}
}
}
Also Link to the online schema editor if it helps understand my issue
Why not just add a name-like property to each object? You can then hide/disable it if you want:
schema:
"anyOf": [
{
"title": "Full Width Image",
"type": "object",
"properties": {
"name": {
"type": "string",
"default": "fullWidthImage"
},
"imageOptions": {
"type": "object",
"title": "Image",
"properties": {...}
...
}
...
}
},
{
"title": "Custom Block",
"type": "object",
"properties": {
"name": {
"type": "string",
"default": "custom"
},
"content": {
"type": "string"
}
}
}
]
uiSchema:
{
"page1": {
"items": {
"name": {
"ui:widget": "hidden"
},
"imageOptions": {...},
...
}
}
formData then should look like this:
{
"page1": [
{
"name": "fullWidthImage",
"imageOptions": {
"imageHeightType": "vh",
"imageHeight": 50
},
"textboxArea": {
"headerText": "Header for selection1",
"headingTag": "h1",
"textBoxOpacity": 15
}
},
{
"name": "custom",
"content": "This is a complety different selection, yet there is no name to tell the difference between these two difference objects"
}
]
}

What is the best way to replace text in json?

So I have a bunch of JSON data and it contains a few fields. for example:
[{
"id": "XXX",
"version": 1,
"head": {
"text": "Main title",
"sub": {
"value": "next"
},
"place": "secondary"
},
"body": [{
"id": "XXX1",
"info": "three little birds",
"extended": {
"spl": {
"text": "song",
"type": {
"value": "a"
}
}
}
},
{
"id": "XXX2",
"info": [
"how are you?"
],
"extended": {
"spl": {
"text": "just",
"non-type": {
"value": "abc"
}
}
}
}
]
}]
what I'm trying to do is kind of conversion table (from a different JSON file)
if a field has the value 'a' replace it with 'some other text..' etc.
I have a service for the JSON pipeline, so I guess this is the right place to do the replacement.
so for this example, I have the JSON above and in my conversion table I have the following terms:
next: forward,
song: music,
a: option1,
just: from
etc...
What you are looking for can be achieved with templates. Replace the variable sections with some specific markers that you can find and replace from some external tools such as perl or sed.
For example, you could have a template.json with something like this:
...
"type": {
"value": "##VALUE##"
}
...
Then when you need the actual JSON, you could pass this though an intermediate script that replaces these templates with actual data.
cat template.json | sed -e 's/##VALUE##/my_value/' > target.json
Alternatively, with Perl:
cat template.json | perl -pi -e 's:\#\#VALUE\#\#:my_value:' > target.json
The best way is to parse it, replace the text in the object, and then stringify it.
The next best way is to use a regular expression.
In this example, I catch exceptions if path cannot be indexed, and use ['type'] instead of .type so it will scale to indexing 'non-type' if you wish.
const data = `[{
"id": "XXX",
"version": 1,
"head": {
"text": "Main title",
"sub": {
"value": "next"
},
"place": "secondary"
},
"body": [{
"id": "XXX1",
"info": "three little birds",
"extended": {
"spl": {
"text": "song",
"type": {
"value": "a"
}
}
}
},
{
"id": "XXX2",
"info": [
"how are you?"
],
"extended": {
"spl": {
"text": "just",
"non-type": {
"value": "abc"
}
}
}
}
]
}]
`
const o = JSON.parse(data)
o[0].body.forEach(b => {
try {
if (b.extended.spl['type'].value === 'a') {
b.extended.spl['type'].value = 'CHANGED'
}
} catch (e) {}
})
const newData = JSON.stringify(o, null, 2)
console.log(newData)
A string replace approach will work if you know and can rely on your source conforming, such as the only "value" is inside "type"
const data = `[{
"id": "XXX",
"version": 1,
"head": {
"text": "Main title",
"sub": {
"value": "next"
},
"place": "secondary"
},
"body": [{
"id": "XXX1",
"info": "three little birds",
"extended": {
"spl": {
"text": "song",
"type": {
"value": "a"
}
}
}
},
{
"id": "XXX2",
"info": [
"how are you?"
],
"extended": {
"spl": {
"text": "just",
"non-type": {
"value": "abc"
}
}
}
}
]
}]
`
const newData = data.replace(/"value": "a"/g, '"value": "NEWVALUE"')
console.log(newData)

JSONSchema and validating sub-object properties

Given this JSON object:
{
"objects": {
"foo": {
"id": 1,
"name": "Foo"
},
"bar": {
"id": 2,
"name": "Bar"
}
}
}
This is an object containing sub objects where each sub object has the same structure - they're all the same type. Each sub-object is keyed uniquely, so it acts like a named array.
I want to validate that each object within the objects property validates against a JSON Schema reference.
If the objects property was an array, such as:
{
"objects": [
{
"id": 1,
"name": "Foo"
},
{
"id": 2,
"name": "Bar"
}
]
}
I could validate this with a schema definition such as:
{
"id": "my-schema",
"required": [
"objects"
],
"properties": {
"objects": {
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"name"
],
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string"
}
}
}
}
}
}
This is achieved because the type is array, and this permits the validation of items.
Is it possible to do something similar, but with nested objects?
Thanks!
You can try something like this:
{
"id": "my-schema",
"type": "object",
"properties": {
"objects": {
"type": "object",
"patternProperties": {
"[a-z]+": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string"
}
},
"additionalProperties": false,
"required": [
"id",
"name"
]
}
}
}
}
}
The above answer works for restricting the object property names to lower-case letters. If you do not need to restrict the property names, then this is simpler:
{
"id": "my-schema",
"type": "object",
"properties": {
"objects": {
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string }
}
},
"required": [
"id",
"name"
]
}
}
}
}
I omitted the inner "additionalProperties": false from the above answer because I find that using that keyword causes more problems than it solves, but it is a valid use of the keyword if you want validation to fail on the inner objects if they have properties other than "name" and "id".

Categories