I am converting a json file to a YAML file with https://github.com/nodeca/js-yaml using safeDump
The outcome is like this
en:
models:
errors:
name: name not found
url: bad url
user:
errors:
name: name not found
url: bad url
photo:
errors:
name: name not found
url: bad url
but I want a script to compress with the references
en:
models:
errors: &1
name: name not found
url: bad url
user:
errors: *1
photo:
errors: *1
Based on the Python script from Anthon https://stackoverflow.com/a/55808583/10103951
function buildRefsJson(inputJson, mappings = null) {
if (!mappings) {
mappings = {}
}
if (typeof(inputJson) === 'object') {
let value
let stringValue
let ref
for (let key in inputJson) {
value = inputJson[key]
stringValue = JSON.stringify(value)
ref = mappings[stringValue]
if (ref) {
inputJson[key] = ref
} else {
mappings[stringValue] = value
buildRefsJson(inputJson[key], mappings)
}
}
}
I transformed it to JavaScript code. And it did the work! Also thanks to Niroj for helping
What you want to do is "compressing" the JSON input to YAML with references for those mappings that
have exactly the same key-value pairs. In order to achieve that you need to be able to find
such matching mappings and one way to do that is by creating a lookup table based on
the string representation of the mapping after sorting the keys.
Assuming this JSON input in input.json:
{
"en": {
"models": {
"errors": {
"name": "name not found",
"url": "bad url"
}
},
"user": {
"errors": {
"name": "name not found",
"url": "bad url"
}
},
"photo": {
"errors": {
"name": "name not found",
"url": "bad url"
}
}
}
}
You can convert it with this Python script to get:
import json
import sys
from pathlib import Path
import ruamel.yaml
in_file = Path('input.json')
def optmap(d, mappings=None):
if mappings is None:
mappings = {}
if isinstance(d, dict):
for k in d:
v = d[k]
sv = repr(v)
ref = mappings.get(sv)
if ref is not None:
d[k] = ref
else:
mappings[sv] = v
optmap(d[k], mappings)
elif isinstance(d, list):
for idx, item in d:
sitem = repr(item)
ref = mappings.get(sitem)
if ref is not None:
d[idx] = sitem
else:
mappings[sitem] = item
optmap(item, mappings)
data = json.load(in_file.open())
optmap(data)
yaml = ruamel.yaml.YAML()
yaml.serializer.ANCHOR_TEMPLATE = u'%d'
yaml.dump(data, sys.stdout)
which gives:
en:
models: &1
errors:
name: name not found
url: bad url
user: *1
photo: *1
The above will also make references to, and traverse, arrays in your JSON.
As you can see your output can be further "compressed" than you though it could be.
I am not fluent enough in JavaScript to have written this answer in that language (without investing too much effort and delivering some ugly code), but the OP obviously understood the intent of optmap() and implemented it in his answer
Sadly there's no solution to convert JSON to YML with references as far as I know, cause there's no such 'references' rule for repeating nodes in JSON. As the spec says, YAML can therefore be viewed as a natural superset of JSON.
Related
I would like to store accounts in a json file.
Something like : accounts{[user: user1, email: email1], [user: user2. email: email2]}
Javascript file
Accounts = {
Nickname: form.getElementsByTagName('input')[0].value,
Email: form.getElementsByTagName('input')[3].value
};
var json = JSON.stringify(Accounts);
fs.appendFile('Accounts.json', json, function(err){});
When I add a second user this code make a new object and it looks like.
Json file
{"NickName": "user1", "Email": "Email1"}{"NickName": "user2", "Email": "Email2"}
Then, when I try to read the file and parse it I receive an unexpected error {
I think the Json file should look like
{
{"Name":"value"},
{"Name2":"value2"}
}
I fix the issue.
Here the solution with a mix of Michael Brune's solution.
// Read accounts.json
const Accounts = fs.readFileSync('Accounts.json', 'utf-8');
// Check if json file is empty
if(Accounts.length !== 0){
var ParsedAccounts = JSON.parse(Accounts);
}
else{
ParsedAccounts = [];
}
ParsedAccounts.push(Account);
const NewData = JSON.stringify(ParsedAccounts, null, 4);
// Write new data to accounts.json
fs.writeFileSync('Accounts.json', NewData);
Basically, I push the new data in ParsedAccounts then I write it in the json file.
Maybe there is another way, but if your file is pretty small, then try like this:
Accounts.json:
{
"other": "other data"
}
index.js
const fileData = fs.readFileSync('Accounts.json');
let parsedFileData
try {
parsedFileData = JSON.parse(fileData);
} catch (error) {
if (error instanceof SyntaxError) {
// Create empty object to append to since parse was not a success
parsedFileData = {};
// Create backup of old, possible corrupt file
fs.writeFileSync('Accounts.backup.json', fileData);
}
}
const newFileData = JSON.stringify({
...parsedFileData,
...newDataToAppend,
}, null, 4);
fs.writeFileSync('Accounts.json', newFileData);
First you can parse the file.
Assign new and old data to a object.
Then convert that to srting JSON.Stringify
The null and 4 are to write a nice pretty file
Then write it to the file directly.
The Backend has an end-point called api/Options/GetEmailMessageTemplate, it returns objects which has this schema:
{
messageType: string, Enum: [ ConfirmationEmail,
ConfirmationTransaction, RecoveryPassword, SetupAdminAccount, ConfirmationApiKey, ConfirmationDepositTransaction ]
language: string, Enum: [ En, Ru, Zh, Es, Et ]
subject: string,
template: string,
isUsed: boolean
}
e.g. response:
{
"messageType": 1,
"language": "En",
"subject": "string",
"template": "string",
"isUsed": true
}
here is another end-point to edit it api/Options/Options/UpdateEmailMessageTemplate which consumes json with the same schema as above. messageType might be either number of an element in Enum or Enum value (e.g. 'ConfirmationEmail')
On the Frontend to list all the data and be able to change it I came up with this approach:
I made an strictly ordered array:
messageTypes: [
{
name: 'Confirmation Email',
success: false,
},
...
]
Success is required to show if the change of this template was successful
I get messageTypeas a number id from backend, I just used it as index in my array (so, for this to work my array must be ordered in the exactly same way Enum of that field is ordered ), to get the name of that messageType and operate with success field
3.api/Options/Options/UpdateEmailMessageTemplate gets messageType using index of the currently being edited element (indexOf)
While this approach worked as was expected I can't help but think there was a better way to handle this.
I would like to hear if there are better practices to handle that
Based on my understanding, you are wanting a way to work with a friendly list of values as well as their id's. One approach would be to create two separate classes. This would enable you to feed the raw response to a single model and you can add whichever methods are needed to translate id > name or the other way around.
You could probably get a little more fancier and use get/set but I'm still a little foggy on the requirements. But, here's an approach that I would take:
/**
* Create a class that knows how to translate it
* Bonus points: this could be populated via your endpoint
* that returns your enum list so you don't have to keep
* updating it if there's a change on the backend
*/
class MessageType {
constructor(messageType) {
this.id = messageType;
const messageTypes = [
'ConfirmationEmail',
'ConfirmationTransaction',
'RecoveryPassword',
'SetupAdminAccount',
'ConfirmationApiKey',
'ConfirmationDepositTransaction'
];
this.name = messageTypes[this.id];
}
}
/**
* Create a class / model for your response.
* This will enable you to add any methods
* needed for translating things how you need
* them. For example, you might want a method
* to convert your model into what the backend
* expects.
*/
class MessageTemplate {
constructor(response) {
this.messageType = new MessageType(response.messageType);
this.language = response.language;
this.subject = response.subject;
this.template = response.template;
this.isUsed = response.isUsed;
}
getJsonPayloadForBackend() {
const payload = { ...this };
payload.messageType = payload.messageType.id;
return payload;
}
}
// Usage
const template = new MessageTemplate({
"messageType": 2,
"language": "En",
"subject": "string",
"template": "string",
"isUsed": true
});
console.log(template);
console.log('data to send to backend', template.getJsonPayloadForBackend())
I have a CSV file and I want to parse it using PapaParse. How do I do this properly?
I have so far:
Papa.parse(fileInput, {
download: true,
complete: function(results) {
console.log(results.data);
console.log(results.errors);
}
});
However, is there a better way to do this? Is this the proper way to get errors? The documentation didn't emphasize download: true or anything so I was wondering if there are any experts on this subject here.
EDIT: Also, am I suppose to further parse the file with papacsv or do it in react. For instance, if I have multiple arrays in my data file which have a similar name reference. Should I initially somehow parse the file so it groups all those references together and how would I go about doing this?
For instance,
Date, Name , Win/Lose
I want to group all the winners together. How do I do that?
The method you are using of Papa parse, is for remote CSV.
download: true is for downloading the remote file.
By using Papa parse, this is the only way of getting errors, data, meta with parse result object.
//If(header:true)
var data = [
{
"date": "8/12/2018",
"name": "foo",
"win/loose": "win"
},
{
"date": "8/12/2018",
"name": "foo",
"win/loose": "loose"
},
{
"date": "8/12/2018",
"name": "foo1",
"win/loose": "win"
},
];
var winners = data.filter(d => d['win/loose'] == 'win');
console.log(winners);
//If you want to group winners and losers then:
var grouped = data.reduce(function(acc, co) {
var key = co['win/loose'];
if(!acc[key]) {
acc[key] = [];
}
acc[key].push(co);
return acc;
}, {});
console.log(grouped);
This'll give you separate array of winners from extracted data.
I want to model the following information into json but am unable to do so.
The server sends result of an operation to the client using the following model
class Result (result:string, additional-info:string)
additional-info could contain a json or a string depending on the use case. Thus its type is String. When I need to send a json in it, I simply send a string with a valid json syntax and I suppose the the Angular client would be able to convert the string into a json using JSON.parse.
The json I want to send to the client looks like
{
"result": "success",
"additional-info": {
"list ": [{
"tag": "sometag",
"description": "some description"
}]
}
}
I checked on jsonlint (https://jsonlint.com/) that the structure is correct.
On the client side (Angular), I am handing the message as follows:
getQuestions(some args){
console.log('response from server:',res)
console.log('response body',res.body)
let jsonResponse:ServerResponseAPI = res.body //should contain result and additional info
console.log("result: "+jsonResponse.result+", additional info:"+jsonResponse.additionalInformation)
let jsonList:string = jsonResponse.additionalInformation
console.log("jsonQuestionList: "+jsonList)
let information:Information = JSON.parse(jsonList)
console.log("information:"+information)
});
}
ServerResponseAPI is defined as
export class ServerResponseAPI{
constructor ( public result:string,
public additionalInformation:string){}
}
When I execute the code, I see the following prints on browser's console but I see that error that additional-info is not define.
response body {result: "success", additional-info: "{"list ": [{"tag": "sometag", "description": "some description"}]}"}
list-management.service.ts:46 result: success, additional info:undefined
I can see that the body contains result and additional-info but after casting the body to ServerResponseAPI, I see that result is success but additional-info is undefined.
in res.body, javascript creates an object
{
"result": "success",
"additional-info": {
"list ": [{
"tag": "sometag",
"description": "some description"
}]
}
}
The object has two keys - result and additional-info. Lets call it Object1
I assign it to an object which has keys result and additionalInfo. Note the difference in naming convention in additionalInfo. In javascript, names of variables are case sensitive, so the above two are different. Lets call this object2
Now result from object1 gets assigned to result from object2 because the keys match (same name result)
additional-info becomes a new key in the object2
additionalInfo key of object2 stays undefined as no key from object1 maps to additionalInfo
To solve the issue, I had to create a additional-info key ServerResponseAPI (alternatively I could have also changed my JSON property name to additionalInfo but I didn't want to change that). This is done in Angular as
export class ServerResponseAPI{
'additional-info':string;
constructor ( public result:string,
public additionalInformation:string){
this['additional-info'] = additionalInformation;
}
}
In my code, I now access the keys as
let jsonResponse:ServerResponseAPI = res.body //contains result and additional info
console.log("result: "+jsonResponse.result+", additional info:"+jsonResponse['additional-info'])
let jsonQuestionList:string = jsonResponse['additional-info']
I'm sending a JSON object to ruby with javascript. But I cannot parse it in there. I tried following stuff but no luck. Also I've been searching around for while now, but I couldn't find anything helpful.
Note that I'm very new ruby.
My trials:
def initialize(game_conf_json)
parsed_conf = JSON.parse(conf_json)
#param1 = parsed_conf['name'][0]
#param2 = parsed_conf['surname'][0]
=begin
I also tried this:
#param1 = parsed_conf['name']
#param2 = parsed_conf['surname']
But no matter what other things I try, I'm getting the following error:
"21:in `[]': can't convert String into Integer (TypeError)"
OR
"can't convert Array into String (TypeError), "
=end
File.open("some_direc/conf.json", "w") do |f|
f.write(game_conf_json)
end
end
I create the json in javascript like this:
var json_of_form_vars = {};
json_of_form_vars.name = name_val;
json_of_form_vars.surname = surname_val;
And send it this way:
$.ajax({
url: "../fundament/dispatch.rb/",
type: "post",
dataType: "json",
data: "conf="+json_of_form_vars,
.....
How can I solve this problem? Is there any proper tutorial for this of problem?
UPDATE1 (After suggestions):
I used JSON.stringify and then passed the object to ruby. And then I finally able to print the object in ruby. It's listed below:
{"name": "METIN", "surname": "EMENULLAHI"}
The method .class claims that it is an array. But I cannot access data with classical ways, like:
array['name']
the error is:
can't convert String into Integer
And when I try to pass it to the JSON.parse in ruby, it gives me the following error:
can't convert Array into String
So I used JSON.parse(conf_array.to_json), but again when accessing the data it gives the same error like arrays:
can't convert String into Integer
What should be done now?
UPDATE2
Here is my cgi handler which passes the URL parameters to appropriate places:
cgi_req = CGI::new
puts cgi_req.header
params_from_request = cgi_req.params
logger.error "#{params_from_request}"
action_todo = params_from_request['action'][0].chomp
base = Base.new
if action_todo.eql?('create')
conf_json = params_from_request['conf']
# This line prints the json like this: {"name": "SOME_NAME", "surname": "SOME_SURNAME"}
logger.error "#{conf_json}"
base.create_ident(conf_json)
end
And in Base class:
def create_ident(conf_json)
require 'src/IdentityCreation'
iden_create = IdentityCreation.new(conf_json)
end
IdentityCreation's constructor is listed above.
UPDATE3:
Ok, I now get at least something out of the array. But when I access a key, it displays the key itself:
parsed_conf = JSON.parse(conf_json.to_json)
#param1 = parsed_conf[0]['name']
#param2 = parsed_conf[0]['surname']
# At this point when I print the #param1, it gives me "name"(the key), not "SOME_NAME"(the value).
Here an example of parsing a JSON string. If you still have problems, publish the generated JSON so that we can try it.
require 'json'
require 'ap'
string = %{
{"configurations" : [
{ "drinks" : [
{"menus" : [
{ "hot" : [
{"id":15,"unit":"0.33", "price":"1", "currency":"Euro", "position": 4},
{"id":15,"unit":"0.33", "price":"1", "currency":"Euro", "position": 6}
] },
{ "cold" : [
{"id":15,"unit":"0.33", "price":"1", "currency":"Euro", "position": 4},
{"id":15,"unit":"0.33", "price":"1", "currency":"Euro", "position": 6}
] },
{ "terminals" : { "id" : 4, "id": 6, "id": 7 } },
{ "keys" : { "debit" : "on", "credit": "off" } }
] }
] } ] }
}
hash = JSON.parse(string)
ap hash
gives
{
"configurations" => [
[0] {
"drinks" => [
[0] {
"menus" => [
[0] {
"hot" => [
[0] {
"id" => 15,
"unit" => "0.33",
"price" => "1",
"currency" => "Euro",
"position" => 4
},
etc..
At the moment you're not actually posting json. You're attemping to post json wrapped inside form encoded parameters. In addition, when you do
"conf=" + json_of_form_vars
javascript will convert json_of_form_vars to a string for you but that conversion isn't the same as dumping to JSON. Javascript default string conversions are pretty useless for objects, so you'll need to use JSON.stringify to get actual json.
Since you're composing the body as a string literal you'll also need to escape any special characters that aren't allowed (or have special meaning) in this context. It's usually easier to let jquery do the heavy lifting, with something like
$.ajax({
url: "../fundament/dispatch.rb/",
type: "post",
dataType: "json",
data: {conf: JSON.stringify(json_of_form_vars)}
I solved it finally. Using all the suggestions made under this post and also my irb experiences with hashes, arrays, json and etc.
So instead of converting my object (conf_json) to json (with to_json), I passed it to JSON.parse as a string like below:
parsed_conf = JSON.parse(%{#{conf_json}})
It seems kind of weird to me, because when try to pass it to function like below, I got the error of can't convert Array into String.
parsed_conf = JSON.parse(conf_json)
But it is working like a charm right now.