I have a flask route where I am trying to pipe a list_object [{}] into a template. It works but the data comes out as a string on the front end. How can I convert the string back into a array_object so I can do stuff with it?
Here is the flask route:
app.py
#app.route('/vgmplayer')
def vgmplayer():
musicdata = music.query.order_by(func.random()).first()
musicimgdata = musicimages.query.filter_by(gamealbum = musicdata.gamealbum).order_by(func.random()).first()
if musicimgdata == None:
musicimgdata = "https://media.idownloadblog.com/wp-content/uploads/2018/03/Apple-Music-icon-003.jpg"
else:
musicimgdata = musicimgdata.gameart
musicobject = [{
"name": musicdata.gametrackname,
"path": musicdata.gamelink,
"img": musicimgdata,
"album": musicdata.gamealbum,
"artists": musicdata.artists,
"platform": musicdata.platform,
"year": musicdata.year,
"genre": musicdata.genre
}]
print(musicobject)
return render_template("musicplayer.html",rvgm = musicobject)
but when I get back the template:
musicplayer.html
<script type="text/javascript">
function grvgm(){
All_song = "{{ rvgm|safe }}"
return All_song
}
</script>
it comes in a string:
Example data:
All_song = grvgm()
"[{'name': '10 - M Stage Jungle B', 'path': 'https://vgmsite.com/soundtracks/zhuzhu-pets-quest-for-zhu-20…-nds-gamerip/ebulpznnyw/10%20-%20M%20Stage%20Jungle%20B.mp3', 'img': None, 'album': 'ZhuZhu Pets - Quest for Zhu', 'artists': None, 'platform': 'DS', 'year': '2011', 'genre': None}]"
I would need the list_dict to not have the qoutes at the end so that the javascript can treat it as an array.
EDIT:
I forgot to mention that I tried:
function grvgm(){
All_song = "{{ rvgm }}"
All_song = JSON.parse(All_song)
return All_song
}
SyntaxError: JSON.parse: expected property name or '}' at line 1 column 3 of the JSON data
Turns out the string was not a valid json format as #Andy pointed out. I had to use a different method. Instead of piping the list_dict into the template, I used a get request to get the list_dict into the front end so I can do stuff with it.
app.py
#app.route('/grvgm', methods=['GET'])
def grvgm():
musicdata = music.query.order_by(func.random()).first()
musicimgdata = musicimages.query.filter_by(gamealbum = musicdata.gamealbum).order_by(func.random()).first()
if musicimgdata == None:
musicimgdata = "https://media.idownloadblog.com/wp-content/uploads/2018/03/Apple-Music-icon-003.jpg"
else:
musicimgdata = musicimgdata.gameart
musicobject = [{
"name": musicdata.gametrackname,
"path": musicdata.gamelink,
"img": musicimgdata,
"album": musicdata.gamealbum,
"artists": musicdata.artists,
"platform": musicdata.platform,
"year": musicdata.year,
"genre": musicdata.genre
}]
print(musicobject)
musicobject = json.dumps(musicobject)
return musicobject
musicplayer.js
async function load_track(index_no){
fetch('/grvgm')
.then(response => response.json())
.then(All_song => {
console.log(All_song)
// stuff goes here
})
By using the JSON.parse() method, you can easily parse the string.
Example:
JSON.parse(All_song)
// returns an array
Related
I have a file with data in it that I am needing to parse and store in a DB. Below, is an example of 2 entries in the file. I'm not quite sure what the structure is (although it looks to be ndJSON). I am trying to parse the data in to a JSON object in order to store it in a DB, but cannot seem to figure it out. Here is what I have so far
var ndjson = {
"sequence-num": "0123456789",
"version": "N1.4",
"record-type": "R",
"session-id": "197-30760303",
"date": "2021-07-23 15:00:53",
"passport-header": { "alg": "ES256", "ppt": "test", "typ": "passport", "x5u": "https://cr.com" },
"passport-payload": { "attest": "A", "dest": { "tn": ["0123456789"] }, "iat": 0123456789, "orig": { "tn": "0123456789" }, "origid": "c699f78a-ebc6-11eb-bfd8-bec0bbc98888" },
"identity-header": "eyJhbGciOiJFUzI1NiIsInBwdCI6InNoYWtlbiIsInR5cCI6InBhc3Nwb3J0IiwieDV1IjoiaHR0cHM6Ly9jci5zYW5zYXkuY29tL1RvdWNodG9uZV82ODNBIn0.eyJhdHRlc3QiOiJCIiwiZGVzdCI6eyJ0biI6WyIxMjUeyJhdHRlc3QiOiJCIiwiZGVzdCI6eyJ0biI6WyIxMj;info=<https://google.com/>;alg=ES256;ppt=\"test\""
}
{
"sequence-num": "0123456788",
"version": "N1.4",
"record-type": "R",
"session-id": "214-30760304",
"date": "2021-07-23 15:00:53",
"passport-header": { "alg": "ES256", "ppt": "test", "typ": "passport", "x5u": "https://cr.com" },
"passport-payload": { "attest": "B", "dest": { "tn": ["0123456788"] }, "iat": 0123456788, "orig": { "tn": "0123456788" }, "origid": "c69d0588-ebc6-11eb-bfd8-bec0bbc98888" },
"identity-header": "eyJhbGciOiJFUzI1NiIsInBwdCI6InNoYWtlbiIsInR5cCI6InBhc3Nwb3J0IiwieDV1IjoiaHR0cHM6Ly9jci5zYW5zYXkuY29tL1RvdWNodG9uZV82ODNBIn0.eyJhdHRlc3QiOiJCIiwiZGVzdCI6eyJ0biI6WyIxMjUeyJhdHRlc3QiOiJCIiwiZGVzdCI6eyJ0biI6WyIxMj;info=<https://google.com/>;alg=ES256;ppt=\"test\""
};
let result = ndjson.split(',').map(s => JSON.parse(s));
console.log('The resulting array of items:');
console.log(result);
console.log('Each item at a time:');
for (o of result) {
console.log("item:", o);
}
When I run this, I get Uncaught SyntaxError: Unexpected token ':' error on line 12 at the 2nd node of "sequence-num": "0123456788",.
Any help is appreciated, thank you!
If you actually have ndJSON(newline-delimited JSON) then each line in the file is valid JSON, delimited by newlines. A simple file would look like this:
{"key1": "Value 1","key2": "Value 2","key3": "Value 3","key4": "Value 4"}
{"key1": "Value 5","key2": "Value 6","key3": "Value 7","key4": "Value 8"}
This differs from the formatted data you've posted here, and the difference is important since once you've formatted it, the valid JSON objects cannot simply be distinguished by the presence of newlines.
So, on the assumption that you do have valid ndJSON, in its original form, you can extract it by using split() on newLines and using JSON.parse() on the resulting array.
This snippet adds a little file handling to allow a file to be uploaded, but thereafter it uses split() and JSON.parse() to extract the data:
"use strict";
document.getElementsByTagName('form')[0].addEventListener('submit',function(e){
e.preventDefault();
const selectedFile = document.getElementById('inputFile').files[0];
let fr = new FileReader();
fr.onload = function(e){
let ndJSON = e.target.result; // ndJSON extracted here
let ndJSONLines = ndJSON.split('\n');
// Process JSON objects here
ndJSONLines.forEach(function(el){
let obj = JSON.parse(el);
Object.keys(obj).forEach(key=>{
console.log(`Key: ${key}, Value: ${obj[key]}`);
});
});
}
fr.readAsText(selectedFile)
});
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Parsing ndJSON</title>
</head>
<body>
<form method="post" enctype="multipart/form-data">
<input type="file" name="inputFile" id="inputFile">
<input type="submit">
</form>
</body>
</html>
Output, based on the sample file above:
Here is what I do
const end_point_url = 'https://ipfs.io/ipfs/bafkqap33ejwgs3tfgerduitomrvhg33oebtg64tnmf2c4lroej6qu6zcnruw4zjsei5ce5dinfzsa2ltebqsa43fmnxw4zbanruw4zjcpufa';
let json = await fetch(end_point_url).
then( resp => resp.text() ).
then( buf => { // NDJSON format ...
return buf.slice(0,-1).split('\n').map(JSON.parse);
}).
catch(console.error);
I'm working on a command that will automatically fetch a file from a link once a day and extract two of the elements in it and send that as a message in a channel.
My issue here is that I'm having issues actually getting the file downloaded. I've been trying several different functions to fetch the file but nothing has worked so far. I have attached one of the functions I've tried below.
async function getQuote () {
const url = "https://quotes.rest/qod?category=inspire";
const path = Path.resolve(__dirname, 'temp', 'qod.json')
const writer = fs.CreateWriteStream(path)
const response = await axios({
url,
method: 'GET',
responseType: 'stream'
})
response.data.pipe(writer)
getQuote();
return new Promise((resolve, reject) => {
writer.on('finish', resolve)
writer.on('error', reject)
})
}
fs.readFile('./temp/qod.json', 'utf8', function (err, data) {
if (err) throw err;
var obj = JSON.parse(data);
msg.channel.send(data);
})
The file I'm trying to work with here looks something like this:
{
"success": {
"total": 1
},
"contents": {
"quotes": [
{
"quote": "What you do speaks so loudly that I cannot hear what you say.",
"length": "61",
"author": "Ralph Waldo Emerson",
"tags": [
"action",
"inspire",
"leadership",
"management",
"tod"
],
"category": "inspire",
"language": "en",
"date": "2020-08-23",
"permalink": "https://theysaidso.com/quote/ralph-waldo-emerson-what-you-do-speaks-so-loudly-that-i-cannot-hear-what-you-say",
"id": "eZ0NtMPtGp8c5eQJOBfJmweF",
"background": "https://theysaidso.com/img/qod/qod-inspire.jpg",
"title": "Inspiring Quote of the day"
}
]
},
"baseurl": "https://theysaidso.com",
"copyright": {
"year": 2022,
"url": "https://theysaidso.com"
}
}
It wants to download as a json file, but when visiting the link, it is listed as a xml document.
How would I go about getting this downloaded and extracting two lines from it? If you're wondering, the two lines are the quote and author lines.
Thanks!
I copy your code and run my local machine and everythin fine.
Limitations are like mirages created by your own mind. When you realise that limitation do not exist, those around you will also feel it and allow you inside their space. - Stephen Richards
Looks like you are trying to write the result to a file and then read from the file which is not efficient. Here's a much simpler way of doing it.
async function getQuote() {
const url = "https://quotes.rest/qod?category=inspire";
const response = await axios(url);
const result = response.data;
/*
result =
{
"success": {
"total": 1
},
"contents": {
"quotes": [
{
"quote": "Limitations are like mirages created by your own mind. When you realise that limitation do not exist, those around you will also feel it and allow you inside their space. ",
"length": "171",
"author": "Stephen Richards",
"tags": [
"inspire",
"motivational",
"positive-thinking",
"self-empowerment",
"self-help",
"self-improvement",
"wealth",
"wealth-creation"
],
"category": "inspire",
"language": "en",
"date": "2020-08-24",
"permalink": "https://theysaidso.com/quote/stephen-richards-limitations-are-like-mirages-created-by-your-own-mind-when-you",
"id": "OLSVpLiSwrWplvCcFgPPiweF",
"background": "https://theysaidso.com/img/qod/qod-inspire.jpg",
"title": "Inspiring Quote of the day"
}
]
},
"baseurl": "https://theysaidso.com",
"copyright": {
"year": 2022,
"url": "https://theysaidso.com"
}
}
*/
//this is an array of quote objects
const quotes = result.contents.quotes;
//extracting first quote object from the array
const quoteObject = quotes[0];
//extracting quote text and author from quote object
const quote = quoteObject.quote;
const author = quoteObject.author;
//the >>> will make it look like a quote in discord.
console.log(`>>> ${quote}\n- ${author}`);
//send the formatted quote to the channel
msg.channel.send(`>>> ${quote}\n- ${author}`);
//if for some reason you want to save the result to a file
fs.writeFile(filePath, result, function(err) {
if (err) throw err;
console.log('Saved!');
});
}
getQuote();
<script src="https://cdn.jsdelivr.net/npm/axios/dist/axios.min.js"></script>
I would suggest simply reading the quote to an object, then creating a string using interpolation and send it on the discord channel:
async function getQuote () {
const url = "https://quotes.rest/qod?category=inspire";
console.log("getQuote: Reading quote...");
// Get the response as an object
const response = await axios({
url,
method: 'GET'
})
// Use destructuring to get the quote and author
let { quote, author } = response.data.contents.quotes[0];
// Format our quote
let data = `${quote} - ${author}`;
// Add a console.log for debugging purposes..
console.log("getQuote: Sending quote:", data);
// Send the quote on the channel
msg.channel.send(data);
}
Todays quote would then look like so:
Limitations are like mirages created by your own mind. When you realise that limitation do not exist, those around you will also feel it and allow you inside their space. - Stephen Richards
I am converting a json file to a YAML file with https://github.com/nodeca/js-yaml using safeDump
The outcome is like this
en:
models:
errors:
name: name not found
url: bad url
user:
errors:
name: name not found
url: bad url
photo:
errors:
name: name not found
url: bad url
but I want a script to compress with the references
en:
models:
errors: &1
name: name not found
url: bad url
user:
errors: *1
photo:
errors: *1
Based on the Python script from Anthon https://stackoverflow.com/a/55808583/10103951
function buildRefsJson(inputJson, mappings = null) {
if (!mappings) {
mappings = {}
}
if (typeof(inputJson) === 'object') {
let value
let stringValue
let ref
for (let key in inputJson) {
value = inputJson[key]
stringValue = JSON.stringify(value)
ref = mappings[stringValue]
if (ref) {
inputJson[key] = ref
} else {
mappings[stringValue] = value
buildRefsJson(inputJson[key], mappings)
}
}
}
I transformed it to JavaScript code. And it did the work! Also thanks to Niroj for helping
What you want to do is "compressing" the JSON input to YAML with references for those mappings that
have exactly the same key-value pairs. In order to achieve that you need to be able to find
such matching mappings and one way to do that is by creating a lookup table based on
the string representation of the mapping after sorting the keys.
Assuming this JSON input in input.json:
{
"en": {
"models": {
"errors": {
"name": "name not found",
"url": "bad url"
}
},
"user": {
"errors": {
"name": "name not found",
"url": "bad url"
}
},
"photo": {
"errors": {
"name": "name not found",
"url": "bad url"
}
}
}
}
You can convert it with this Python script to get:
import json
import sys
from pathlib import Path
import ruamel.yaml
in_file = Path('input.json')
def optmap(d, mappings=None):
if mappings is None:
mappings = {}
if isinstance(d, dict):
for k in d:
v = d[k]
sv = repr(v)
ref = mappings.get(sv)
if ref is not None:
d[k] = ref
else:
mappings[sv] = v
optmap(d[k], mappings)
elif isinstance(d, list):
for idx, item in d:
sitem = repr(item)
ref = mappings.get(sitem)
if ref is not None:
d[idx] = sitem
else:
mappings[sitem] = item
optmap(item, mappings)
data = json.load(in_file.open())
optmap(data)
yaml = ruamel.yaml.YAML()
yaml.serializer.ANCHOR_TEMPLATE = u'%d'
yaml.dump(data, sys.stdout)
which gives:
en:
models: &1
errors:
name: name not found
url: bad url
user: *1
photo: *1
The above will also make references to, and traverse, arrays in your JSON.
As you can see your output can be further "compressed" than you though it could be.
I am not fluent enough in JavaScript to have written this answer in that language (without investing too much effort and delivering some ugly code), but the OP obviously understood the intent of optmap() and implemented it in his answer
Sadly there's no solution to convert JSON to YML with references as far as I know, cause there's no such 'references' rule for repeating nodes in JSON. As the spec says, YAML can therefore be viewed as a natural superset of JSON.
I have a JSON link who is giving me data like -
"data": [{
"name": "123Seguro",
"domain": "123seguro.com",
"id": 1,
"slug": "123seguro"
}, {
"name": "Earn.com",
"domain": "21.co",
"id": 2,
"slug": "earn-com"
}, {
"name": "2C2P",
"domain": "2c2p.com",
"id": 3,
"slug": "2c2p"
}, {
"name": "4thWay",
"domain": "4thway.co.uk",
"id": 5,
"slug": "4thway"
}, {
"name": "51credit.com",
"domain": "51credit.com",
"id": 6,
"slug": "51-credit-card-manager"
}, ........................................
Is there is a way to take these data to Google sheet in 4 columns as it is giving 4 column data?
I tried with a script like this -
function fetch() {
var response = UrlFetchApp.fetch('mylink.json');
var res1 = response.getContentText();
var json = JSON.parse(res1);
var name = json["name"];
Logger.log(name);
}
But it is not reading the data and logger returning "UNDEFINED". can somebody help here, please.
Edit-1
On a small research, I found a script like this -
function IMPORTJSON(url,xpath){
try{
// /rates/EUR
var res = UrlFetchApp.fetch(url);
var content = res.getContentText();
var json = JSON.parse(content);
var patharray = xpath.split("/");
//Logger.log(patharray);
for(var i=0;i<patharray.length;i++){
json = json[patharray[i]];
}
//Logger.log(typeof(json));
if(typeof(json) === "undefined"){
return "Node Not Available";
} else if(typeof(json) === "object"){
var tempArr = [];
for(var obj in json){
tempArr.push([obj,json[obj]]);
}
return tempArr;
} else if(typeof(json) !== "object") {
return json;
}
}
catch(err){
return "Error getting data";
}
}
Here I am applying the formula like this - =IMPORTJSON("https://theAPILINK.json","data")
And the response is like
0
1
2
3
4
.
.
.......And at the moment I am changing from "data" to "name" it is showing "Node Not Available".
Any suggestions, please. :)
First of all it's important to understand your JSON data structure:
Data
-> 0
-> -> Name: 123Seguro
-> -> Domain: 123seguro.com
-> -> id: 1
-> -> slug: 123seguro
-> 1
-> -> Name: Earn.com
-> -> Domain: 21.co
-> -> id: 2
-> -> slug: earn-com
-> 2
....
So to reach your data, you'll always need to first open data and then select a specific "row".
function fetch() {
var response = UrlFetchApp.fetch('mylink.json');
var res1 = response.getContentText();
var json = JSON.parse(res1);
var name = json["name"];
Logger.log(name);
}
But it is not reading the data and logger returning "UNDEFINED". can somebody help here, please.
This is because you first need to open data as explained above. Then define the item of which you want to know the name. Try something like Logger.log(json["data"][0]["name"]). This should return the name of the first item in your JSON response.
Here I am applying the formula like this - =IMPORTJSON("https://theAPILINK.json","data") And the response is like 0 1 2 3 4 . . .......And at the moment I am changing from "data" to "name" it is showing "Node Not Available".
So you tell the function where to find your data and the root node (data). When you change data to name it is unable to find an element called like that. So it will return the error you've mentioned.
I would recommend to dive into your first fetch example instead of using the other function. It'll help you to understand the mechanism and process the data as you require.
I have this python code, which fetches json and parses it:
from django.http import HttpResponse
import json, requests
def find(request):
context = {}
platformUrl = 'https://www.igdb.com/api/v1/platforms'
platformReq = requests.get(platformUrl, headers={'Authorization': 'Token token="1234"'})
platformData = json.loads(platformReq.text)
platformList = platformData['platforms']
print platformList
It outputs this with the print statement:
[{u'slug': u'saturn', u'id': 32, u'name': u'Sega Saturn'}, {u'slug': u'mac', u'id': 14, u'name': u'Mac'}, {u'slug': u'vc', u'id': 47, u'name': u'Virtual Console (Nintendo)'}
I would like to pass that data to javascript and have the ID and name from the json put in this javascript. (selectize.js) This would probably require some type of for loop, could be javascript, or even djangos?:
options: [
{id: 1, title: 'Spectrometer'},
{id: 2, title: 'Star Chart'},
{id: 3, title: 'Electrical Tape'}
]
Thanks
EDIT:
Following #satoru's link, I looked into it, and updated my code to this:
from django.http import HttpResponse
import json, requests
def find(request):
context = {}
platformUrl = 'https://www.igdb.com/api/v1/platforms'
platformReq = requests.get(platformUrl, headers={'Authorization': 'Token token="1234"'})
platformList = json.dumps(platformData)
print platformList
return render_to_response('find.html',{'platformList':platformList})
It now outputs this:
{"platforms": [{"slug": "saturn", "id": 32, "name": "Sega Saturn"}, {"slug": "mac", "id": 14, "name": "Mac"}, {"slug": "vc", "id": 47, "name": "Virtual Console (Nintendo)"}
How would I then pass that into javascript?
If the API already return a JSON-encoded response, you can just use platformReq.text. After passing it to render_to_response, you have access to a JSON-encoded string in your template.
Now you can design your Javascript module to be configurable with an object, for example:
<script src="my_module.js"></script>
<script>
MyModule.initialize({{ json_encoded }})
</script>
Check out this template library to convert your platform list to a JSON object from within the template
Example(Inside your template):
<script src="your_standalone_js.js">
<script>
var js_variable = {{platformList | jsonify }};
function_in_standalone_js(js_variable);
</script>
your_standalone_js.js:
var function_in_standalone_js = function(js_variable){
var options = js_variable['platforms'];
//What ever you need to do with options
};