I'm trying to fetch contents inside a script tag from a url using node-fetch and then trying to json parse the data but i keep getting a return undefined.
I'm trying to get the content from the variable game from the html below and then stringify and then parse the json but it returns undefined.
Page html:
<!DOCTYPE html>
<head>
<meta charset="UTF-8" />
<title>Document Title</title>
</head>
<body>
<div id="welcome-div">
<p>Welcome to the my website</p>
</div>
<script>
var game = new Game({
stage: "prod",
servers: {
v32306117: {
id: "v32306117",
name: "name #1",
hostname: "hostname1",
port: 80,
},
v32306125: {
id: "v32306125",
name: "name #2",
hostname: "hostname2",
port: 80,
}
},
userGroup: 0
});
game.init(function() {
game.assetManager.load([{
"name": "\/asset\/image\/map\/grass.png",
"url": "\/asset\/image\/map\/grass.png"
}]);
game.debug.init();
game.run();
});
</script>
</body>
</html>
Fetch function:
const fetch = require("node-fetch");
async function serversFetch() {
try {
const servers = await fetch("https://get-servers.herokuapp.com/");
const data = await servers.text();
const servers_data = data.substring(
data.lastIndexOf("var game =") + 20,
data.lastIndexOf("game.init") - 10
);
return JSON.stringify(servers_data);
} catch (error) {
console.log(error);
}
}
(async () => {
const data = await serversFetch();
console.log('data', data);
const info = JSON.parse(data);
console.log('info', info.servers); // returns undefined
})()
if i console log info.servers it comes back undefined but if i console log just info it logs the output below.
info {
stage: "prod",
servers: {
v32306117: {
id: "v32306117",
name: "name #1",
hostname: "hostname1",
port: 80,
},
v32306125: {
id: "v32306125",
name: "name #2",
hostname: "hostname2",
port: 80,
}
},
userGroup: 0
}
The issue you are running into is because JSON.stringify only works on JavaScript objects and servers_data is a string. What this results in is info being a string later on and that's why console.log(info.servers) logs undefined. When you checked console.log(info), it only had the appearance of working correctly since it was logging the string value of the object. You could test this by performing console.log(typeof info) and you'll see it's of type string.
What you are looking for is to have servers_data be a valid JSON string instead of being a string of a JavaScript object (which is missing all those double-quotes around object property names which JSON requires). The first option might be to brute force it and replace those properties with properties wrapped in double-quotes, i.e. servers: for "servers:" (the colon is included to make it more unique, but it's still not bulletproof). That doesn't help you with properties like v32306117 which are likely unique and can't be replaced easily using the brute force replacement which is looking for known properties.
The next option would likely be to create a parser for the string which could parse it into an abstract syntax tree (AST) for JavaScript. You could then map that to the AST of JSON easily and then reform it as a string using a parser which understands the AST of JSON. Most parsers which use ASTs can parse a string to an AST and create a string from an AST. These parsers are often written using a recursive descent parser. Though this is a great exercise for programmers, you could likely find some libraries which implement AST parsers for JavaScript and JSON. Also, it's a bit of overkill with what you are trying to accomplish.
The final option which I think would be the easiest to implement and easiest to maintain would be to use JSON5, which is a superset of JSON. Using JSON5.parse, you could parse servers_data as is and not have to worry about JSON formatting. This is because JSON5 accepts properties without the double-quotes and is much more lenient about the formatting (double-quotes vs single-quotes, etc.).
Related
Edit to clarify:I have an object from an API response that I get using the mounted function, data is saved not displayed yet. I need to be able to filter that data by allowing a user to input text in an input box before showing it on the page, then find where that keyword was used in a specific key value(name). Then show the results on a page but include other key/value pairs from the api array. This is what the api response looks like:
class: (...)
facets: (...)
numberFound: (...)
results: Array(202)
[0 … 99]
0:
class: "SearchResult"
contentGuid: "7f19462f-6c25-43a9-bdb5-479f5f42fbde"
dateUpdated: "2018-03-27T16:46:31Z"
description: "Converting a Word Document to Adobe Acrobat PDF Learning Services Converting a Word Document to Adobe Acrobat PDF Enterprise Converting a Word Document to Adobe Acrobat PDF / Reference ..."
document: Object
documentGuid: "035f5c69-d406-4c16-86ca-de12773a0963"
documentId: 154424
documentVersionId: 44043
fileId: 74213
format: "PDF"
id: "Document#1#44043"
isFavorite: false
languages: "English"
name: "Converting a Word Document to Adobe Acrobat PDF"
numberOfIndexedCoobs: 0
numberOfSharedLinks: 1
packageType: "PDF"
previewId: 74213
publicLinkTokens: Array(1)
resourceType: "Other"
score: 0.0054571675
snippets: Object
updatedById: 994
updatedByName: "Michael"
versionName: "3"
For example if someone enters "Adobe" in the search box, I would need to search for the word "adobe" in the name value for the entire object, and only show the ones that have "abobe" somewhere in the name value.
My thought was to get the document name split it, then do an includes() to check for the search term. This works but I can't seem to figure out how to get it all to work together and get the results on the screen, plus get additional information, such as document Id from the original results. this is what I have so far:
async getResults() {
return axios
.get(this.url, {
headers: {
"Content-Type": "application/json",
"Bravais-prod-us-Context": this.getCookie(),
},
})
.then((res) => {
this.search = res.data;
this.search.results.forEach((doc) => {
this.results = doc.document.name
.toLowerCase()
.split(" ")
.includes(this.termSearch.toLowerCase());
console.log(doc.document.name.split(" "));
console.log(this.results);
});
})
.catch((error) => console.log(error));
},
I need to show the original title(some words and acronyms are capitalized) plus the doc id(for url links) and a description, all of this info is in the initial api response.
<div v-for="" v-bind:key="">
{{ ???? }}
</div>
That works in the console but how do I get this back together and on the screen?? Any help is appreciated, not looking for someone else to do my coding, just need some advice.
I would start by diving your logic. At the moment you have a single function that makes an api call and then searches through the results. It would be better suited to have the api call in a separate method so that if the user searches multiple times it doesn't call the api each time. We can easily solve this by adding an extra method that checks if the results object is populated and decides which methods to call.
Casting all strings to lowercase is a good idea to normalize the data. There might be other ways but this works for it's intended purpose. However, splitting a string is unecessary as the includes() method searches through the whole string. See the MDN docs for String.prototype.includes()
To search within an array you can use the filter() method, which will create a new array with all elements that pass the implemented test. See the MDN docs for Array.prototype.filter().
With this in hand, we can write our logic as:
async handleSearch(searchString) {
if (!this.results.length) {
this.getResults()
}
this.searchResults(searchString)
},
async getResults() {
return axios.get(this.url, {
headers: {
"Content-Type": "application/json",
"Bravais-prod-us-Context": this.getCookie(),
},
}).then((res) => {
this.results = res.data.results
}).catch((error) => console.error(error));
},
searchResults(searchString) {
this.filteredResults = this.results.filter(item => {
let name = item.name.toLowerCase();
let searchTerm = searchString.toLowerCase();
return this.name.includes(searchTerm)
})
}
Your input field will call the handleSearch() method, and then you can write you html as such:
<div v-for="result in filteredResults" :key="result.id">
<p>Name: {{result.name}}</p>
<p>Description: {{result.description}}</p>
</div>
Well say I have a database, which is a specific "mock" for testing purposes, hence an memory based sqlite database -- where to main database uses postgres:
import knexBuilder = require("knex");
const theDb = knexBuilder({
client: 'sqlite3',
connection: {
filename: ":memory:"
},
useNullAsDefault: true,
pool: {
min: 1,
max: 1,
idleTimeoutMillis: 360000 * 1000
},
});
await theDb.schema.createTable("test", (table) => {
table.increments('id');
table.json("json_test").nullable();
});
When I call column info on this await theDb.table('test').columnInfo();
this is the response:
{
"id":{"type":"integer","maxLength":null,"nullable":false,"defaultValue":null},
"json_test":{"type":"json","maxLength":null,"nullable":true,"defaultValue":null}
}
However when I try to do an insert-retrieve circle like this:
await theDb.insert({
id: 1,
test_json: {
"abc": 10,
"def": "hello",
"ghi": 20,
},
}).into("test");
const retrieved = await theDb.select('id', 'test_json').from('test').where('id', 1);
The result is: [{id: 1, test_json: null}];. Which seems to indicate that the json is not stored? Printing the query with "toString()" shows it does put the json in the values:
insert into `test` (`id`, `test_json`) values (1, {"abc":10,"def":"hello","ghi":20})
I notice that SQLITE has by default no support for JSON, however there is a supported JSON extension for SQLite: https://www.sqlite.org/json1.html
So how would I tell knex "json is fine, just use the extension"? Or any other idea where I can use json without code breaking for tests? The rest of the code depends on JSON and it would be kind of silly to change it everywhere to use text/JSON serializing "if ran during testing".
I'm a little bit new to programming and very new to JS so I apologize for the beginner question.
I'm trying to iterate through this data and get each tracks name and artist but I'm having an issue. Currently I'm trying something like this.
If anybody has any insight or suggestions I would appreciate it greatly.
I'm using a rails backend with JS frontend. Thank you!
function selectTracks(){
fetch(BACKEND_URL)
.then(response => response.json())
.then(playlist => {
playlist.data.forEach(playlist => {
`<h4> ${playlist.attributes.track.name}</h4>
<h4>${playlist.attributes.track.artist}></h4> `
// let newPlaylist = new Playlist(playlist, playlist.attributes)
console.log(fetch)
// document.getElementById("playlist-container").innerHTML += newPlaylist.renderPlaylistCard();
debugger
}
)}
)
}
My serializer looks like this
{
data: [
{
id: "1",
type: "playlist",
attributes: {
name: "Country Songs",
id: 1,
track_id: 10,
track: {
id: 10,
name: "First Song",
artist: "Randy",
created_at: "2020-06-17T02:09:07.152Z",
updated_at: "2020-06-17T02:09:07.152Z"
}
},
relationships: {
track: {
data: {
id: "10",
type: "track"
}
}
}
}
]
}
You need to replace forEach with map. The 'forEachloop doesn't return anything. But themapmethod return an array. (An array of HTML elements in your case
fetch(BACKEND_URL)
.then(response => response.json())
.then(playlist => {
return playlist.data.map(playlist => {
`<h4> ${playlist.attributes.track.name}</h4>
<h4>${playlist.attributes.track.artist}></h4> `
// let newPlaylist = new Playlist(playlist, playlist.attributes)
console.log(fetch)
// document.getElementById("playlist-container").innerHTML += newPlaylist.renderPlaylistCard();
debugger
}
)}
)
Your code technically works assuming that the BACKEND_URL is correct and the json is valid. But, in its current state, it doesn't do anything with the data. If you output the h4 tags, for instance, you should see them written to the screen.
document.write(
`<h4> ${playlist.attributes.track.name}</h4>
<h4>${playlist.attributes.track.artist}</h4> `
)
Or alternatively you could log the values out to prove that you are processing the data correctly:
console.log(playlist.attributes.track.name, playlist.attributes.track.artist)
If not, the next thing to check is the validity of your json. I'm assuming that you copied your json from the browser which will strip some quotes for readability. View the source to ensure that the key names are correctly wrapped in quotes like this:
{
"data": [
{
"id": "1",
"type": "playlist",
"attributes": {
"name": "Country Songs",
...
If you are using ActiveModel Serializers, they should be formatted correctly.
If your json is valid and you can write the h4 tags to the page with the correct data in them, then the problem probably lies in your Playlist class.
Another handy tool for diagnosing fetch() problems is in Chrome Developer Tools. Go to the Network and click the XHR filter. This will allow you to inspect the fetch request and see if the response is valid and the data is what you expect. Other browsers have a similar feature.
I have been trying to figure out how to do 2fa with webauthn and I have the registration part working. The details are really poorly documented, especially all of the encoding payloads in javascript. I am able to register a device to a user, but I am not able to authenticate with that device. For reference, I'm using these resources:
https://github.com/cedarcode/webauthn-ruby
https://www.passwordless.dev/js/mfa.register.js
And specifically, for authentication, I'm trying to mimic this js functionality:
https://www.passwordless.dev/js/mfa.register.js
In my user model, I have a webauthn_id, and several u2f devices, each of which has a public_key and a webauthn_id.
In my Rails app, I do:
options = WebAuthn::Credential.options_for_get(allow: :webauthn_id)
session[:webauthn_options] = options
In my javascript, I try to mimic the js file above and I do (this is embedded ruby):
options = <%= raw #options.as_json.to_json %>
options.challenge = WebAuthnHelpers.coerceToArrayBuffer(options.challenge);
options.allowCredentials = options.allowCredentials.map((c) => {
c.id = WebAuthnHelpers.coerceToArrayBuffer(c.id);
return c;
});
navigator.credentials.get({ "publicKey": options }).then(function (credentialInfoAssertion)
{
// send assertion response back to the server
// to proceed with the control of the credential
alert('here');
}).catch(function (err)
{
debugger
console.error(err); /* THIS IS WHERE THE ERROR IS THROWN */
});
The problem is, I cannot get past navigator.credentials.get, I get this error in the javascript console:
TypeError: CredentialsContainer.get: Element of 'allowCredentials' member of PublicKeyCredentialRequestOptions can't be converted to a dictionary
options at the time navigator.credentials.get is called looks like this:
I've tried every which way to convert my db-stored user and device variables into javascript properly encoded and parsed variables but cannot seem to get it to work. Anything obvious about what I'm doing wrong?
Thanks for any help,
Kevin
UPDATE -
Adding options json generated by the server:
"{\"challenge\":\"SSDYi4I7kRWt5wc5KjuAvgJ3dsQhjy7IPOJ0hvR5tMg\",\"timeout\":120000,\"allowCredentials\":[{\"type\":\"public-key\",\"id\":\"OUckfxGNLGGASUfGiX-1_8FzehlXh3fKvJ98tm59mVukJkKb_CGk1avnorL4sQQASVO9aGqmgn01jf629Jt0Z0SmBpDKd9sL1T5Z9loDrkLTTCIzrIRqhwPC6yrkfBFi\"},{\"type\":\"public-key\",\"id\":\"Fj5T-WPmEMTz139mY-Vo0DTfsNmjwy_mUx6jn5rUEPx-LsY51mxNYidprJ39_cHeAOieg-W12X47iJm42K0Tsixj4_Fl6KjdgYoxQtEYsNF-LPhwtoKwYsy1hZgVojp3\"}]}"
This is an example of the serialised JSON data returned by our implementation:
{
"challenge": "MQ1S8MBSU0M2kiJqJD8wnQ",
"timeout": 60000,
"rpId": "identity.acme.com",
"allowCredentials": [
{
"type": "public-key",
"id": "k5Ti8dLdko1GANsBT-_NZ5L_-8j_8TnoNOYe8mUcs4o",
"transports": [
"internal"
]
},
{
"type": "public-key",
"id": "LAqkKEO99XPCQ7fsUa3stz7K76A_mE5dQwX4S3QS6jdbI9ttSn9Hu37BA31JUGXqgyhTtskL5obe6uZxitbIfA",
"transports": [
"usb"
]
},
{
"type": "public-key",
"id": "nbN3S08Wv2GElRsW9AmK70J1INEpwIywQcOl6rp_DWLm4mcQiH96TmAXSrZRHciZBENVB9rJdE94HPHbeVjtZg",
"transports": [
"usb"
]
}
],
"userVerification": "discouraged",
"extensions": {
"txAuthSimple": "Sign in to your ACME account",
"exts": true,
"uvi": true,
"loc": true,
"uvm": true
}
}
This is parsed to an object and the code used to coerce those base64url encoded values is:
credentialRequestOptions.challenge = WebAuthnHelpers.coerceToArrayBuffer(credentialRequestOptions.challenge);
credentialRequestOptions.allowCredentials = credentialRequestOptions.allowCredentials.map((c) => {
c.id = WebAuthnHelpers.coerceToArrayBuffer(c.id);
return c;
});
Hope that helps. The JSON data is retreived via a fetch() call and the byte[] fields are encoded as base64url on the serverside.
When I apply "console.log" on an JS Object, the console output this thing :
I20170421-14:54:09.786(2)? Document {
I20170421-14:54:09.787(2)? _id: 'KQ7mdidtcxsQsqNjr',
I20170421-14:54:09.787(2)? name: 'eos test',
I20170421-14:54:09.787(2)? number: 69526,
I20170421-14:54:09.788(2)? part: 'bus',
I20170421-14:54:09.788(2)? active: true,
I20170421-14:54:09.789(2)? cron: 6,
What is this 'Document' ??? How I can remove to comapre this Object with the same without 'Document'...
I'm lost !
This document is an output of 'findOne'. I use Meteor with some packages (mongo#1.1.16, aldeed:simple-schema, aldeed:collection2, mdg:validated-method, mdg:validation-error, dburles:collection-helpers).
Thanks :)
You could compare manually (obj2.field1 === obj2.field1) or just use JSON.stringify in both objects and compare the resulting string.
But in your case I think you are not retrieving the object from mongo as you should.
Use fetch() after the query: DocumentCollection.find({}).fetch() or just .findOne() instead.
Also, if you want to compare 2 Meteor documents, you can use:
_.isEqual(doc1, doc2) (underscore)