I am looking for a better way of passing data to my index.js file in a webdev application. Note I really only have about a month of webdev experience so this is likely due to lack of experience. here is my software flow:
I query data in a route handler before the page is rendered. I then pass this data to the rendered page (note i need to keep some of the key-vals hidden. However aggregation works).
exports.getPlotView = async(req, res, next) =>{
//grab the module to query from, stored as var.testModel
const qParse = new PlotQueryParse(req.query).parseObj();
// console.log(qParse)
// const testblockName = qParse.testblock+"Name" ;
// const limitName = qParse.limitname;
const aggregationObj = {$match:
{'jobId':qParse.jobId, '<key2>':<val2>, "<key>":"<val>"}
}
const data = await qParse.testModel.aggregate([aggregationObj]);
console.log(data[0])
const dataString = JSON.stringify(data[0]);
//parse the url to make the query
res.status(200).render('testPlotView', {
pageHeader: "Test",
subHead: "Test summary",
IPn: "IPn",
inData:dataString
});
}
data is passed to pug template. The template uses this as a hidden element
extends base
block content
div.hide_data #{inData}
div#dataviz
now in my index.js script (listens for evenets), the data is loaded from the page and then stored for post processing. I would like to directly access the variable instead of having it hidden then accessing the DOM element.
window.addEventListener('load', function(){
if(window.location.href.includes('testplotdata')){
console.log('its a me mario')
//if we are in a test plot data page, lets plot
var jsonObject = JSON.parse(document.querySelector('.hide_data').innerHTML);
console.log(jsonObject['testData'])
//post processing code ....
}
})
Again, I want a way to grab my queried data without saving it as a DOM element then accessing it in my external event listener script.
Thanks!
Instead of storing data in HTML, add inline script to your template to store it in a global variable instead. So replace div.hide_data #{inData} with:
script.
var inData = !{inData}; // inData passed by backend must be a string
// representing a valid JS object (JSON will do)
Now you just access inData as a ready native object in your external script (which you need to make sure load after the above script, putting it at the end of <body> will do)
// No need: var jsonObject = JSON.parse(document.querySelector('.hide_data').innerHTML);
console.log(inData); // Go ahead with the data
Related
I would like to run some rules against pages. These rules are essentially functions that check the page for information. They can be simple as in 'check if the page has a title tag' or more complex like 'check if all links on the pages are whitelisted based on example.com/allowed_links.json'.
The rules would be run on the page on-demand only and come from a trusted source.
My first approach has been to create a rule service that sends back an javascript array of rules. All the client then has to do is go over the array and run each function in it. The response is a standard object {rule: [name], pass: [true|false], message: [some message about success/failure]}
Since this is on demand only, we fetch the rules from the service and run 'eval' on it.
EDIT: The response from 'mysite/rules' looks like this
RULESYSTEM.rules.push(function1() {...});
RULESYSTEM.rules.push(function2() {...});
...
const RULESYSTEM = {
rules: [],
};
let rules = fetch('mysite/rules')
let rulesscript = await rules.text();
eval(rulescript)
...
//eval will populate the previously declared rules array.
let pass = true;
for(let i=0; i < RULESYSTEM.rules.length; i++) {
let rule = RULESYSTEM.rules[i];
//This obj has only one property. Get that one.
let result = rule();
pass = pass && result.pass;
}
...
This works perfectly fine. However it is receiving a lot of pushback as 'eval' is considered evil and to be avoided at any cost. The security is not an issue here since the source is within the organization itself and thus trusted.
Are there any other ways to implement a system like this.
It would appear that all you're attempting to do is to retrieve JSON data and transforming it into a javascript object.
fetch('mysite/rules')
.then(res=>res.json())
.then(data=>{
//TODO: handle data which is your object/array
console.log(data)
})
Thus no need for eval. Also you need to remember that fetch returns a promise and that rules.text() and rules.json() also returns a promise. The way you've currently written it won't work anyway.
According to MDN
The json() method of the Body mixin takes a Response stream and reads it to completion. It returns a promise that resolves with the result of parsing the body text as JSON.
To answer your question:
Is it possible to return javascript code as JSON
That's clearly a no, however there are alternatives ! You can simply fetch javacsript as text (as you've done) and programmatically create a script tag, load your javascript text in it and append it to your HTML file.
Or even better, you can simply dynamically create a script tag with the URL of your server endpoint sending javascript and append it to your HTML file.
Something like:
const script = document.createElement("script");
script.onload = function(){
console.log("script loaded");
}
script.src = '/some/url/here';
document.body.appendChild(script);
I am going to add this as an answer. I will use some dummy data you can query based on an endpoint
Route("get-functions")
Response getJSFunctions(List<string> js_to_load){
var options = getData(); //returns a list
var guid = new Guid()
var fp = File.open(guid.toString() + ".js", "w+")
var out = "var fns = [" + options.join("\n") + "];";
fp.write(out);
fp.write(" var runner = options => fns.forEach(fn => fn(options) );");
fp.close()
return new Response({url: guid.toString() + ".js"})
}
Js:
$.json("get-functions", data => {
let script = document.createElement("script");
script.src = data.url;
document.head.appendChild(script)
runner(options);
});
So what is happening is that you build a Temp JS FIle containing all JS Functions you want run, then add that file dynamically to the runtime. Then you have a function which will always be available called runner, which you can immediately call.
Runner will iteratively call each function with a global list of options across the functions, which you can define on the clientside.
Instead of using C#, you can use any serverside or even Javascript if you are using node as your backend. You need DB Access and file creation access on the host machine. You will also want to delete the GUID files ocassionally as they are just one and done use files, so you can delete them all every 5 minutes or something.
I dont have the means right now to create a running sample. I can create something a little later with Python if you like as the backend.
I am working with two Express JS applications one is an API and second is application that is using this API by making requests and displaying received informations to user.
In API route I'm sending image as response:
router.get('/:customer_id',authController.isAuthenticated,(req,res) => {
.
. Retrieving customer data
.
return res.sendFile('/uploads/'+foundCustomer.doc_path);
});
And later another application is getting this document:
router.get('/:customer_id',(req,res) => {
var options = {
url: 'http://'+config.API.user+':'+config.API.password+'#'+config.API.host+':'+config.API.port+'/customers/'+req.params.customer_id
};
request(options,(err,response,body)=>{
return res.render('customer/show',{
document: ?, // Send document as parameter to view
});
});
});
In this point I want to render customer/show(EJS view engine) with customer document, but I don't want to save this document in my application files, because document is only needed to display in view (customer details and document are stored in another application).
I was trying to create temporary directory in my application structure, but it is difficult to manage deleting those not needed documents (Application has many users and at the same time many customers can be displayed).
Another solution that I was trying to implement is to make Ajax request on client side and latter append received document to <object data='document'>. But this request has to be authenticated with user and password, so I realised that storing credentials on client side javascript is not the best idea...
I am not sure that is it even possible to render and display image without saving in application files?
I would be grateful for any help, maybe the best workaround is to somehow manage temporarily saved documents.
Why not create a File object inside EJS template then use that for src attribute on an <img> ? You're already getting the raw buffer/blob from your image API server. Store it inside template.
From https://developer.mozilla.org/en-US/docs/Web/API/Blob/Blob
// place this code (store this variable) inside of your EJS template
// so it can be used by the client-side JS
var aBlob = new Blob( array[, options]); // Where array is the raw buffer data returned from your image API server
See https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL
var objectURL = URL.createObjectURL( aBlob ); // Where object is a Blob object
See https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/srcObject
const img = document.createElement('img');
img.src = objectURL;
Final solution (tested), using axios to make API request:
In my route I'm going to make HTTP request to my API to retrieve PDF file(document):
axios.get(`http://my-api/customer/id`).then(response => {
var photo = new Buffer(response.data, 'binary').toString('base64');
return res.render('customers/show',{
document: document
});
});
In my ejs view, I'm using HTML object tag to display received PDF:
<object data="data:application/pdf;base64,<%-document%>"></object>
I'm trying to pass initial state from server to client by stringifying that state and storing it in global variable. Below is simplified version of what I'm doing.
// home.tsx
res.render('home', { appState: {script: '<script></script>'} });
// home.pug
doctype html
html
head
script.
window.__initialState__ = !{JSON.stringify(appState)};
body
#root
Sadly the result looks like this
<script>window.__initialState__ = {"script":"<script></script>
which is not valid Javascript. My endgoal is to have server's appState object assigned to window.__initialState__.
The 1st part of your code is very confusing. However, as a general rule, you should always encode any HTML passed from the server to the client as string.
So you need to change the part of your server code that generated the line below:
let scriptObject = {script: '<script></script>'};
to this (assuming that you're using pug and node.js):
var htmlencode = require('htmlencode');
let scriptObject = {script: htmlencode.htmlEncode('<script></script>')};
My application needs to read in a large dataset and pipe it to the client to manipulate with D3.js. The problem is, on large datasets, the reading/loading of the file contents could take a while. I want to solve this using streams. However, I'm unsure of how to do so in the context of the Sails framework.
What I want to do is read the contents of the file and pipe it to a rendered page. However, I can't figure out how to pipe it through if I use something like res.view('somePage', { data: thePipedData });.
I currently have something like this:
var datastream = fs.createReadStream(path.resolve(DATASET_EXTRACT_PATH, datatype, dataset, dataset + '.csv'));
datastream.pipe(res);
...
return res.view('analytics', { title: 'Analytics', data: ??? });
What's the best way to approach this?
Based on your example it seems like the best course of action would be to set up a separate endpoint to serve just the data, and include it on the client via a regular <script> tag.
MyDataController.js
getData: function(req, res) {
/* Some code here to determine datatype and dataset based on params */
// Wrap the data in a Javascript string
res.write("var theData = '");
// Open a read stream for the file
var datastream = fs.createReadStream(
path.resolve(DATASET_EXTRACT_PATH, datatype, dataset, dataset + '.csv')
);
// Pipe the file to the response. Set {end: false} so that res isn't closed
// when the file stream ends, allowing us to continue writing to it.
datastream.pipe(res, {end: false});
// When the file is done streaming, finish the Javascript string
datastream.on('end', function() {
res.end("';");
});
}
MyView.ejs
<script language="javascript" src="/mydata/getdata?datatype=<%=datatype%>&etc.."></script>
MyViewController.js
res.view('analytics', {datatype: 'someDataType', etc...});
A slight variation on this strategy would be to use a JSONP-style approach; rather than wrapping the data in a variable in the data controller action, you would wrap it in a function. You could then call the endpoint via AJAX to get the data. Either way you'd have the benefit of a quick page load since the large data set is loaded separately, but with the JSONP variation you'd also be able to easily show a loading indicator while waiting for the data.
I am trying to visualize some Data with d3 which is stored inside of a MongoDB. My question is about something like a best practice to create div elements for every data through the JADE template and afterwards call a method to draw different charts.
My main problem is that I am losing the reference to my data after displaying the HTML file and I do not want to query the DB a second time.
Schema
# Create Schema
executionSchema = new Schema(
timestamp: Number,
components: [{
uid: String,
type: { type: String },
samples: [Number],
execution_times: [Number]
}]
)
The data is initially retrieved and given to the JADE template:
Index coffee
exports.index = (req, res) ->
Execution.find (err, executions, count) ->
res.render "index", title: "Debugger", executions: executions
return
return
Afterwads, the index.JADE creates divs for every component inside of execution[0]
- each component in executions[0].components
div(class="panel panel-primary")
div(class="panel-heading") UID: #{component.uid}
div(class="panel-body")
p(style='white-space:pre;')
| Type: #{component.type}
- var uid = component.uid
div(id=uid)
This is everything right now, since I am not able to call a JavaScript method outside of the JADE file. Any ideas?
Thank you.
If I understand correctly, you want this data to be available on the clientside script without actually querying the db the second time?
You have two options here. The first one is to just add this line right bellow your current Jade code:
- each component in executions[0].components
// div creation stuff here
// ...
script
window.executions = JSON.stringify(executions);
Now your client-side scripts will be able to access the executions object and access that data like this:
var data = JSON.parse(executions);
Not sure if this is efficient though. If you do not want to query the db for the second time, it could be because the dataset is large or db slow?
Well the other way to do this with only a single db query is to render the page without the divs, and do not query the db at all. Then use the Javascript to fetch the executions (ajax call) and render it client-side once the data is loaded.
That depends on your use case though.