I am trying to visualize some Data with d3 which is stored inside of a MongoDB. My question is about something like a best practice to create div elements for every data through the JADE template and afterwards call a method to draw different charts.
My main problem is that I am losing the reference to my data after displaying the HTML file and I do not want to query the DB a second time.
Schema
# Create Schema
executionSchema = new Schema(
timestamp: Number,
components: [{
uid: String,
type: { type: String },
samples: [Number],
execution_times: [Number]
}]
)
The data is initially retrieved and given to the JADE template:
Index coffee
exports.index = (req, res) ->
Execution.find (err, executions, count) ->
res.render "index", title: "Debugger", executions: executions
return
return
Afterwads, the index.JADE creates divs for every component inside of execution[0]
- each component in executions[0].components
div(class="panel panel-primary")
div(class="panel-heading") UID: #{component.uid}
div(class="panel-body")
p(style='white-space:pre;')
| Type: #{component.type}
- var uid = component.uid
div(id=uid)
This is everything right now, since I am not able to call a JavaScript method outside of the JADE file. Any ideas?
Thank you.
If I understand correctly, you want this data to be available on the clientside script without actually querying the db the second time?
You have two options here. The first one is to just add this line right bellow your current Jade code:
- each component in executions[0].components
// div creation stuff here
// ...
script
window.executions = JSON.stringify(executions);
Now your client-side scripts will be able to access the executions object and access that data like this:
var data = JSON.parse(executions);
Not sure if this is efficient though. If you do not want to query the db for the second time, it could be because the dataset is large or db slow?
Well the other way to do this with only a single db query is to render the page without the divs, and do not query the db at all. Then use the Javascript to fetch the executions (ajax call) and render it client-side once the data is loaded.
That depends on your use case though.
Related
I've got a variable, say data containing data in the form of an Array with each item having a unique ID.
app.get('/products/:id', function (req, res) {
res.send(data.map(data => "" + data.id + "")) //basically gets the data of the element in the Array whos Id has been given to the server.
})
I have sent the data from the server to the front-end on a GET request. But how do I create a seperate webpage for each element dynamically in the data array? where do i have to write the html and the css? I want a way with which I can create a page for each element like domain.com/products/id which displays information about the data entry which matches the Id . Do need to use pug? hbs?ejs? I' so confused.
So I found I had to use Javascript Templates to send data to a view. I used ejs, which went pretty good!
Here's how it went:
1. fetch my data form my DB, which in this case is MongoDB using db.findOne().
2. We get an array, let's say data. send the variable to my view using the same res.render syntax, just in a cooler way.
app.get('/blogs/:id',(req,res)=>{
const data = //find function
res.render('page.ejs', {body:data});
})
:id creates a page for every element in the DB.
and now the view that is, the public/page.ejs file has a global body variable, which
we can now use to show our blogs.
3. the front end markup in pages.ejs:
<div class="blogs">
<%=body.forEach (item)=>{%>
<p><%=item.blog%></p><br>
<%=}%>
</div>
We call a forEach function on the array, and create a paragraph element for each item in the array, that is for each blog.
Please not that <%, <%= and %> are EJS' tags. Read about them more in the official docs.
Thanks Mohammad for letting me know about this. (From comments)
I am looking for a better way of passing data to my index.js file in a webdev application. Note I really only have about a month of webdev experience so this is likely due to lack of experience. here is my software flow:
I query data in a route handler before the page is rendered. I then pass this data to the rendered page (note i need to keep some of the key-vals hidden. However aggregation works).
exports.getPlotView = async(req, res, next) =>{
//grab the module to query from, stored as var.testModel
const qParse = new PlotQueryParse(req.query).parseObj();
// console.log(qParse)
// const testblockName = qParse.testblock+"Name" ;
// const limitName = qParse.limitname;
const aggregationObj = {$match:
{'jobId':qParse.jobId, '<key2>':<val2>, "<key>":"<val>"}
}
const data = await qParse.testModel.aggregate([aggregationObj]);
console.log(data[0])
const dataString = JSON.stringify(data[0]);
//parse the url to make the query
res.status(200).render('testPlotView', {
pageHeader: "Test",
subHead: "Test summary",
IPn: "IPn",
inData:dataString
});
}
data is passed to pug template. The template uses this as a hidden element
extends base
block content
div.hide_data #{inData}
div#dataviz
now in my index.js script (listens for evenets), the data is loaded from the page and then stored for post processing. I would like to directly access the variable instead of having it hidden then accessing the DOM element.
window.addEventListener('load', function(){
if(window.location.href.includes('testplotdata')){
console.log('its a me mario')
//if we are in a test plot data page, lets plot
var jsonObject = JSON.parse(document.querySelector('.hide_data').innerHTML);
console.log(jsonObject['testData'])
//post processing code ....
}
})
Again, I want a way to grab my queried data without saving it as a DOM element then accessing it in my external event listener script.
Thanks!
Instead of storing data in HTML, add inline script to your template to store it in a global variable instead. So replace div.hide_data #{inData} with:
script.
var inData = !{inData}; // inData passed by backend must be a string
// representing a valid JS object (JSON will do)
Now you just access inData as a ready native object in your external script (which you need to make sure load after the above script, putting it at the end of <body> will do)
// No need: var jsonObject = JSON.parse(document.querySelector('.hide_data').innerHTML);
console.log(inData); // Go ahead with the data
my website relies on a database which is a big JSON file like this:
var myjsonData =
[ {
"ID": 0,
"name": "Henry",
"surname": "McLarry",
"...": "...",
}]
I do generate this data every month at high cost to me, therefore I would like to avoid calling it straight in my html <head>, because this will allow any user to download the full database in no time.
I would like to build a "something" that can only call specific items from the json file (just the only one I want to show) without "exposing" the full .json onto client side.
today I use the call
var myvar= myjsonData.ID.Name
to get "Henry" into myvar, I would like to build something like
var myvar = mycallfunction(ID,Name)
I did try with PHP as intermediary but the ajax calls from javacript doesn't allow me to fetch the data.
Can I use JQuery with the JSON Url to get only the item I need?
What you can do is parse your json for an object. So you can get any value you want from json.
Example:
var myjsonData = '{"ID": 0,"name": "Henry","surname": "McLarry"}';
obj = JSON.parse(myjsonData);
console.log(myjsonData.ID); //print the id
console.log(myjsonData.name); //print the name
console.log(myjsonData.surname); //print the surname
So you have a NoSQL Database which has only one kind of Document that is the full JSON element you use in your website. In that scenario you have three options:
Depending on the NoSQL Database you're using you can limit the fields which will be returned(I.e: For MongoDB you can look here: https://docs.mongodb.com/manual/tutorial/project-fields-from-query-results/)
Change the way you store you data into more modular documents and make the logic to connect them in you application. So instead of one big document you'll have modular ones as Users, Products, Transactions and etc and you can use your application to query them individually.
Build a Server Side logic as an API to deal with your data and provide only what you need, so the API(Which can be node.js, php, or any you may like) will get the full JSON it`s endpoints will only the data you want. For example: myapi.com/getUser, myapi.com/getProducts and so on.
If you're able to provide more info on the technologies you're using that would help us. Hope that helped :).
I am working with two Express JS applications one is an API and second is application that is using this API by making requests and displaying received informations to user.
In API route I'm sending image as response:
router.get('/:customer_id',authController.isAuthenticated,(req,res) => {
.
. Retrieving customer data
.
return res.sendFile('/uploads/'+foundCustomer.doc_path);
});
And later another application is getting this document:
router.get('/:customer_id',(req,res) => {
var options = {
url: 'http://'+config.API.user+':'+config.API.password+'#'+config.API.host+':'+config.API.port+'/customers/'+req.params.customer_id
};
request(options,(err,response,body)=>{
return res.render('customer/show',{
document: ?, // Send document as parameter to view
});
});
});
In this point I want to render customer/show(EJS view engine) with customer document, but I don't want to save this document in my application files, because document is only needed to display in view (customer details and document are stored in another application).
I was trying to create temporary directory in my application structure, but it is difficult to manage deleting those not needed documents (Application has many users and at the same time many customers can be displayed).
Another solution that I was trying to implement is to make Ajax request on client side and latter append received document to <object data='document'>. But this request has to be authenticated with user and password, so I realised that storing credentials on client side javascript is not the best idea...
I am not sure that is it even possible to render and display image without saving in application files?
I would be grateful for any help, maybe the best workaround is to somehow manage temporarily saved documents.
Why not create a File object inside EJS template then use that for src attribute on an <img> ? You're already getting the raw buffer/blob from your image API server. Store it inside template.
From https://developer.mozilla.org/en-US/docs/Web/API/Blob/Blob
// place this code (store this variable) inside of your EJS template
// so it can be used by the client-side JS
var aBlob = new Blob( array[, options]); // Where array is the raw buffer data returned from your image API server
See https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL
var objectURL = URL.createObjectURL( aBlob ); // Where object is a Blob object
See https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/srcObject
const img = document.createElement('img');
img.src = objectURL;
Final solution (tested), using axios to make API request:
In my route I'm going to make HTTP request to my API to retrieve PDF file(document):
axios.get(`http://my-api/customer/id`).then(response => {
var photo = new Buffer(response.data, 'binary').toString('base64');
return res.render('customers/show',{
document: document
});
});
In my ejs view, I'm using HTML object tag to display received PDF:
<object data="data:application/pdf;base64,<%-document%>"></object>
My application needs to read in a large dataset and pipe it to the client to manipulate with D3.js. The problem is, on large datasets, the reading/loading of the file contents could take a while. I want to solve this using streams. However, I'm unsure of how to do so in the context of the Sails framework.
What I want to do is read the contents of the file and pipe it to a rendered page. However, I can't figure out how to pipe it through if I use something like res.view('somePage', { data: thePipedData });.
I currently have something like this:
var datastream = fs.createReadStream(path.resolve(DATASET_EXTRACT_PATH, datatype, dataset, dataset + '.csv'));
datastream.pipe(res);
...
return res.view('analytics', { title: 'Analytics', data: ??? });
What's the best way to approach this?
Based on your example it seems like the best course of action would be to set up a separate endpoint to serve just the data, and include it on the client via a regular <script> tag.
MyDataController.js
getData: function(req, res) {
/* Some code here to determine datatype and dataset based on params */
// Wrap the data in a Javascript string
res.write("var theData = '");
// Open a read stream for the file
var datastream = fs.createReadStream(
path.resolve(DATASET_EXTRACT_PATH, datatype, dataset, dataset + '.csv')
);
// Pipe the file to the response. Set {end: false} so that res isn't closed
// when the file stream ends, allowing us to continue writing to it.
datastream.pipe(res, {end: false});
// When the file is done streaming, finish the Javascript string
datastream.on('end', function() {
res.end("';");
});
}
MyView.ejs
<script language="javascript" src="/mydata/getdata?datatype=<%=datatype%>&etc.."></script>
MyViewController.js
res.view('analytics', {datatype: 'someDataType', etc...});
A slight variation on this strategy would be to use a JSONP-style approach; rather than wrapping the data in a variable in the data controller action, you would wrap it in a function. You could then call the endpoint via AJAX to get the data. Either way you'd have the benefit of a quick page load since the large data set is loaded separately, but with the JSONP variation you'd also be able to easily show a loading indicator while waiting for the data.