Creating page content from a large JSON - javascript

I have a very large JSON object, which serves as a data source from which my page is created. Currently I am able to load the whole object, create DIVs from it and the display it. However when the JSON is very big it takes a lot of time and then suddenly the whole content is displayed. I would like to load each child separately, or display the loaded content after each node (appending).
The JSON has the following structure:
{
id:"0",
text:"",
children:[{
id:"0-0",
text:"",
children:[...]
},
{
id:"",
text:"0-1",
children:[...]
}]
}
When I load this object I call a function which creates the container based on the root node and then a recursive function which gets executed for every child.
function loadRoot(wholeJson){
var rootDiv = '<div id="frag-' + wholeJson.id + '"></div>';
return rootDiv;
}
function loadChildren(wholeJson){
rootDiv = $('#frag-' + wholeJson.id);
wholeJson.children.forEach(function(child){
loadChild(child);
});
}
function loadChild(node){
var newDiv = // create the div here..
node.children.forEach(function(child){
loadChild(child);
});
newDiv += "</div>" //close it
return newDiv;
}
This works, but as I mentioned it loads the whole content at once. How can I achieve the displaying of every child after it's created and not only at the end of script execution, please?

I had a similar problem. The reason why this happens is that javascript first processes your function in the background, and when it is all done, it's changes will be made. I do not know if its the best way to go. But setTimeout with a anonymous function worked for me. Just set the waiting time really low.

Related

Trigger javascript after DOM is fully updated by another script

I have a script that does 2 main things in a sequence:
It pulls data from an XML file through a GET request, and it appends the result into the DOM.
I have this "MacyJS" script which basically puts the content within a nicely calculated grid (a bit like Masonry).
My issue is that the first part is taking quite some time (I don't mind, it's meant for my own private use) to get the result and append it to the DOM. So that the "MacyJS" already ran before, and thus not rendering properly the HTML in a nice grid.
You can see I tried to put a delay for MacyJS, but of course this is not precise and doesn't always work in time, since the GET/append sometimes takes more than 1.5s.
How can I automatically run "MacyJS" only once the DOM has fully been updated with the new HTML/content ?
Thanks a lot for your help.
Code below for reference:
$(document).ready(function(){
// THE FIRST SCRIPT
var x=20;
$.get("my xml file here", function(data) {
var $xml = $(data);
$xml.find("item").each(function(i, val) {
var $this = $(this),
item = {
description: $this.find("description").text(),
guid: $this.find("guid").text()
};
$('#grid').append($('<div class="parsed appear"><a target="_blank" href="' +item.guid +'">' +item.description +'</a></div>'));
return i<(x-1);
});
});
// THE SECOND SCRIPT
setTimeout(function(){
var macy = Macy({
container: '#grid',
margin: 20,
columns: 4,
});
}, 1500);
});

C3 charts not rendering from jQuery data

So I'm just getting into some simple UI stuff, and I'm stuggling with Javascript. I managed to get a couple of for-loops to create tags and render some charts, works great.
But I then used jquery to fetch a block of JSON data from a Spring Boot REST service.
$.getJSON('api/randomData?chartCount=' + chartsToMake).done( function(data) {
// Extract list of servers from the data...
var hostList = []
$.map(data, function(row) { hostList.push(row.host); });
// Make it a unique list
var uniqueHostList = hostList.unique();
// Iterate over the unique list of servers
uniqueHostList.forEach( function(host) {
var tag = "chart" + host.replace(/\./g, "");
var hostdata = data.filter( (v,i,a) => v.host === host)
console.log("Processing Server:" + host + " with " + hostdata.length + " rows");
// Create DOM element to bind the chart to
document.getElementById("chartBlocks").innerHTML += "<div id=\"" + tag.replaceAll('#','') + "\"></div>";
// Create the Chart here
var chart = c3.generate({
bindto: "#" + tag,
data: {
json: hostdata,
keys: { value: ['lowerband'] }
}
});
});
});
This gets an array of JSON strings back, each object contains one metric for a server, all the HTML tags are inserted, but only the last chart draws up.
I added lots of console.out() stuff to try and debug this, it has the data and everything seems to be working, and the last chart looks fine, but the other 3 above it dont populate.
I've been pulling my hair out trying to work out why, please help!
PS. I created a github project here which is a simple maven/spring boot application.
https://github.com/tfindlay-au/c3demo
There is a working page called "working.html" and "index.html" which doesnt work.
FWIW - it feels like a variable scope thing or maybe a timing thing if I'm trying to generate the chart before the data is avilable or something. Not sure if that helps.
"only the last chart draws up."
document.getElementById("chartBlocks").innerHTML += "<div id=\"" + tag.replaceAll('#','') + "\"></div>";
Because that line (at first I thought it completely replaced the content but then I saw the += ) has bad side-effects for the existing content of chartblocks. Specifically it wipes out the event functionality (edit: and data) which setting up a c3.chart has attached to elements in that chart. When you then set innerHTML in chartblocks again, all that stuff is replaced by the innerHTML string, which is just a literal copy of the structure of the dom elements - wiping out any previously attached event handlers or data properties.
You instead need to append an extra div to chartblocks, which leaves the existing sibling charts in peace, and since c3 uses the d3 library you can do it like this:
d3.select("#chartBlocks").append("div").attr("id", tag.replace('#',''));

Inject/execute JS code to IPython notebook and forbid its further execution on page reload

I'm writing the library which has to embed javascript code to IPython notebook and execute it. The HTML/JS code looks like:
<div id="unique_id"></div>
<script>
var div = document.getElementById("unique_id");
// Do the job and get "output"
div.textContent = output; // display output after the cell
</script>
And the python code:
from IPython import display
display.display(display.HTML(code))
The side-effect is that the javascript code is stored in the output of the cell in notebook, and every time when the page is reloaded or the notebook is opened it will run again.
Are there any way of forbidding the code to be executed on reload? Or is it possible to run the javascript code without saving it within the output?
I've figured out the hack.
The trick is to use update=True argument of the IPython.display.display() which will replace the output with a new one (see here for an example).
So what is needed to be done: first output javascript that does the job, and then waits until the div with a certain ID is created, to fill it with the output. Once this display() is called, we could call display a second time updating the first one with the actual HTML with the div. So the javascript code once finished will fill it with the results, but the code itself will not be saved.
Here's the test code:
First, define the callback function (it looks like, it is important here to display it as HTML("<script> ... </script>") rather than Javascript(...)):
from IPython.display import display, HTML, Javascript
js_getResults = """<script>
function getResults(data, div_id) {
var checkExist = setInterval(function() {
if ($('#' + div_id).length) {
document.getElementById(div_id).textContent = data;
clearInterval(checkExist);
}
}, 100);
};
</script>"""
display(HTML(js_getResults))
And then execute the update trick in one cell:
js_request = '$.get("http://slow.server/", function(data){getResults(data, "unique_id");});'
html_div = '<div id="unique_id">Waiting for response...</div>'
display(Javascript(js_request), display_id='unique_disp_id')
display(HTML(html_div), display_id='unique_disp_id', update=True)
After the callback of get() is executed, the content Waiting for response... will be replaced with the output from the server.
After running into the same issue of Javascript executing on every notebook open, I adapted #Vladimir's solution to a more general form:
Use fresh unique IDs on every render (since old ID is saved with the HTML output of the notebook).
No polling to determine when HTML element is rendered.
Of course, when the notebook is closed, no HTML modifications done by JS are saved.
Key Insight: Replace Cell Output
from IPython.display import clear_output, display, HTML, Javascript
# JavaScript code here will execute once and will not be saved into the notebook.
display(Javascript('...'))
# `clear_output` replaces the need for `display_id` + `update`
clear_output()
# JavaScript code here *will* be saved into the notebook and executed on every open.
display(HTML('...'))
Making it Work
The challenge here is that the HTML and Javascript blocks can be rendered out of order, and the code which manipulates the HTML element needs to only execute once.
import random
from IPython.display import display, Javascript, HTML, clear_output
unique_id = str(random.randint(100000, 999999))
display(Javascript(
'''
var id = '%(unique_id)s';
// Make a new global function with a unique name, to prevent collisions with past
// executions of this cell (since JS state is reused).
window['render_' + id] = function() {
// Put data fetching function here.
$('#' + id).text('Hello at ' + new Date());
}
// See if the `HTML` block executed first, and if so trigger the render.
if ($('#' + id).length) {
window['render_' + id]();
}
''' % dict(unique_id=unique_id)
# Use % instead of .format since the latter requires {{ and }} escaping.
))
clear_output()
display(HTML(
'''
<div id="%(unique_id)s"></div>
<!-- When this script block executes, the <div> is ready for data. -->
<script type="text/javascript">
var id = '%(unique_id)s';
// See if the `Javascript` block executed first, and if so trigger the render.
if (window['render_' + id]) {
window['render_' + id]();
}
</script>
''' % {'unique_id': unique_id}
))
To keep the notebook clean, I would put this plumbing code into a separate .py file and import it from Jupyter.

JS vs DOM timing: .remove() element visually happens, but travesal still includes it

The short description of the functionality that we are trying to achieve: we have a list of source objects on the left, a person can drag new items from the list to a list on the right, items thus get added to the list on the right; they can also remove items from the list on the right. The list on the right then gets saved whenever it is changed. (I don't think the specifics of how/where it is being saved matter...)
I am having a problem with a bit of timing in the JavaScript vs. DOM elements realm of things. Items that are already on the list on the right can be removed. We have some code that fires on a 'remove/delete' type icon/button on a DOM element, that is supposed to remove the element from the DOM visually and permanently (i.e. it doesn't need to be brought back with a 'show'). This visual change should then also show up in the JSON object that is built when the JS traverses the DOM tree to build the new updated list.
However, this chunk of JS code that runs immediately after this .remove() is called, the element that should have just been removed still shows up in the JSON object. This is not good.
Here are what I believe to be the relevant bits of code operating here. This lives in a web browser; much of this is in the document.ready() function. A given list can also have subsections, hence the sub-list parts and loops.
The on-click definition:
$('body').on('click', '.removeLine', function() {
var parent=$(this).parent().parent().parent(); //The button is a few DIVs shy of the outer container
var List=$(this).closest('article'); //Another parent object, containing all the
parent.fadeOut( 300,
function() {
parent.slideUp(300);
parent.remove();
}
);
sendList(List); // This builds and stores the list based on the DOM elements
});
And then later on, this function definition:
function sendList(List) {
var ListArray=[],
subListArray=[],
itemsArray = [],
subListName = "";
var ListTitle = encodeText(List.find('.title').html());
// loop through the subLists
List.find('.subList').each(
function(index, element) {
subListName=($(this).find('header > .title').html()); // Get sublist Title
subListID=($(this).attr('id')); // Get subList ID
// loop through the line items
itemsArray=[];
$(this).find('.itemSearchResult').each(
function(index, element) {
// Build item Array
if( $(this).attr('data-itemid')!= item ) {
itemArray.push( $(this).attr('data-itemid'));
}
}
);
// Build SubList Array with items Array
subListArray.push(
{
"subListName": subListName,
"subListID" : subListID,
"items" : itemsArray
}
);
}
); <!-- end SubList Loop -->
// Complete List Array with subListArray
ListArray ={
"ListName": ListTitle,
"ListID": List.attr('id'),
"subLists": subListArray
};
// Send New List to DataLists Object - the local version of storage
updateDataLists(ListArray);
// Update remote storage
window.location= URLstring + "&$Type=List" + "&$JSON=" + JSON.stringify(ListArray) + "&$objectID=" + ListArray.ListID;
};
It seems to be the interaction of the 'parent.remove()' step and then the call to 'sendList()' that get their wires crossed. Visually, the object on screen looks right, but if we check the data being sent to the storage, it comes through WITH the object that was visually removed.
Thanks,
J
PS. As you can probably tell, we are new at the Javascript thing, so our code may not be terribly efficient or proper. But...it works! (Well, except for this issue. And we have run into this issue a few times. We have a workaround for it, but I would rather understand what is going on here. Learn the deeper workings of JS so we don't create these problems in the first place.)
There's a few things going on here, but I'm going to explain it by approaching it from an asynchronous programming perspective.
You are calling sendList before the element gets removed from the DOM. Your element doesn't get removed from the DOM until after your fadeOut callback gets executed (which takes 300ms).
Your sendList function gets called immediately after you begin the fadeOut, but your program doesn't wait to call sendList until your fadeOut is finished - that's what the callback is for.
So I would approach it by calling sendList in the callback, after your DOM element has been removed like this:
$('body').on('click', '.removeLine', function() {
var el = $(this); //maintain a reference to $(this) to use in the callback
var parent=$(this).parent().parent().parent(); //The button is a few DIVs shy of the outer container
parent.fadeOut( 300,
function() {
parent.slideUp(300);
parent.remove();
sendList(el.closest('article'));
}
);
});

replacing or managing json data from ajax request

I've got a page that makes an ajax request and gets data back in json format.
I needed to sort this data before adding it to the DOM, so it is put into an object with the following code
function(data) {
var birthDates = {};
var uids = {};
$.each(data.users, function() {
if (!uids[this.uid]) {
uids[this.uid] = [];
uids[this.uid].push(this);
}
if (!birthDates[this.birthDate])
birthDates[this.birthDate] = [];
birthDates[this.birthDate].push(this);
});
for (var d in birthDates) {
var date = d;
$('div#holdDates').append('<ul class="dayList" id="' + date + '"><li class="date" >' + date + '</li></ul>');
$.each(birthDates[date], function() {
$('ul#' + date).append('<li class="show" id="' + this.uid + '">' + this.name + '</li>');
});
}
$('li.show').click(function() {
var getuid = $(this).attr('id');
$showArr = uids[getuid];
// now I can get the extended data about the user
this all works great when the page is loaded for the first time, however I'm running into two problems, both as a result of making a second ajax request
1) if i make the same ajax request (giving the same variables, so the same data comes back again), then the data gets added to the newly created objects (uids, and birthDates) twice, and I can't figure out how to keep that as unique
2) sometimes (and i haven't been able to debug to figure out why) i don't get any of the extended user data from uids object. (the stuff I do after the li click
Any ideas? Am i doing this efficiently?
I find it strange that you can't empty an object that you've created, but apparently everything I'm reading says that you can't.
Addition
well, after posting this, the next thing I was doing was building a dynamic tag cloud which is dependent on the returned json.
so, now I run into the same problem again. I need to tag-cloud to be new after each request. I really hope there is a way to get rid of 'legacy' data in javascript.
Thanks,
Pete
Everything declared with the var keyword inside of your function(data) will be created anew each time the function is called.
What's the symptom of your problem? Have you actually looked at the value of the uids variable in firebug and seen that items are being duplicated, or do you just see your dates/names getting doubled up on the page?
My suspicion is that this is a result of not clearing the DOM elements that you are calling .append() on before you display your results.
Try adding to the beginning of the function:
$('div#holdDates').empty();
as well as to your date display loop:
$('ul#'+date).empty();
and for good measure, at the beginning of function:
$('li.show').unbind('click');

Categories