I am trying to use arborjs to generate a graph of a computer network where I can setup the length of the edges between each nodes to represent the ping response time. The API itself supports doing this, but I need to build up the whole stack of drawing/update routines from scratch if I use it directly.
In this demo page http://arborjs.org/halfviz/#/a-new-hope on the other hand, we can simply provide a simple text-representation of the graph (from an XHR json file for ex) and the library handles all the work, what it doesn't, is the support of edge lengths between nodes.
I am wondering if anyone had played with this, and how to tweak the parsing part of the library to take in consideration the length of the edges?
Thank you.
Related
Normally, I am able to find the answer I am looking for, however, I have come across an issue that I am not finding resolution for yet..
Given a MessageEvent whoms body contains a 1-... second video file,
webm, as a binaryString. I can parse this as a dataURL and update the
src, however, I would like to instead build a growing buffer that can
be streamed to the srcObj, as if it were the mediaDevice ?
I am working on a scalable API for broadcasting video data that has as few dependencies as possible.
String trimming is possible as well, maybe just trim the binary string using a regex that removes all header data and continuously append to srcObj. The stream may be in excess of 1 GB total chunks, meaning src="..." may not be scalable friendly in terms of growing the string over time, additional solutions may include toggling different video sources to achieve a smoother transition. I can manipulate the binary string in php on the server or use a python, cpp, ruby, node, service as long as it routes the output to the correct socket.
I am not utilizing webRTC.
Thanks, the Stack Overflow community is awesome, I do not get to say that often enough.
I'm running NodeJS on the server-side, and I'm trying to do a bit of automated image processing to determine the 'base colour' of an image.
Here are the steps of what I want to do:
Take an image (on a remote server, so passed through with a URL) and get its dimensions
Use dimensions to calculate the centre of the image
Take a 10px x 50px (WxL) rectangle around the centre-point
Get the RGB value for each of these pixels (500 per image)
Output the average value of the pixels
I know such things are possible in PHP, but I'd like to use Node is possible. I've seen tutorials on using Node-imagick for basic processing (like resizing and cropping) but don't know where to start with more advanced analysis like this.
Questions
(a) Is this possible with Node?
(b) What libraries will allow me to do this?
A: yes
B: gm
here are some more characters to make this long enough for stackoverflow...
node-itk may be helpful to you.
Node-ITK is a node.js wrapper, which is built on top of ITK. It was built to facilitate noe.js' use in rapid prototyping, education, and web services for Medical Image Processing.
https://npmjs.org/package/node-itk
I have raw data in text file format with lot of repetitive tokens (~25%). I would like to know if there's any algorithm which will help:
(A) store data in compact form
(B) yet, allow at run time to re-constitute the original file.
Any ideas?
More details:
the raw data is consumed in a pure html+javascript app, for instant search using regex.
data is made of tokens containing (case sensitive) alpha characters, plus few punctuation symbols.
tokens are separated by spaces, new lines.
Most promising Algorithm so far: Succinct data structures discussed below, but reconstituting looks difficult.
http://stevehanov.ca/blog/index.php?id=120
http://ejohn.org/blog/dictionary-lookups-in-javascript/
http://ejohn.org/blog/revised-javascript-dictionary-search/
PS: server side gzip is being employed right now, but its only a transport layer optimization, and doesn't help maximize use of offline storage for example. Given the massive 25% repetitiveness, it should be possible to store in a more compact way, isn't it?
Given that the actual use is pretty unclear I have no idea whether this is helpful or not, but for smallest total size (html+javascript+data) some people came up with the idea of storing text data in a greyscale .png file, one byte to each pixel. A small loader script can then draw the .png to a canvas, read it pixel for pixel and reassemble the original data this way. This gives you deflate compression without having to implement it in Javascript. See e.g. here for more detailled information.
Please, do not use a technique like that unless you have pretty esotheric requirements, e.g. for a size-constrained programming competition. Your coworkers will thank you :-)
Generally speaking, it's a bad idea to try to implement compression in JavaScript. Compression is the exact type of work that JS is the worst at: CPU-intensive calculations.
Remember that JS is single-threaded1, so for the entire time spent decompressing data, you block the browser UI. In contrast, HTTP gzipped content is decompressed by the browser asynchronously.
Given that you have to reconstruct the entire dataset (so as to test every record against a regex), I doubt the Succinct Trie will work for you. To be honest, I doubt you'll get much better compression than the native gzipping.
1 - Web Workers notwithstanding.
I have a server that generates pngs very rapidly and I need to make this into a poor-man's video feed. Actually creating a video feed is not an option.
What I have working right now is a recursive loop that looks a little like this (in pseudo-code):
function update() {
image.src = imagepath + '?' + timestamp; // ensures the image will update
image.onload = function () {update()};
}
This works, however after a while, it crashes the browser (Google Chrome, after more than 10 minutes or so). These images are being updated very frequently (several times a second). It seems the images are caching, which causes the browser to run out of memory.
Which of these solutions would solve the problem while maintaining fast refresh:
HTML5 canvas with drawImage
HTML5 canvas with CanvasPixelArray (raw pixel manipulation)
I have access to the raw binary as a Uint8Array, and the image isn't too large (less than 50 kb or so, 720 x 480 pixels).
Alternatively, is there anyway to clear old images from the cache or to avoid caching altogether?
EDIT:
Note, this is not a tool for regular users. It's a tool for diagnosing analog hardware problems for engineers. The reason for the browser is platform independence (should work on Linux, Windows, Mac, iPad, etc without any software changes).
The crashing is due to http://code.google.com/p/chromium/issues/detail?id=36142. Try creating object URLs (use XHR2 responseType = "arraybuffer" along with BlobBuilder) and revoking (using URL.revokeObjectURL) the previous frame after the next frame is loaded.
Edit: You really should be processing these into a live low-fps video stream on the server side, which will end up giving you greatly decreased latency and faster load times.
#Eli Grey seems to have identified the source of your crashing. It looks like they have a fix in the works, so if you don't want to modify your approach hopefully that will be resolved soon.
With regard to your other question, you should definitely stick with drawImage() if you can. If I understand your intention of using the CanvasPixelArray, you are considering iterating over each pixel in the canvas and updating it with your new pixel information? If so, this will be nowhere near as efficient as drawImage(). Furthermore, this approach is completely unnecessary for you because you (presumably) do not need to reference the data in the previous frame.
Whether fortunately or not, you cannot directly swap out the internal CanvasPixelArray object stored within an HTML5 canvas. If you have a properly-formatted array of pixel data, the only way you can update a canvas element is by calling either drawImage() or putImageData(). Right now, putImageData() is much slower than drawImage(), as you can see here: http://jsperf.com/canvas-drawimage-vs-putimagedata. If you have any sort of transparency in the frames of your video, you will likely want to clear the canvas and then use drawImage() (otherwise you could see through to previous frames).
Having said all that, I don't know that you really even need to use a canvas for this. Was your motivation for looking into using a canvas so that you could avoid caching (which now doesn't seem to be the underlying issue for you)?
If the "movie" is data-driven (ie. based on numbers and calculations), you may be able to push MUCH more data to the browser as text and then have javascript render it client-side into a movie. The "player" in the client can then request the data in batches as it needs it.
If not, one thing you could do is simply limit the frames-per-second (fps) of the script, possibly a hard-coded value, or a slider / setable value. Assuming this doesn't limit the utility of the tool, at the very least it would let the browser run longer w/o crashing.
Lastly, there are lots of things that can be done with headers (eg. in the .htaccess file) to indicate to browsers to cache or not cache content.
iPad, you say ?.. Nevertheless, i would advice using Flash/video or HTML5/video.
Because WebKit is very easily crashed with even moderate influx of images, either just big images or just a huge number of small ones..
From the other side, XHR with base64 image data or pixel array MIGHT work. I have had short polling app, which was able to run for 10-12 hours with XHR polling server every 10 seconds.
Also, consider delta compression, - like, if its histogram with abscissa being time scale - you can only send a little slice from the rigth, - of course, for things like heat-maps, you cannot do that.
These images are being updated very frequently (several times a
second).
.. if its critical to update at such a high rate - you MUST use long polling.
Is it possible (and supported cross-browser) to embed an image into the XML of an AJAX response, and then load that image using JavaScript?
I have a system that does some calculations based on the number of sessions running through it. The results are then graphed, and returned in two parts:
1) XML containing information about the graph, totals, and Image map data allowing the user to click on relevant areas.
2) The graph image.
As the data can change between the two requests (and could be expensive to calculate), I'd prefer to do it in a single request (return the image with the XML). The current implementation caches the stats for a small period so that the results of multiple requests will still match. As the amount of data that needs to be cached is going to be increasing (from ~2.5K to ~1.2MB), I would like to try an alternative method.
NOTE: I do not want to use inline b64 PNG images as they are not supported in IE.
Can you not store the image on the server and send the URL to the client?
As this seems like more work that it's worth, I've decided that a simpler solution would be:
1) Send XML data to the client with the details of what is to be graphed.
2) Client sends a request for the image, including the data to graph (similar to the Google Chart API).
This decouples the chart rendering from the data, and then it can be used in the future to generate generic charts for other data sets. The other benefit is that it doesn't require any caching server-side since only 1 request is used.
You might want to check this link to see if this concept will work. This link maybe useful.
I think trying to combine both set of data in the XML would be interesting.
Have you considered using Google's Chart API?
Well actually you can but it's probably not going to be worth it. It seems like a vector based method such as canvas and the VML alternatives for IE would be a better alternative for rendering graphs. Then you only have to pass the graph data to the browser.
If storing is not an option. Create image in memory and flush the binary image data to response from a script, like a php file. Just use the correct header:
Content-type: image/png
The image gets regenerated every time.
To be absolutely sure the image does not get cached, append some random parameter to the querystring.
You could return the image to the ajax-client, and then include the XML data in a X-HTTP-Header for the image.
But you need to find out if it is possible to read the X-header from the ajax-client though.