I know that it's possible to take a series of images and use javascript to set opacity from 1 to 0 very quickly for each one in rapid secession. I have actually done it before very successfully, although I only used 41 images at approx. 720p.
My question though, is whether or not it would be practical to make an entire video (4-10 minutes long) using only html, css, and javascript. Obviously that would be too many images to leave in the cache, so you'd have to empty the cache every so often of particular images, and also this would require a pretty good internet connection, but do you think it would be worth while to attempt?
are there any obvious pro's and cons that you could think of for trying this?
(to be clear, I'm not asking for the code to achieve it as I have already developed most of it, just what you think in terms of practicality of it as opposed to youtube or vimeo etc...)
Plainly put, I don't think it's practical.
Here's a few reasons why:
Number of images. Videos tend to be around 30 frames per second.
For a 4 min video, that's roughly 7,200 images.
Download time. Downloading 7,200 images at 720 pixels high would require a
significant amount of bandwidth, not only for the user, but also for
the server.
DOM load. 7,200 images would require just as many DOM
elements positioned correctly behind each other.
Rendering. Each
time we do an animation (fadeout), the browser has to calculate what
element is visible and what that means to the user (pixel color,
etc.)
Of course, we can optimize this:
Download the images that is needed at that specified time. If the connection can download 30 images/second, then we load what we need and then initiate the play sequence.
Remove elements when it's done. As the sequence goes on, we can assume that those images are no longer needed. When those images are done, we can destroy those elements and thus free up a few resources.
Put images on multiple sub-domains/locations. Browsers tend to download 1 asset per domain at a single moment. Modern browsers may download 6 or more. But if we were to split those assets to multiple subdomains, we can increase that number of simultaneous downloads and thus increase response time.
Create large sprites. Have the sequence on one or several large sprites, and use Javascript to correctly position the element. This allows us to have a single/few downloads and removes the overhead from multiple downloads.
Videos are encoded so that the browser doesn't have to do all those pixel calculations and animations. Each frame isn't a full image, but rather a delta between the current and next frame. (I'm overly simplifying this.)
But this method is constantly used for pieces that need interactivity, or want to get past iOS auto-play hangups. Again, not practical for large sequences, but definitely doable for shorter sequences.
Related
I'm creating an online psychology experiment in JavaScript with the goal of studying people's ability to recognize images quickly (this RSVP, Rapid Serial Visual Presentation, paradigm is a common approach in psychophysics for isolating the feedforward, pre-attentive stages of visual processing). I aim to show a sequence of images for brief durations (e.g. 30 ms, 50 ms, 70 ms, etc.). For valid results, it's important to get the presentation times as precise as possible. Usually, this is done in a lab with specialized software or equipment, but this setup isn't an option for me. I recognize that some slop is an inevitable side effect of doing this experiment in a browser, but I'm hoping to (A) minimize it as much as possible, and (B) quantify, to the extent that I can, the amount of imprecision that actually occurred.
Right now, I'm preloading images, and am using setTimeout() to control stimulus duration. I'm assuming that experiment participants will use a variety of monitors (so refresh rates may differ) and browsers.
By eye, the timings of the images with my current approach seem to be wrong, and, in particular, highly variable. This is consistent with things I've read saying that variability for brief presentation times can be high.
Are there tools available to control timing more precisely in JavaScript? Additionally, are there tools that can measure/estimate the time an image was actually shown on screen? Finally, are there other options I should look at, e.g. compiling the sequence of images into a video, gif, etc.?
setTimeout does not specify when an action should take place. It only states that an action should take place once the timer has expired; but there's no way to guarantee when exactly the event will take place (https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout).
When working on (psychological) expiriments, and when you have no control over the participant's environment, I would argue (apart from not entertaing such an uncontrolled environment in such cases) for claiming the whole tab the user is using (effectively blocking any interaction while the image is shown) or, as you already mention yourself, using an animated gif or sequence of images.
Or you can monitor and store the time the image was actually shown to your participant (using checking when the image comes into the viewport), and compare that timestamp with the participant's interaction. That would perhaps give the most valid results, as it will eliminate the inpredictability of setTimeout.
I am gonna use my webcam as a source and show my view on webpage , than I will manipulate my view like (blacknwhite , fiseye, etc.) and show that manipulated video in my canvas.
An example ( http://photobooth.orange-coding.net/ )
Ok everything is cool for now . I can capture that manipulated canvas as a image.
Is there any way to record that manipulated canvas as video?
I also found an example (https://www.webrtc-experiment.com/ffmpeg/audio-plus-canvas-recording.html)
But when I tried that code on my webcam recording project , it's just recording my source view(not blacknwhite) . It is not implementing my effect to record.
Any idea or is it possible ?
Thank you.
Recording video in the browser is like getting blood out of a stone. If you hit it hard and long enough against your head, there will be blood, eventually. But it's a painful experience, you it will certainly give you a headache!
There is currently no way of recording video in real-time from a canvas element. But there is proposed a Mediastream Recording API which includes video (and it excludes the canvas part). Currently only audio is supported, and only if FF.
You can grab an image as often as possible and use it as a sequence, but there are several issues you will run into:
You will not get full frame-rate if you choose to grab the image as JPEG or PNG (PNG is not very useful with video as there is no alpha)
If you choose to grab the raw data you may achieve full frame rate (note that frame rate for video is typically never above 30 FPS) but you will fill up the memory very quickly, and you would need a point in time to process the frames into something that can be transferred to server or downloaded. JavaScript is single threaded and no matter how you twist and turn this stage, you will get gaps in the video when this process is invoked (unless you have a lot of memory and can wait until the end - but this not good for a public available solution if that's the goal).
You will have no proper sinc like time-code (to sync by) so the video will be like the movies from Chaplins day, variable. You can get close by binding high-resolution timestamps but not accurate enough as you will have no way of getting the stamp at the very time you grab the image.
No sound is recorded; if you do record audio in FF using the API, you have no way to properly sync the audio with the video anyways (which already has its own problems ref. above)
Up until now we are still at single frame sequences. If you record one minute # 30 fps you have 60x30 frames, or 1800 pictures/buffers per minute. If you record in HD720 and choose grabbing the raw buffer (the most realistic option here) you will end up with 1800 x 1280 x 720 x 4 (RGBA) bytes per minute, or 6,635,520,000 bytes, ie. 6.18 GB per minute - and that's just in raw size. Even if you lower the resolution to lets say 720x480 you'll end up with 2.32 GB/min.
You can alternatively process them into a video format, it's possible, but currently there are next to none solutions for this (there has been one, but it had varying result which is probably why it's hard be found...), so you are left to this yourselves - and that is a complete project involving writing encoder, compressor etc. And the memory usage will be quite high as you need to create each frame in separate buffers until you know the full length, then create a storage buffer to hold them all and so forth. And even if you did, compressing more than 6 GB worth of data (or event "just" 2 GB) is not gonna make user or browser very happy (if there is any memory left)...
Or bite the dust and go with a commercial Flash based solution (but that excludes your image processing and pretty much takes over the camera... so not really an option in this case).
The only realistic option, IMO, is to wait for the aforementioned API - this will let your browser do all the hard work, in compiled optimized code, enable frame by frame compression leaving the memory pretty much intact, and give very little headache compared to the alternative(s) above. There may be an option to apply shaders to the stream at one point, or integrate it with some canvas processing (not on the table in this proposal AFAICS) so recording real-time from a canvas will still be a challenge.
This is where server side processing comes in...
(of course, a screen recorder is an option which is of curse completely non-integrated, but will enable you to demo your effects at least...).
The ability of a browser (Chrome for example) to handle a number of images, is limited only by the hardware of the computer on which it is displayed or also by the software itself?
I'm trying to develope an image viewer that must content lot of files that must be accesible instantly depending on the demand of the user and sometimes when i go over 350 files of 300kb the page frozens.
Thank you all for your help!!
It is probably limited by both.
There are limits to how large a struct is allowed to be for instance, because it has to fit in a fixed amount of bytes.
(see this question for instance)
(also, you're not yet running into the max size of an int so that is probably not what is happening right now.)
Besides this there are several more constraints.
That said, loading a gigantic amount of images every time you open a page is probably not a good idea.
Take some inspiration from how others (like google in their image search) have solved this problem simply by not loading the images until they are needed.
I think the best approach would be to have some small thumbnails of each image and on request of the user(click) load the bigger one, like Timothy said.
If you need it to be faster you can preload images. So for example if you have a list of images and the user scrolls through you just load the next n images. To free space you can "delete" the ones the user did already passed.
Does anyone know why Javascript performance would be affected by the loading of lots of external JPG/PNG images into HTML5 Image() objects, totalling approx 200Mb-250Mb. Performance also seems to be affected by cache. Ie. if the cache is full(-ish) from previous browsing the performance on the current site is greatly reduced.
There are 2 says i can crudely solve it.
clear cache manually.
minimise browser, wait about 20 secs and re-open the browser after which time the iOS/browser has reclaimed the memory and runs the JS as it should.
I would have expected the iOS to reclaim required memory to run the current task, but it seems not. Another workaround is to load 200Mb of 'cache clearing' images into Image() objects, then remove these by setting the src = "". This does seem to help, but its not an elegant solution...
please help?
First and foremost read the excellent post on LinkedIn Engineering blog. Read it carefully and check if there are some optimizations that you can also try in your application. If you tried all of them and that still haven't solved your performance issues read on.
I assume that you have some image gallery or magazine-style content area on your page
How about having this image area in a separate iframe? What you could do then is this:
Have two iframes. Only one should be visible and active in time.
Load images into first iframe. Track the size of loaded images. If exact size tracking is hard
numberOfLoadedImages * averageImageSize
might be a pretty good aproximation.
As that number approaches some thresshold start preloading the currently visible content into second iframe.
Flip the visibility of iframes so the second one becomes active.
Clear the inner content of the first frame.
Repeat the whole procedure as necessary.
I don't know for sure if this would work for you but I hope that WebKit engine on iPad clears the memory of frames independently.
EDIT: It turned out you're writing a game.
If it's a game I assume that you want to have many game objects on the screen at the same time and you won't be able to simply unload some parts of them. Here are some suggestions for that case:
Don't use DOM for games: it's too memory-heavy. Fortunately, you're using canvas already.
Sprite your images. Image sprites not only help reducing the number of requests. They also let you reduce the number of Image objects and keep the per-file overhead lower. Read about using sprites for canvas animations on IE blog.
Optimize your images. There are several file size optimizers for images. SmushIt is one of them. Try it for your images. Pay attention to other techniques discussed in this great series by Stoyan Stefanov at YUI blog.
Try vector graphics. SVG is awesome and canvg can draw it on top of canvas.
Try simplifying your game world. Maybe some background objects don't need to be that detailed. Or maybe you can get away with fewer sprites for them. Or you can use image filters and masks for different objects of the same group. Like Dave Newton said iPad is a very constrained device and chances are you can get away with a relatively low-quality sprites.
These were all suggestions related to reduction of data you have to load. Some other suggestions that might work for you.
Preload images that you will need and unload images that you no longer need. If your game has "levels" or "missions" load sprites needed only for current one.
Try loading "popular" images first and download the remaining once in background. You can use separate <iframe> for that so your main game loop won't be interrupted by downloads. You can also use cross-frame messaging in order to coordinate your downloader frame.
You can store the very most popular images in localStorage, Application Cache and WebSQL. They can provide you with 5 mb of storage each. That's 15 megs of persistent cache for you. Note that you can use typed arrays for localStorage and WebSQL. Also keep in mind that Application Cache is quite hard to work with.
Try to package your game as a PhoneGap application. This way you can save your users from downloading a huge amount of data before playing the game. 200 megs as a single download just to open a page is way too much. Most people won't even bother to wait for that.
Other than that your initial suggestion to override cache with your images is actually valid. Just don't do it straight away. Explore the possibilities to reduce the download size for your game instead.
I managed to reduce the impact by setting all the images that aren't currently in the viewport to display:none. This was with background images though and I haven't tested over 100Mb of images, so can't say whether this truly helps. But definitely worth of trying.
I need to dynamically load and put on screen huge number of images — it can be something like 1000–3000. Usually these pictures are of different size, and I'm getting their URLs from user. So, some of these pictures can be 1024x800 or 10x40 pixels.
I wrote a good JS script showing them nicely on one page (ala Google Images Search style), but they are still very heavy on RAM used (a hundred 500K images on one page is not good), so I thought about the option of really resizing images. Like making an image that’s 1000x800 pixels something like 100x80, and then forget (free the ram) of the original one.
Can this be done using JavaScript (without server side processing)?
I would suggest a different approach: Use pagination.
Display, say, 15 images. Then the user click on 'next page' and the next page is shown.
Or, even better, you can script that when the user reaches the end of the page the next page is automatically loaded.
If such thing is not what you want to do. Maybe you want to do a collage of images, then maybe you can check CSS3 transforms. I think they should be fast.
What you want to do is to take some pressure from the client so that it can handle all the images. Letting it resize all the images (JavaScript is client side) will do exactly the opposite because actually resizing an image is usually way more expensive than just displaying it (and not possible with browser JS anyway).
Usually there is always a better solution than displaying that many items at once. One would be dynamic loading e.g. when a user scrolls down the page (like the new Facebook profiles do) or using pagination. I can't imagine that all 1k - 3k images will be visible all at once.
There is no native JS way of doing this. You may be able to hack something using Flash but you really should resize the images on the server because:
You will save on bandwidth transferring those large 500K images to the client.
The client will be able to cache those images.
You'll get a faster loading page.
You'll be able to fit a lot more thumbnail images in memory and therefore will require less pagination.
more...
I'm (pretty) sure it can be done in browsers that support canvas. If this is a path you would like to take you should start here.
I see to possible problems with the canvas approach:
It will probably take a really long time (relatively speaking) to resize many images. Because of this, you're probably going to have to look into utilizing webworkers.
Will the browser actually free up any memory if you remove the image from the DOM and/or delete/null all references to those images? I don't know.
And some pretty pictures of a canvas-resized image:
this answer needs a ninja:--> Qk