I have been working on an experiment to render HTML into a canvas image, by having javascript read all the necessary information from the loaded DOM. As canvas lacks many of the standard parts of CSS, especially when it comes to text formatting, a lot of work arounds and performance intense processes need to be done (letter-spacing for one). The intent is and never will be to make a fool proof HTML renderer, as it simply won't be possible, but instead try and make as accurate as it can be.
For the sample pages, Google Chrome usually loads them significantly faster than FF. However, for some pages (usually the bigger ones), Chrome completely freezes, where as Firefox loads them fine. Now, I have been trying to pinpoint where exactly things go haywire, but haven't had much luck as it doesn't end up outputting anything in Chrome.
Does Chrome have some limit of how many canvas draws can be performed within a certain time span, or how much system resources a page can use? How can I start untangling the bottleneck if I can't get any sort of feedback from the page at all (as it just hangs up)?
Examples (what it should do, is render a canvas image on top of the page, which should look more or less same than the actual HTML page. You can toggle the canvas image (show/hide) by clicking it. Please don't open them either if you got unsaved work in your browsers, as it may end up hanging them as well.):
simple test, works fine in FF/Chrome
another simple test, works fine in FF/Chrome
Complete page, works fine in FF/Chrome
Complete page, only works in FF < 4, Chrome freezes
They all use the same js which can be found here.
I am not looking for a blazing fast script, as with the type of emulation this renders the images, I don't think it could even be done. Simply trying to find ways to make it perhaps slightly more efficient, without losing any of its current functionality.
Where to begin?
Break it down.
Use the same example and cut how much you do it it (your rendering code) in half. Does it still not work? In half again, etc. Did it work? Put back half of what you took out.
As in, get rid of all attempted text rendering, or all border/padding code. Just comment it out. Does it work then?
Or try just commenting out ctx.drawImage(img,x,y); on line 199 and nothing else. Does it work then?
If you're lucky you'll be able to determine a critical point where Chrome is spending a lot of time doing something.
Have you tried using Chrome's built-in performance profiler?
The problem appeared to be with the css attribute background-repeat, and specifically repeat-x. Commenting out
for(bgx=(x+background_position_left);bgx<=w;){
drawImage(image,bgx,(y+background_position_top));
bgx = bgx+image.width;
}
Fixed the issue at least for chrome, and looking at that it most likely was an endless loop as Kinlan proposed, but why exactly it gets stuck only on newer versions of FF and chrome is something I'll need to look more in detail (most likely not having the image.width available yet, or something similar).
Related
I am looking for a proper way to implement lazy loading of images without harming printability and accessibility, and without introducing layout shift (content jump), preferrably using native loading=lazy and a fallback for older browsers. Answers to the question How lazy loading images using JavaScript works?
included various solutions none of which completely satisfy all of these requirements.
An elegant solution should be based on valid and complete html markup, i.e. using <img src, srcset, sizes, width, height, and loading attributes instead of putting the data into data- attributes, like the popular javascript libraries lazysizes and vanilla-lazyload do. There should be no need to use <noscript> elements either.
Due to a bug in chrome, the first browser to support native lazyloading, images that have not yet been loaded will be missing in the printed page.
Both javascript libraries mentioned above, require either invalid markup without any src attribute at all, or an empty or low quality placeholder (LQIP), while the src data is put into data-src instead, and srcset data put into data-srcset, all of which only works with javascript. Is this considered an acceptable or even best practice in 2020, and does this neither harm the site accessibility, cross-device compatibility, nor search engine optimization?
Update:
I tried a workaround for the printing bug using only HTML and CSS #media print background images in this codepen . Even if this worked as intended, there would be a necessary css directive for each and every image, which is neither elegant nor generic. Unfortunately there is no way to use media queries inside the <picture> element either.
There is another workaround by Houssein Djirdeh at at lazy-load-with-print-ctl1l4wu1.now.sh using javascript to change loading=lazy to loading=eager when a "print" button is clicked. The same function could also be used onbeforeprint.
I made a codepen using lazysizes.
I made another codepen using vanilla-lazyload .
I thought about forking a javascript solution to make it work using src and srcset, but this must probably have been tried before, the tradeoff would be that once the lazyloading script starts to act on the image elements, the browser might have already started downloading the source files.
Just show me your hideous code, I don't want to read!
If you don't want to read my ramblings the final section "Demo" contains a fiddle you can investigate (commented reasonably well in the code) with instructions.
Or there is a link to the demo on a domain I control here that is easier to test against if you want to use that.
There is also a version that nearly works in IE here, for some reason the "preparing for print" screen doesn't disappear before printing but all other functionality works (surprisingly)!
Things to try:
Try it at different browser sizes to see the dynamic image requesting
try it on a slower connection and check the network tab to see the lazy loading in action and the dynamic change in how lazy loading works depending on connection speed.
try pressing CTRL + P when the network connection is slow (without scrolling the page) to see how we load in images not yet in the DOM before printing
try loading the page with a slow network connection and then using FILE > PRINT to see how we handle images that have not yet loaded in that scenario.
Version 0.1, proof of concept
So there is still a long way to go, but I thought I would share my solution so far.
It is complex (and flawed) but it is about 90% of what you asked for and potentially a better solution than current image lazy loading.
Also as I am awful at writing clean JS when prototyping an idea. I can only apologise to any of you brave enough to try and understand my code at this stage!
only tested in chrome - so as you can imagine it might not work in other browsers, especially as grabbing the content of a <noscript> tag is notoriously inconsistent. However eventually I hope this will be a production ready solution.
Finally it was too much work to build an API at this stage, so for the image resizing I utilised https://placehold.it - so there are a few lines of redundant code to be removed there.
Key features / Benefits
No wasted image bytes
This solution calculates the actual size of the image to be requested. So instead of adding breakpoints in something like a <picture> element we actually say we want an image that is 427px wide (for example).
This obviously requires a server-side image resizing solution (which is beyond the scope of a stack overflow answer) but the benefits are massive.
First of all if you change all of your breakpoints on the site it doesn't matter, so no updating picture elements everywhere.
Secondly the difference between a 320px and 400px wide image in terms of kb is over 40% so picking a "similarly sized" image is not ideal (which is basically what the <picture> element does).
Thirdly if people (like me) have massive 4K monitors and a decent connection speed then you can actually serve them a 4K image (although connection speed detection is an improvement I need to make in version 0.2).
Fourthly, what if an image is 50% width of it's parent container at one screen size, 25% width of it's parent container at another, but the container is 60% screen width at one screen size and 80% screen width at another.
Trying to get this right in a <picture> element can be frustrating at best. It is even worse if you then decide to change the layout as you have to recalculate all of the width percentages etc.
Finally this saves time when crafting pages / would work well with a CMS as you don't need to teach someone how to set breakpoints on an image (as I have yet to see a CMS handle this better than just setting the breakpoints as if every image is full width on the screen).
Minimal Markup (and semantically correct markup)
Although you wanted to not use <noscript> and avoid data attributes I needed to use both.
However the markup you write / generate is literally an <img> element written how you normally would wrapped in a <noscript> tag.
Once an image has fully loaded all clutter is removed so your DOM is left with just an <img> element.
If you ever want to replace the solution (if browser technology improves etc.) then a simple replace on the <noscripts> would get you to a standard HTML markup ready for improving.
WebP
Of course this solution requests WebP images if supported (its all about performance!). On the server side you would need to process these accordingly (for example if an image is a PNG with transparency you send that back even if a WebP image is requested).
Printing
Oh this was a fun one!
There is nothing we can do if we send a document to print and an image has not loaded yet, I tried all sorts of hacks (such as setting background images) but it just isn't possible (or I am not clever enough to work it out....more likely!)
So what I have done is think of real world scenarios and cover them as gracefully as possible.
If the user is on a fast connection we lazy load the images, but we don't wait for scroll to do this. This could mean a bit more load on our servers but I am acting like printing is highly important (second only to speed).
If the user is on a slow connection then we use traditional lazy loading.
If they press CTRL + P we intercept the print command and display a message while the images are loading. This concept is taken from the example OP gave by Houssein Djirdeh but using our lazy loading mechanism.
If a user prints using FILE > PRINT then we instead display a placeholder for images that have not yet loaded explaining that they need to scroll the page to display the image. (the placeholders are approximately the same size as the image will be).
This is the best compromise I could think of for now.
No layout shifts (assuming content to be lazy loaded is off-screen on page load).
Not a 100% perfect solution for this but as "above the fold" content shouldn't be lazy loaded and 95% of page visits start at the top of the page it is a reasonable compromise.
We use a blank SVG (created at the correct proportions "on the fly") using a data URI as a placeholder for the image and then swap the src when we need to load an image. This avoids network requests and ensures that when the image loads there is no Layout Shift.
This also means the page is semantically correct at all times, no empty hrefs etc.
The layout shifts occur if a user has already scrolled the page and then reloads. This is because the <img> elements are created via JavaScript (unless JavaScript is disabled in which case the image displays from the <noscript> version of the image). So they don't exist in the DOM as it is parsed.
This is avoidable but requires compromises elsewhere so I have taken this as an acceptable hit for now.
Works without JavaScript and clean markup
The original markup is simply an image inside a <noscript> tag. No custom markup or data-attributes etc.
The markup I have gone with is:
<noscript class="lazy">
<img src="https://placehold.it/1500x500" alt="an image" width="1500px" height="500px"/>
</noscript>
It doesn't get much more standard and clean as that, it doesn't even need the class="lazy" if you don't use <noscript> tags elsewhere, it is purely for collisions.
You could even omit the width and height attributes if you didn't care about Layout Shift but as Cumulative Layout Shift (CLS) is a Core Web Vital I wouldn't recommend it.
Accessibility
The images are just standard images and alt attributes are carried over.
I even added an additional check that if alt attributes are empty / missing a big red border is added to the image via a CSS class.
Issues / compromises
Layout Shift if page already scrolled
As mentioned previously if a page is already scrolled then there will be massive layout shifts similar to if a standard image was added to a page without width and height attributes.
Accessibility
Although the image solution itself is accessible the screen that appears when pressing CTRL + P is not. This is pure laziness on my part and easy to resolve once a more final solution exists.
The lack of Internet Explorer support (see below) however is a big accessibility issue.
IE
UPDATE
There is a version that nearly works in IE11 here. I am investigating if I can get this to work all the way back to IE9.
Also tested in Firefox, Edge and Safari (mobile), seems to work there.
ORIGINAL
Although this isn't tested in Firefox, Safari etc. it is easy enough to get to work there if there are issues.
However accessing the content of <noscript> tags is notoriously difficult (and impossible in some versions) in IE and other older browsers and as such this solution will probably never work in IE.
This is important when it comes to accessibility as a lot of screen reader users rely on IE as it works well with JAWS.
The solution I have in mind is to use User Agent sniffing on the server and serve different markup and JavaScript, but that is complex and very niche so I am not going to do that within this answer.
Checking Latency
I am using a rather crude way of checking latency (to try and guess if someone is on a 3G / 4G connection) of downloading a tiny image twice and measuring the load time.
2 unneeded network requests is not ideal when trying to go for maximum performance (not due to the 100bytes I download, but due to the delay on high latency connections before initialising things).
This needs a complete rethink but it will do for now while I work on other bits.
Demo
Couldn't use an inline fiddle due to character count limitation of 30,000 characters!
So here is the current JS Fiddle - https://jsfiddle.net/9d5qs6ba/.
Alternatively as mentioned previously the demo can be viewed and tested more easily on a domain I control at https://inhu.co/so/image-concept.php.
I know it isn't the "done thing" linking to your own domains but it is difficult to test printing on a jsfiddle etc.
The proper solution for printable lazy loading in 2022 is using the native loading attribute.
<img loading=lazy>
The recommendation to use a custom print button has been obsoleted as chromium issue 875403 got fixed.
Prior recommendations included adding a custom print button (which did not fix the problem when using the native browser print functionality) or using JavaScript to load images onBeforePrint the latter not being considered a good solution, as loading=lazy, as a "DOM-only" solution, must not rely on JavaScript.
Beware that, even after the bug fix, some of your users might still visit your site with a buggy browser version.
#Ingo Steinke Before one dwells into answers for the concerns that you have raised, one has to go back and think about why lazy loading came about and for what detriment it solved on initiation as framework of thought. Keyword framework of thought... it is not a solution and I would go on a leaf to say it has never been a solution but framework of thought.
Why we wanted it:
Minimise unnecessary file fetching from server - this is bandwidth critical if one is running a large user base. So it was the internet version of just in time as in industrial production.
Legacy browser versions and before async and defer were popularised in JS/HTML, interactivity with the browser window remained hampered until all content was loaded.
Now broad band as we know it has only been around since the last 6-7 years in real sense of manner and penetration. We wanted it because we didn't want to encounter no.2 on low bandwidth. To be honest, there was and still is a growing concern and ideology of minifying and zipping JS and CSS files - all because that round trip to server and back should be minimised so that next item in the list could be fetched. Do keep in mind browsers tend to limit simultaneous downloading connections to around 6 at a time per window or active window. There is reasons why Google popularised the 3 second rule. If above were to let run on as it than 3 second rule will fall on its head as if it did not have legs.
So came along thought frameworks.
Image as CSS background: This came as it did not mess up the visual aspect of the page. Everything remained as it is in its place and then suddenly became colourful. It was time when web pages seemed to have elastic fit i.e. it was that bag which once filled with air suddenly poped-transformed into jumping castle. This was increasingly become bad idea as front end developer. So fixing height and with of the container then run images as background helped and HTML5 background alignment properties upgraded them self accordingly. There was even variant and still used as in use multiple backgrounds one being loading spiril or low end blured image version on top of which actual intended image was fetched. Since level down bacground would be fetched and populated everywhere in single instance of downloading it created a more pleasing visual and user knew what to expect. worked in printing as well even if intended image did not download.
Then came JS version of it by hijacking DOM either through data-src, invalid image tags removing src, and what not. only trigger the change when content is scrolled to. Obviously there would be lag but that was either countered through CSS approach implemented in JS or calculating scroll points and triggering event couple of pixel ahead. They all still work on the same premise.
There is one question that begs to be asked and you have touched it in your pretext .... none of it controls or alters browser native functionality. Browser might as will go fetch the item even before your script had anything to do with any thing.
This is the main issue here. BOM does not care and even want to care about what your script is asking to do all it knows if there is a src property fetch the content. None of the solutions have changed that. If we could change that functionality then thought framework would become solution.
I still believe browsers should not change that just for the sake of it and thus never gained tracking in debates. What browsers have done is pre-fetching known as speculative or look-ahead pre-parser, It is the single biggest improvement in browsers that deserves it credit. Just as we type url in address bar on every chnage of string browser is pre-fetching the content even though I had not typed the whole url. I had specially developed a programme where I grabbed anything that was received at server from these look-ahead pre-parsers. It takes less than second to get response at most times and browsers begin to process it all including images and JS. This was counter the jerky delayed elastic prone display as discussed in No.1 and No.2. It did not reduce the server hit however. The reason why we are doing lazy loading any ways. But some JS workaround gained traction as there was no src property so pre-parser did not fetch the image and was only done so when user actually sent to the page and events were triggered. Some browser have toyed with the idea of lazy loading them self but let go if it as it did not assume universal consistency in standard.
Universal Standard is simple if there is src property browser will fetch the item no if and buts. Imagine if that was not the case OMG hell would break loose on poor front-end developer.
So deep down what you are raising in debate is the question regarding BOM functionality as I have discussed above. There is no work around for it. In your case both for screen and print version of display. How to make sure images are loaded when print command is sent. Answer is simple for BOM print is after the fact. Fact ebing screen display and before the fact being everything before that at BOM/DOM propagation level. Again you cannot change that.
So you have to make trade off. Trade off would come in the form of another thought framework. rather than assuming everything is print ready make it print ready on user command. There is div that pops up and shows printed version of document and then print from there on. UI could be anything it would only take second or so as majority of the content would be loaded any ways and rest will take short amount of time. CSS rules for print could mighty handy in this respect. You can almost see it in action in may places on the internet.
conclusion as we stand today where we are with BOM functionality bundling the screen display and print display with lazy load is not what lazy lading was intended for thus does not provide any better solution then mere hacks. So you have to create your UI based on your context separating the two, to make it work properly.
I'm trying to optimize the Javascript in our browser app to make sure that animations always run at 60fps. I'm using the Chrome (57) Timeline tool analyse bottlenecks. Many are well solvable, but I also encounter a few very mysterious phenomena:
The screen shot below shows an empty timeline (during a long running CSS transformation), but still the frame exceeds 16ms. Where do I find what is going on there?
The other weird phenomenon is shown below, where part of a function's logic has been moved into a requestAnimationFrame() statement. That logic is nicely moved along further in time, still it seems to stretch the whole frame to 21.3ms. Why is that happening?
I hope someone can shed some light on these cases.
Thanks!
Pascal
I'm having a issue which is hard to debug. I'm using a Javascript library (the JQuery Flexslider plugin) in a number of different places on my site. It's all working fine except for one particular phone where it doesn't work and slows down everything on the page.
So far, I've only seen it happen on this one device. Other devices of the same type do not have the issue. This person has an iOS that's a few versions out of date and not much memory, so I think it's a memory issue.
An old hack was to move the carousel element that has the issue on the page with Javascript, but I want to find and fix the root issue.
How can I start debugging this? I'm not sure how to test for a memory issue on a device.
If you're on a Mac, then you can plug in and use remote debugging via Safari, where you'll have access to the tools, including the profiler (not sure the state of Safari support in Windows). There are numerous resources for showing how to remotely debug a device, unless it is a really old version of iOS you should be fine, you’ll have to enable the develop menu via settings but after that its plain sailing if you know your debugging tools.
I'd agree that it doesnt really sound like a memory issue, although jQuery tends to be hungry in that respect, I dont know the plugin in question but the quality of plugins is hugely variable in jQuery-land. Old phones and old versions of jQuery certainly never played well together.
When you say one phone, you mean one type of phone + iOS version? The question isnt clear, its almost reads like you have 2 identical phones/os's where 1 works and 1 does not.
If you use Chrome you can use the Heap profiler
First open your developer tool and start recording.
Next start using your page and try to replicate your issue, stop recording and review the stats.
This is likely not a memory issue, but a cpu issue. The way jQuery does animation is processor constrained on older dvices. Factors that are easiest to handle include:
size of the page (html length and complexity)
animation steps, length, and complexity
You have a couple of options here, but the simple answer is you are asking too much of the older processor. Assuming you are using this plugin http://www.woothemes.com/flexslider/ you could try disabling or simplifying some of the transition effects. animation and animationSpeed would be the first I would suggest.
If you are interested in not changing the experience for most users you could consider tying into the start and end functions on the callback api and checking the time it took to perform the first transition, then reinitialize a simpler version of the slideshow for that device.
The hard thing here is there isn't really a right answer. If one of the above options doesn't fix it you're likely looking at choosing/building a different slideshow, degrading the experience for everyone, or determining the best way you feel comfortable with choosing who gets the degraded experience.
I'm making a small game using the HTML5 canvas element. It works great, except that it has a scrolling background with obvious tearing happening in Firefox and Chromium browsers in Ubuntu. I'm pretty sure it's buffered because there isn't any of the flickering I'd expect; just tearing. Is there any way to work around this or time rendering to right after the last screen refresh?
Currently there is no way to control the actual repainting of a canvas element(which if there was, could actually help in increasing performance I guess). So one can only hope that the browser actually does something intelligent, rather than screwing up like in your case.
I myself have quite some experience with the canvas element and know of its quirks. I ran into some kind of "repaint lag" several times by now, where obviously the actual numbers behind the scenes are correct and "smooth", but the graphics still have a somewhat "jumpy" behavior, which in fact is really annoying.
Only thing I can imagine that could have an effect in your case, is activating VSync in the driver settings of your Graphics Card.
If you'd like to provide a link to your game that might be helpful too, since I'm also running Ubuntu here.
I am thinking as a challenge i should write a javascript based game. I want sound, images and input. A background to simulate a screen (like 640x480 with all my images in it) would be useful to separate the rest of the page from the 'game'. What should i look at?
Some things i would need
Framecontrol. A way to get the current time (or delta).
Image, displaying it and moving it. How do i display full image. Knowing pixel access may be cool.
Input A way to lock it in a box (like flash does) is cool.
Sound play simple sounds on demand (like when i get a hit). Several sounds at once would be great
Bottlenecks. What are things that will kill the CPU?
Restrictions. What cant i do? I hear i cant 'sleep' to wait. I must set a callback
Good or best pratice. What are good things i can do to either keep speed up or to lower glitch or compatibility problems.
I'm going to answer this looking at things from a mootools javascript perspective:
Framecontrol. A way to get the current time (or delta).
periodical()
Image, displaying it and moving it. How do i display full image.
setStyles()
Input A way to lock it in a box (like
flash does) is cool.
Plain old CSS
Sound play simple sounds on demand
(like when i get a hit).
Swiff, remote();
Bottlenecks. What are things that will
kill the CPU?
Internet Explorer.
Restrictions.
3D ... ?
What are good things i can do ... to
lower glitch or compatibility
problems.
Use a framework.
As a starting point, you may want to write it for Opera, as Opera provides a game canvas that will help you out.
For some examples of games in javascript:
http://dev.opera.com/articles/view/3d-games-with-canvas-and-raycasting-part/
http://my.opera.com/WebApplications/blog/show.dml/200788
This one is interesting, it is demos of games using the canvas element.
http://www.canvasdemos.com/tag/games/
The best way to see where the problems are is to start writing the game, and then you will see what may be a problem. By looking at demos you can get an idea what performance issues they encountered. For example, a full 3D Doom game will have problems, but, as the first article above explains, there are some ways to optimize for javascript.
Once you get it working with Opera, then you can look at Firefox 3.5+ and Safari, as well as Chrome, and see if you can make some changes to have it work on those. How many platforms it works on depends on how much work you want to do for it. For a proof-of-concept pick the easiest browser for your task.
The best place to start would be to get very familiar with the <canvas> tag (it allows you to draw anything on screen)
This may help a lot:
http://benfirshman.com/projects/jsnes/
its an online NES emulator that renders everything on screen - the source is also available
Hope that helps =)