The Yahoo Javascript library (YUI), JQuery and less so Google maps all allow you to reference their files using the following format:
<script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/yahoo-dom-event/yahoo-dom-event.js"></script>
This does a request for the script from their servers, which will also pass to their web server the HTTP referrer. Do Yahoo etc. use this to produce statistics on which websites get what traffic? Or is this a conspiracy theory?
Of course their servers most of the time will be a lot faster than any small company would buy, so using the hosted version of the script makes more sense.
Chris,
I work on the YUI team at Yahoo.
We host only YUI on yui.yahooapis.com; Google hosts YUI and many other libraries on its CDN. I can tell you from the Yahoo side that we don't monitor site usage of YUI from our CDN. We do track general growth of yui.yahooapis.com usage, but we don't track which sites are generating traffic. You're right to suggest that we could track usage -- and we state as clearly as we can in our hosting docs that you should only use this kind of service if the traffic logs generated on our side don't represent a privacy concern for you.
In general, though, I don't regard CDN traffic for library usage to be a reliable measurement of anything. Most YUI usage, even at Yahoo, doesn't use yui.yahooapis.com or Google's equivalent, and I'm sure the same is true for other libraries. And even when a site is using YUI from our servers, we wouldn't have comprehensive traffic data of the kind you'd get from Google Analytics or Yahoo Analytics -- because not all pages would use YUI or the CDN uniformly.
Given the advantages of the hosted service -- including SSL from Google and YUI combo-handling from Yahoo -- I see the CDN as being a big win for most implementers, with little downside.
-Eric
Of course they produce statistics - at minimum they need to know how many resources they spend on hosting these scripts. And it's also nice to know who uses your code.
I don't think it's a bad thing.
And using a hosted version makes even more sense because your visitors might have the script already cached after visiting another site.
Sure, they can easily have statistics about which sites use YUI and how often, and also which parts of YUI API are more populare (among small sites). However, they cannot know what exactly web site visitors do with their libs.
Given, that they (Google & Yahoo) index a lot of web pages, they can get an even more precise statistics if they analyze their indexes. So you cannot hide that you are using YUI if your site is public.
The same applies to Google maps and jQuery.
Related
I (or a lot of German people) need your help.
In Germany, more and more website operators are receiving a legal letter with a warning and are supposed to pay around €170. The problem is that it doesn't stop there, which means that if you pay the €170, someone else can come right away and warn you again.
It's about Google Fonts. Many Wordpress websites use themes that load Google Fonts. The German court has decided that it is not allowed to send the IP address to the Google because of Google Fonts and this is a violation of the rights of the customers.
Since I run a few websites, I'm now looking for a solution, but to be honest I'm coming up against technical limits. So I want to open this thread to discuss possibilities.
I have listed issues below, I will add them to my solutions.
I can think of the following options:
Create a child theme and then load the Google fonts locally. (Issue: 1st 2nd 3rd 4th)
Service worker that rewrites the URLs
(Issue: 5.)
Nginx rewrite, rewrite the php output and replace google fonts url
(Issuing: 1. 4.)
More?
Issues:
If you have e.g. integrated a script (Google Maps, Recaptcha, Intercom, ...) it can happen that Google Fonts are reloaded by Javascript.
Theme updates.
A lot of work when multiple customers.
Plugins load elements on certain pages or only later after it can happen that google fonts are loaded again.
Only works if the service worker is installed
I am open for any idea. It looks like Google will not fix this.
There is no easy technical fix. The only long-term fix is to review how you include any third-party content on your websites, in case this embedding causes any visitor personal data to flow to such third parties.
This is not a new development. A lot of the relevant compliance steps already entered the (German) mainstream in the early 2010s when the problem was Facebook's “Like button”. The generally accepted solution for that is that the third party content is not loaded directly. Instead, a placeholder widget is rendered that indicates which content would be available there. Then, the user can give consent with one click and the actual embedded content is loaded.
With Google Fonts, no such consent flow is needed or appropriate. All of the fonts on Google Fonts are Open Source licensed – you are allowed to use and distribute them for free, but subject to the license conditions (like making the license notice available to users). So on a technical level, it is easy to self-host the fonts in question.
What is tricky is efficiently rewriting the requests caused by your websites to point to your own servers instead of to the Google servers. You have identified a couple of approaches and have identified their pros and cons. Some comments:
Client-side rewriting sounds very fragile, I'd avoid it.
Server-side rewriting can be very powerful, but would also be somewhat fragile. The main advantage of such rewrites would be that it doesn't just handle Google Fonts embeds from your themes, but also requests inserted by server-side plugins.
Updating the theme is the only reliable long-term solution. Creating a child theme might be a suitable stop-gap measure until the theme developer fixes the problem. Similarly, it may be necessary to temporarily modify WordPress plugins.
I think that as a band-aid, server-side rewrites will be good enough to prevent many automated scanning tools used by these cease-and-desist lawyers from sounding the alarm on your sites.
However, you have correctly identified that especially JavaScript could cause problems for achieving actual compliance. This is why you should revisit your decisions about what plugins and scripts you have integrated. Loading third party JavaScript has pretty much the same legal consequences as loading fonts from Google, so you should only do it if it's actually necessary for your site (where necessity depends on the user's perspective), or if the user has given valid consent. For example, you can use the placeholder widget technique mentioned above for embedded content like Google Maps or Intercom, whereas loading a Captcha may indeed be strictly necessary on some pages.
For testing these issues, I'd recommend installing Firefox with the uBlock Origin addon, and setting the addon to hard mode. This will block all third-party/cross-origin requests. You can then allowlist those domains that are under your direct control, or are provided by your data processors (who are contractually bound to only use the personal data as instructed by you, and are considered first-party for GDPR purposes), or domains for which you have a legal basis to share the data (e.g. a “legitimate interest” to load stuff that is strictly necessary for your site to work, or to investigate what requests are made when the user gives consent).
IANAL but the two sections may be relevant.
Using their APIs. From what I can tell nothing here explicitly forbids proxying.
API Prohibitions on sublicensing. The last part of the statement and offer it for use by third parties means you're okay as long as you're not offering it for other people to use.
I do have Google Fonts Proxy Docker Image which I use for my own stacks, I don't offer the use of my running proxy for use with other services. It does not mean you can't simply deploy my image on your own servers.
This won't resolve your 3rd party Google services such as Maps though.
Speed tests report that my site loads in about 8 seconds on mobile and 2 on desktop.
When looking at the "waterfall" of the asset/script loading it seems to be mostly due to 3rd party analytic scripts like zendesk, hubspot, and google analytics.
Is there anyway to optimize a site for mobile when using these type of scripts?
As far as the files go on my site I've optimized them nearly as much as I could. I've even used a cron (probably not a great idea) to cache the google analytics script locally and fetch a newer version every few days.
I've looked into using Tag Manager or something like Segment to optimize the script loading of these files, but I'm not sure if that will actually improve performance or if those services are mostly just for convenience.
I've also looked at service workers for mobile app caching, but I'm not sure if that will help either and don't want to dive into learning how to use them if it won't actually make a difference.
To sum up, is there a way to speed up mobile with multiple 3rd party analytic scripts or am I just going to have to forgo using some of them or possibly add a mobile version or AMP version without using them or some of them.
Since you've tagged AMP in your question, I'll answer from the AMP perspective:
AMP solves this problem by using a mediator that connects directly to the endpoints of various analytics providers, replacing their clunky and power hungry JS scripts.
Of course, AMP comes with its own set of constrains and is really only targeted at powering static content, but if you're willing to go that route, learn more about AMP Analytics here.
Can somebody please answer the question below regarding the Google API for Charts?
1- Is Google Charts API completely independent of any other JS libraries or software like Flash etc?
2- Does it work completely offline (using the new version)?
3- Is it completely free for commercial usage?
Thank you.
The API is dependent on Google's internal libraries (nothing you have to specifically load), but generally not any external libraries. The exceptions are for the AnnotatedTimeLine and Motion charts, which are Flash-based. 3rd-party visualizations may be dependent on other libraries.
You cannot use the API offline. Downloading the API and hosting it locally is strictly forbidden in the Terms of Service.
The API is free for almost all commercial usage, see Terms of Service for details.
The google charts api uses js libraries.
You can't use it offline
your computer must have live access to http://www.google.com/jsapi in order to use charts. This is because the visualization libraries that your page requires are loaded dynamically before you use them. The code for loading the appropriate library is part of the included jsapi script, and is called when you invoke the google.load() method. Our terms of service do not allow you to download the google.load or google.visualization code to use offline.
(https://developers.google.com/chart/interactive/faq#offline)
3 . The api is completly free (https://developers.google.com/chart/terms?hl=nl#license)
If I wanted to get a javascript library published to the ajax CDNs hosted by Google or Microsoft, what would I have to do?
Are there any formal requirements for this, like number of users etc?
I doubt there are any formal requirements except that the lib has to be wildly popular, and probably will have to be regarded to be of high quality by the companies running the CDNs.
Google's Ajax libraries main page has this to say:
Google works directly with the key stake holders for each library effort and accepts the latest stable versions as they are released. Once we host a release of a given library, we are committed to hosting that release indefinitely.
I'd say if you feel your library is popular and good enough - seeing as Google for example are hosting 12 projects at the moment, yours would have to be in the world wide top twenty by some measure though! - simply talk to Google and Microsoft and see what they say.
Here is a blog post that could provide you with some contacts to approach. Also, the author seems to be somehow affiliated with Google (he's talking about "we").
The Google Ajax Library Blog may also be a good resource.
I am seeking to (legitimately) plant bugging in my web pages to collect and report information about website performance.
Preference for internally hosted. While I expect that there are commercial offerings out there (e.g. Google Analytics) I'm keen to find something we can run entirely in-house (its not a public website and may contain sensitive data).
Also, I'm looking for something where it can report back to an independent URL - i.e. not relying on adding in a reverse-proxy / recording results within existing webserver logs. Indeed, I'd prefer something which does not require access to the webserver logs logs at all (other than those for the URL the bug reports back to).
I need to be able to monitor bulk traffic - so things tools like pagespeed and tamperdata are not appropriate.
I've tried googling but just seem to be getting lots of noise about the performance of javascript and web pages rather than how to actually measure these.
TIA
You could use the open source analytics software Piwik and write a plugin for it that sends the performance data to it.
Thanks chiborg. I'd kind of forgotten about this it was so long ago I asked. Yes, I was aware of PiWik - but not been very impressed with either its implementation nor the quality of documentation.
I'm currently working on a solution using Boomerang.