I've recently finished working with Google API to transport all of the Google Analytics data into my own database. However, I realized that I will have to run my script constantly to get the data up-to-date daily (Or run Cron to do it everyday).
Can someone suggest me a different solution? I've seen from somewhere that putting "some" code in the Javascript to pass the event variable to both Google Analytics and my own server, which I think is more complete solution. I've been searching through the web to find this solution, and I'm continuing to do so at the moment. I am very new to this and any help or direction will be very much appreciated. Please let me know!
With the (soon to be deprecated) asychronous Analytics code (ga.js) there are the _setLocalGifPath and _setLocalRemoteServerMode methods (see documentation) that allowed to send raw data to a local server. This was for backward compatibilty with the (now defunct) Urchin Analytics software (Google acquired Urchin in the early 2000s and developed their product into what is now Google Analytics).
Since there is now completely new Analytics code based on the measurement protocol this does not work with Universal Analytics.
Edited to add: Refering to your comment, you can set a hit callback for interaction events (pageviews, events etc). This is a custom Javascript function that is triggered to moment Universal Analytics sends an interaction hit. However you cannot use this to get exactly the same data Analytics gets - with Universal Analytics the data is evaluated on the server, so you don't have data about campaign source, session duration, pagedepth etc on the client side anymore (used to be the case with the "old" analytics). So I'm not sure hitCallback will be much help to you.
Related
I want to programmatically know the traffic channel for the current page using javascript google analytics (or reporting) API. Like organic, google search, direct, referral, etc.
I can get referral with
tracker.get('refferer')
but I need to know the traffic channel.
For example channels
Can I get it? And if I can, then how?
no, usually you can not access it since channels are based upon source and medium combinations and these fields are processed on the server side based on referrer, document.location and previous visits information. Google Analytics provides a chart of the processing flow on the help pages
Of course, you can replicate some of the processing on the client side, by parsing utm labels from the location.href or checking the referrer but 1) this would require a ton of scripting 2) this wouldn't necessary match Google Analytics data
Some workaround can be to use classic analytics ga.js snippet alongside your usual analytics code with any dummy property ID. in this case ga.js code would process much of the source data on the client site and these data would be available from the utm* cookies
The source / medium information is available only on the landing page, if you want to have it available on a subsequent page you need to store it in a cookie. Here's an example how to do it: https://www.youtube.com/watch?v=x2pR4bCW7Mg
At list you can get from which URL you were visited:
Javascript -> document.referrer
PHP -> $_SERVER['HTTP_REFERER']
ASP -> Request.ServerVariables("HTTP_REFERER")
I am looking to implement Google Analytics using analytics.js rather than ga.js. I would like to implement this into my hybrid HTML5 app which is served within an iOS shell. I am aware of the iOS SDK but I am trying to maintain one code base for different platforms.
I need to consider my app recording page impressions within a disconnected scenario, so I was going to store page impressions in localStorage and then when an internet connection is detected, then send each page impression individually to the Analytics service using the ga() function call.
In theory this works but I need to be able to send the page impression timestamp with each page rather than Analytics applying the timestamp when it hits their server.
Can this be achieved?
Thanks,
Rit
You are correct the iOs sdk lets you use something called Dispatching this allows you to send data that is up to 4 hours old.
This works directly with the Measurement Protocol where it is called Queue Time
Used to collect offline / latent hits. The value represents the time delta (in milliseconds) between when the hit being reported occurred and the time the hit was sent. The value must be greater than or equal to 0. Values greater than four hours may lead to hits not being processed.
As far as I can see analytics.js doesn't allow you to do this, but it is just a JavaScript library that sends data to the Measurement Protocol. It would probably be tricky getting that to work with Analytics.js which is probably why Google hasn't added it.
My advice to you is to store your data as you say locally then send it directly to Google though the Measurement Protocol. Just remember it has to be less then 4 hours old, you will have to hope that network returns by then.
Background: I'm helping a client upgrade their Classic Analytics (using dc.js) to Universal Analytics and we're taking all precautions by upgrading on a staging website prior to pushing live.
We did have the live website transfer it's data to Universal Analytics in the process so when we're ready to push, it would be seamless and just a matter of validating the new universal code. My team and our client both did research and as far as we were aware, there were no issues in doing this.
My Question: Just a few days ago my client emailed me about their In-Page Analytics not reporting any revenue or transaction data from their ecommerece data. Click percentages are still showing up however.
My theory is that the transfer to Universal Analytics had something to do with it but I can't find any reference to this issue anywhere. There are a few variables that should be noted:
The last time anyone used this feature was about a month ago.
I did notice that "Enhanced Link Attribution" was turned on at some
point but the change history doesn't reflect when the change was
made. So I turned it off today (didn't do anything right away it seems).
The transfer to Universal Analytics reported as successful and no script changes were made.
Any insight, in-page analytics documentation or tests greatly appreciated!
In my site I have:
...
<script type="text/javascript" src="https://www.google.com/jsapi"></script>
...
The script above is the Google script to load up other resources dynamically.
(eg Google charts API)
This works 99.99% of the time.
However, I just got a client that for some reasons got his company restricting access to google.com.
As a consequence of this my website simply threw a JavaScript error.
Now I know how to handle that, and I can check if window.Google exists.
but my question is
"what's the standard way to deal with this? "
In other words if you embed 3rd party JavaScript how best do you deal with their JS not available?
NOTE: VERY IMPORTANT
You can not host the chart code locally or on an intranet.
SEE FAQ from Google: https://developers.google.com/chart/interactive/faq#localdownload
Can I download and host the chart code locally, or on an intranet?
Sorry; our terms of service do not allow you to download and save or
host the Google.load or Google.visualization code.
There is no real alternative. Due to Google's terms of service you cannot use Google API without access to google.com.
Check the connection to Google and iform user that function is not available
Develop your own or use non-google api. Still you can use Google if available
The solution is that your client's company review their content filtering policies. Google are quite clear in their previous answer concerning offline access:
…your computer must have live access to http://www.google.com/jsapi in order to use charts.
You are using a third-party solution according to their terms and conditions, which naturally imposes limits on how that solution may be used by your clients. You need to stand firm or find a more liberally-licensed solution. (At any rate, you are more likely to succeed at convincing your client's IT department than petitioning Google to change their TOS.)
For the more general case of third party JS APIs that may not load but for which you are allowed to keep a local copy on your server, see this question.
You can try it like this:
Instead of using the direct link to the Google libraries you want to use, use a link which points to your server:
<script type="text/javascript" src="https://www.myserver.com/jsapi"></script>
When your server gets an incoming request to this URL, your server now makes a request to Google to get the API and sends the response to the client.
That means you do not install the API anywhere locally or on a server and always get the most actual version directly from Google. People also do not need access to Google (as in the company you mentioned) and therefore can use your service.
Use Firebug or the Chrome Dev Tools to inspect your HTML source once the charts scripts are loaded. Access the scripts in your browser and save them locally, then serve them from your own server. This isn't recommended, of course, but if you don't have any other choice...
For example, checking the code of one of the pages I use it on, the core script for the Google Charts library is located at:
https://www.google.com/uds/api/visualization/1.0/3d781368978b51b3ca00a01566dccf40/format+en,default,corechart.I.js
Use the javascript window.onload to check whether the api has loaded or not, if no then load it from your server.
You already know how to check whether or not your library has been loaded (checking the object), if it fails, than what you can do within giving constrains:
Keep checking the object with timer and trying to download library, displaying message for a user
In case first one fails, you have two ways again:
Stopping your application and displaying an error: "Application error... try later"
Or downloading different library as a fallback
Are you progressively enhancing or gracefully degrading the page? If so, what do you display to users without JavaScript for this chart? A table? A list? This is what you should leave in the page and only start changing it once google's JS is available. Either that, or find an alternative library like raphaeljs that lets you keep all your code within your project.
IF (BIG IF) you are not worried about the interactivity the Google Charts and want to display them to the user just to see - maybe add your own javascript to it but not depend on the Google Javascript at all, this can turn the google charts into a image that you can display to the user.
Also this requires access to install a command line tool on the server.
http://code.google.com/p/wkhtmltopdf/ is a command line tool that will generate an image from an html page. If you build a simple page that only shows the chart you want and point the wkhtmltoimage tool at the local html file it will load the Google Charts javascript and generate the chart then generate an image out of the results.
YES I understand this is VERY kludgy and is adding a big tool for a small problem but with the browser restriction and the Google Terms of Service this will solve most of the problem.
You can try going straight to google and if it fails (if google is restricted) you can bounce the request off of your server which forwards the request using CURL to google. If that doesn't work then Google is most likely down. This should cover the issue that you described in your question, but there isn't really a fix for if google itself actually goes down. It should, however, give your application access around domain restrictions because the request will be routed to your server rather than straight to google. I use this architecture for all requests so that I don't have ajax requests routed to random servers. It allows me to control what interacts with my front end using my backend. There are other benefits to this, especially if you are using something like AngularJS with NodeJS because you can decouple a lot of your third party libraries. This however, is beyond the scope of your question!
Basically, it works like this (pseudo code):
If(!Browser->Google->Browser){
return Browser->MyServer->Google->MyServer->Browser;
}
An answer has been accepted already, but still I would like to leave an additional aspect elaborating on the comment I made above ....
It has been accepted that the Google Server is the only place from where the API can be loaded. We don't know whether the client's IT manager will re-think their content policy, they might have good reasons for that.
Given a non-100% availability of all the components along the path between a user browser and the Google API, sooner or later a user will end up in an error situation; statistically this is unavoidable.
What is not acceptable (and avoidable) for a user is to receive an "unspecific" JS error making him/her believe there's a bug on the page. So my solution would be to trap the failure loading the Google API and display a message "Third party components temporarily unavailable - Please try later".
This will demonstrate to the user that
we know what's going on
there's nothing we can do about it now
but it's not totally unexpected and still somehow under control
just wondering if anyone knows what technologies are used by the tracking software?
Edit: I meant client-side. How is data sent to the Google API? Long polling? Streaming? =)
Thanks guys!
The tracking is at function-calling time. No long polling or streaming.
Every time a Google Analytics function is triggered (whether upon page load, or when a page event like onClick is triggered), it executes a function within the ga.js file. Commonly, that could be trackPageView, but there are many others, as you can see here: http://code.google.com/apis/analytics/docs/tracking/asyncUsageGuide.html .
That function mainly does two things:
It also places numerous first-party
cookies on the end-users computer, if
allowed, in order to enable tracking
beyond a single page.
It requests a blank image file called
_utm.gif from google-analytics.com, with a long query string attached it. That query string contains all of the details that Google Analytics tracks.
Google's servers record that gif request on their logs, which are then processed on Google's side; the lag for the data appearing in GA can be anywhere between 3 and 24 hours, depending on what is being tracked or computed.
That query string contains various parameters that together are put together by Google to create an accurate picture of the visitor's journey
Here's a reference as to what parameters Google Analytics collects:
http://code.google.com/apis/analytics/docs/tracking/gaTrackingTroubleshooting.html#gifParameters
The _utm.gif call for loading talkingpointsmemo.com, for example, looks like this:
http://www.google-analytics.com/__utm.gif?utmwv=4.7.2&utmn=1687340155&utmhn=www.talkingpointsmemo.com&utmcs=UTF-8&utmsr=1920x1080&utmsc=24-bit&utmul=en-us&utmje=1&utmfl=10.1%20r53&utmdt=Talking%20Points%20Memo%20%7C%20Breaking%20News%20and%20Analysis&utmhid=1157392983&utmr=-&utmp=%2F&utmac=UA-927537-1&utmcc=__utma%3D147706162.633472310.1273842954.1279564084.1279662542.44%3B%2B__utmz%3D147706162.1279564457.43.23.utmcsr%3Dgoogle%7Cutmccn%3D(organic)%7Cutmcmd%3Dorganic%7Cutmctr%3Degypt%2520IGLHRC%3B
On any given page load, there could be multiple _utm.gif requests made, 1 for each type of request made.
On client side: JavaScript and a tracking image.
On server side: I don't know.
Well, they have this honking big database called DataStore, of their own design. It's not just there for Google App Engine, they use it internally too. They even store their Web crawling results in it.
Essentially, most Google data ends up in (an internal partition of) DataStore.
EDIT: Oops, I just realized this doesn't answer the question. I thought I'd read "where to they store the data?" I'll leave it up, though, in case anyone is interested.