I am looking to implement Google Analytics using analytics.js rather than ga.js. I would like to implement this into my hybrid HTML5 app which is served within an iOS shell. I am aware of the iOS SDK but I am trying to maintain one code base for different platforms.
I need to consider my app recording page impressions within a disconnected scenario, so I was going to store page impressions in localStorage and then when an internet connection is detected, then send each page impression individually to the Analytics service using the ga() function call.
In theory this works but I need to be able to send the page impression timestamp with each page rather than Analytics applying the timestamp when it hits their server.
Can this be achieved?
Thanks,
Rit
You are correct the iOs sdk lets you use something called Dispatching this allows you to send data that is up to 4 hours old.
This works directly with the Measurement Protocol where it is called Queue Time
Used to collect offline / latent hits. The value represents the time delta (in milliseconds) between when the hit being reported occurred and the time the hit was sent. The value must be greater than or equal to 0. Values greater than four hours may lead to hits not being processed.
As far as I can see analytics.js doesn't allow you to do this, but it is just a JavaScript library that sends data to the Measurement Protocol. It would probably be tricky getting that to work with Analytics.js which is probably why Google hasn't added it.
My advice to you is to store your data as you say locally then send it directly to Google though the Measurement Protocol. Just remember it has to be less then 4 hours old, you will have to hope that network returns by then.
Related
I am fairly new to javascript, I do know basics. I am looking to build my own (from scratch) java script library just like google analytics.js that will track user behavior on websites. Basically I'm looking to collect data like
Click through data
Dwell time
Page hits etc..
I spent lot of time trying to find website/tutorials to get me started on this but I keep ending up on google analytics.js or some private tools.
What I am looking for :
Is there any good starting point/resource/website which can help me build this js library
Are there reference for archetecture of end to end system including back-end?
Any open-source library that I can directly use?
Some things I already looked into
Chaoming build your own analytics tool
Splunk BYO analytics
At it's most basic, the architecture of such an application would only require a client, server, and database.
You can use basic javascript functions to record specific user actions on the frontend and then push them to your server. To identify your users you can set a cookie with a unique id. Then, everytime you send data to your server, you will get the specific user request as well so you can keep track of their actions. (Be careful of privacy laws first though).
For page hits, simply send a response to the server everytime someone opens your site - so call this function as soon as your Javascript loads. On the server, send a request to increment the appropriate value in your database.
For user dwell time, write a function that records the date when the user first hits your site and then count how long they stay there. Push your data to the server every so often and save updates to the user record by adding the new time spent to the current time spent. You could also watch for when a user is about to exit out of the site and then send the data all at once that way - although this method is more fragile.
For clicks and hovers, set up onclick and mouseover event handlers on your links or whatever elements you want to track. Then push the url of the link they clicked or whatever data you want - like "Clicked navbar after 200 seconds on site and after hovering over logo`.
If you want suggestions on specific technologies, then I suggest Node.js for your server side code and MongoDB for your database. There are many tutorials out there on how to use these technologies together. Look up javascript events for a list of the different things you can watch for on the frontend.
These are the building blocks you need. Now you just have to work on defining the data you want and using these technologies to get it.
I want to make a custom tracking system for web events. I have looked into multiple per-excsiting systems, but I want something terribly simple - yet very accurate.
I want to be able to track the following:
Page view even
Time on that page
or:
Video started playing event
Time of video watched
My first initial thought was to do a simple javascript reporting back to the server, but what happens if the user closes the window? How do I know they stopped viewing? And how can I get accurate measurements down to 1/10th of a second? So I thought of a websocket solution, as it know when a user has discounted. I ended up with Socket.io, but I want to make sure there is no better or smarter way to achieve this?
How would you approach his challenge? What is the smartest way to engineer this?
A Websocket connection which reports back to the server frequently was my first thought as well, but if you send 10 messages every second, even that might be too much for a websocket, especially when connectivity isn't top-notch.
Since the server doesn't require the information absolutely immediately, consider batching requests instead - save/update the information into Local Storage every 0.1 seconds, but don't send it to the server then - instead, every 30 or 60 seconds, or on pageload, take the current data in Local Storage and send it to the server, and clear Local Storage so that the next request a minute from now doesn't send duplicate data.
We use Google analytics with Google Tag manager for our ecommerce platform based to track the conversion, etc.,
In the thank you page, its used to track the order value and the revenues.
Consistently we see a difference of about 15 to 20 % difference between GA data and the data from the core platform.
Tried to find a pattern among the missed orders but couldn´t ascertain one easily. The GA recorded orders include devices like Desktop, tablet and mobile and we see different browsers too.
Need inputs to analyze this better.
Note: Thank you page is loaded by a redirect from the payment gateway system
15-20% difference is not good at all. Google says you should expect better than 95% accuracy, and to keep at it if you're not getting those numbers.
Note: The more "techy" crowd that your website has, the more folks you'll run into "Do not track" or with Ad-Blocking tech in their browser. Normally, you'd want to try to baseline that difference using device category filters to see if the gap is bigger for desktop (most phones/tablets don't use ad-blocking).
First, question. If the user lands on the thankyou page, and hits the refresh button, does it send another "conversion" to GA? IF so, you want to make sure that you build in logic that prevents duplicate conversions to be sent if the user was not making duplicate purchases. A browser refresh is not a purchase, don't record it as such.
Second, if the page takes forever to load, or you have users that have bad internet, then that could increase the difference. They might be closing the browser or exiting site before GA client has a chance to send the final conversion to the server. So how is the performance of your thankyou page?
Are you sure you're looking at the correct business data? I've been told GA numbers are off by the business before and it turned out they messed up their own query in the transaction system (and they had been doing so for years!). It is a long shot, but if you feel super confident about your GA measurement setup, then run it by the folks giving you the transaction numbers.
Finally, if you can't get the difference down then move to the Measurement Protocol server-side implementation of GA. You simply need to record the IP address of the user and their GA client id, and then construct an HTTPS GET request using the Measurement Protocol fields for a valid hit. Server side measurement is the most accurate way to do this, but requires code updates in the ecommerce platform itself.
In many devices (desktop and mobile) the clock is wrong, hence when try to determine an event time according to the client clock, that is results to inaccurate. Analytics services such as Google Analytics and Mixpanel recommend to use server timestamp to avoid that inaccuracy.
In case that the client clock is must (not server clock), what is the solution for that case?
For instance, in mobile SDK, a solution might be to execute an API call to time server and ignore the clock on the device.
Generally the analytics provider uses the timestamp from when the tracking request from the client was made. For example from the Mixpanel documentation:
If [the date] property is not included in your request, Mixpanel will use the time the event arrives at the server.
Offline use is more problematic, see this question for example - by default Google Analytics will still use the timestamp when the tracking event was received.
That questions also mentions a queue time parameter to record events that occurred in the past. The value is relative and indicates how many milliseconds in the past the event occurred. That means it doesn't matter whether the client time is wrong, as long as the it advances at the correct speed.
That leaves the issue of people changing their clocks between when an event occurred and when it is tracked. This is probably rare enough to not be much of an issue in practice. Google Analytics' queue parameter is also ignored if it's too far in the past, and presumably if it's in the future.
just wondering if anyone knows what technologies are used by the tracking software?
Edit: I meant client-side. How is data sent to the Google API? Long polling? Streaming? =)
Thanks guys!
The tracking is at function-calling time. No long polling or streaming.
Every time a Google Analytics function is triggered (whether upon page load, or when a page event like onClick is triggered), it executes a function within the ga.js file. Commonly, that could be trackPageView, but there are many others, as you can see here: http://code.google.com/apis/analytics/docs/tracking/asyncUsageGuide.html .
That function mainly does two things:
It also places numerous first-party
cookies on the end-users computer, if
allowed, in order to enable tracking
beyond a single page.
It requests a blank image file called
_utm.gif from google-analytics.com, with a long query string attached it. That query string contains all of the details that Google Analytics tracks.
Google's servers record that gif request on their logs, which are then processed on Google's side; the lag for the data appearing in GA can be anywhere between 3 and 24 hours, depending on what is being tracked or computed.
That query string contains various parameters that together are put together by Google to create an accurate picture of the visitor's journey
Here's a reference as to what parameters Google Analytics collects:
http://code.google.com/apis/analytics/docs/tracking/gaTrackingTroubleshooting.html#gifParameters
The _utm.gif call for loading talkingpointsmemo.com, for example, looks like this:
http://www.google-analytics.com/__utm.gif?utmwv=4.7.2&utmn=1687340155&utmhn=www.talkingpointsmemo.com&utmcs=UTF-8&utmsr=1920x1080&utmsc=24-bit&utmul=en-us&utmje=1&utmfl=10.1%20r53&utmdt=Talking%20Points%20Memo%20%7C%20Breaking%20News%20and%20Analysis&utmhid=1157392983&utmr=-&utmp=%2F&utmac=UA-927537-1&utmcc=__utma%3D147706162.633472310.1273842954.1279564084.1279662542.44%3B%2B__utmz%3D147706162.1279564457.43.23.utmcsr%3Dgoogle%7Cutmccn%3D(organic)%7Cutmcmd%3Dorganic%7Cutmctr%3Degypt%2520IGLHRC%3B
On any given page load, there could be multiple _utm.gif requests made, 1 for each type of request made.
On client side: JavaScript and a tracking image.
On server side: I don't know.
Well, they have this honking big database called DataStore, of their own design. It's not just there for Google App Engine, they use it internally too. They even store their Web crawling results in it.
Essentially, most Google data ends up in (an internal partition of) DataStore.
EDIT: Oops, I just realized this doesn't answer the question. I thought I'd read "where to they store the data?" I'll leave it up, though, in case anyone is interested.