We Keep Coding

How to create Custom Dimension & get Unique PageView with GA4?

We're not sure if this is the right place for us to obtain the answers of such. We've contacted the Firebase Support Team but did not get the answers that we want except a few links to the online documentation that we've mostly been through before. After further clarifying our requests, we've not been receiving any response from them for 5 days; therefore we might as well try our luck here.
1. How to create a Custom Dimension in GA4?
As we understand, GA4 are all events now, including the Hit in Universal Analytics (UA), but how can we map from our UA custom dimensions to the GA4 model as shown below?
When we tried to create the AccCode custom dimension in GA4, we have no idea what to enter under the Event parameter dropdown list as it also cannot be dropped down whatsoever:
May I know how can we proceed from here and what should we enter for the Event parameter value?
2. How to get Unique PageView (UPV) in Firebase GA4 API?
In UA or GA v3, this is how we get our Page View and Unique PageView:
return jwt.authorize()
.then((response) => {
return google.analytics('v3').data.ga.get({
'auth': jwt,
'ids': 'ga:' + gaConfig.ViewerID,
'start-date': validatedDateRange.strStartDate,
'end-date': validatedDateRange.strEndDate,
'metrics': 'ga:pageviews,ga:uniquepageviews',
"dimensions": gaConfig.AccCodeDimension,
'filters': ${gaConfig.PageUrlDimension}!#googleusercontent.com;${gaConfig.PageUrlDimension}!#blogspot.com${!accCode ? "" : ";" + gaConfig.AccCodeDimension + "==" + accCode}`,
'sort': `${gaConfig.AccCodeDimension},-ga:pageviews,-ga:uniquepageviews`
}).then((gaQuery) => {
// Do something here
});
Below is the sample code that we found from the Firebase GA4 documentation:
import firebase from "firebase/app";
import "firebase/analytics";
// Initialize Firebase
firebase.initializeApp(firebaseConfig);
// Initialize Analytics and get a reference to the service
const analytics = firebase.analytics();
analytics.logEvent('select_content', {
content_type: 'image',
content_id: 'P12453',
items: [{ name: 'Kittens' }]
});
But the above sample code seems to be far from giving us an idea on how to achieve the same result as did in GA v3. It's more like logging of event data, potentially for our custom dimensions as what we did in the UA's tracking code. Examples for data pulling don't seem to be available in the documentation here. We need an equivalent example on how we can achieve the same result with Firebase GA4 API so that we can quickly migrate over to GA4 before July 2023.
We hope that someone here can help us to resolve the above two issues as quickly as possible because they involve changing the core engine of our app, which requires vast amount of development time and testing while the clock is ticking. Thank you so much in advance!

After so much of the studies and R&D, we realized that for what we're trying to achieve has nothing to do Firebase at all -- we can purely focus on GA and its latest API, which is still on Beta while some are on Alpha. But for the custom dimension creation, below is the answer:
Creating Custom Dimensions in GA4
As per the question described, the custom dimension creation process can be very confusing, especially to the UA users due to the change of data model and concepts. But what you need to know is that, you need to finalize your event parameters before mapping them over to the custom dimensions on GA console because the event parameter cannot be edited once the custom dimension is created on GA console:
So what you need to do is to extend your existing UA tracking code as shown below before creating your custom dimensions on GA console:
gtag('event','page_view', { // the page_view event is for web
"acc_code": "{{{AccCode}}}", // acc_code is your event parameter
"acc_name": "{{{AccName}}}", // This is another event parameter
"viewer_id": "{{{ViewerID}}}",
"viewer_title": "{{{ViewerTitle}}}",
"viewer_url": "{{{gaUrl}}}"
});
gtag('config', 'G-XXXXXXXX'); // This is your GA4 MEASUREMENT_ID
Data Query in GA4
For data query in GA4 that is equivalent to the given example in the question, please refer to the GA4 Data API here.
What about the Unique Pageview metric?
According to the GA documentation here, looks like Unique Pageview is no longer available in GA4:
I'm not sure if this is still subject to change, but you may need to write your own code, perhaps using sessionStorage or session cookies to track your own unique pageviews per user session for every page viewed.

Google Analytics: User does not have sufficient permissions for this profile

I am unable to access the Google Analytics API from rails. I keep getting insufficient permissions even though I have already set the service account.
I have created a service account:
Go to the project in console.developers.google.com/iam-admin/serviceaccounts
I go to Service Accounts > Create Service Accounts
Then select account and add a key. This will download a JSON file with the service account.
After having the service account and have the secret JSON file and the email. Then I go to Google Analytics:
Go to Admin > Select Property > Property User Management > Add User
Copy the Service Account email and select Read & Analyze
Once I have that, pretty much use the following code: https://gist.github.com/CoryFoy/9edf1e039e174c00c209e930a1720ce0 to get the reports.
But I keep getting forbidden: User does not have sufficient permissions for this profile. Even when I try to get the access_token and use it in javascript:
gapi.analytics.auth.authorize({
serverAuth: {
access_token: 'accessToken'
}
});
I get:
error:{
code: 403
errors: [
{
domain: "global"
message: "User does not have sufficient permissions for this profile."
reason: "insufficientPermissions"
}
]
}
Am I missing something? This used to work and I never had a problem with it, but for some reason it stopped working. Any guidance will be appreciate it.
Thank you!
UPDATE
UA (Universal Analytics) will stop processing data on July 1st 2023. It means will stop support for it, so you will need to update to Analytics 4.
Articles by Google
Universal Analytics will be going away
[GA4] Introducing the next generation of Analytics, Google Analytics 4

This has something to do with how you create an analytics property. Since they remove the views from analytics 4, now you need to create a universal analytics property.
In account click on "Create Property"
Fill the fields and click on "Show advance options"
Activate "Create a Universal Analytics property"
Add the website url in the field and then make sure "Create both a
Google Analytics 4 ...." and "Enable enhanced measure interactions on
your sites...." are marked.
Then click next and that is it.
The code stays the same as this is an analytics change.
UPDATE
UA (Universal Analytics) will stop processing data on July 1st 2023. It means will stop support for it, so you will need to update to Analytics 4.
Articles by Google
Universal Analytics will be going away
[GA4] Introducing the next generation of Analytics, Google Analytics 4

Google places AutoComplete widget is generating a new Session Key per every request

Recently, Google has updated some billing policies as part of that they've introduced SessionTokens for the Autocomplete requests.
As part of that, every request in the same session will be sent the same token in the request and it's automatic if we use the AutoComplete widgets provided by Google APIs.
But, I'm seeing a different token generated for every request in the same session. I'm I missing anything, please guide me. Here is the pluker for the sample, there I observed a different token generated for each keystroke.
var options = {
types: ['(cities)']
};
var searchBox = new google.maps.places.Autocomplete(input, options);

My Code for the page
<div class="searchField">
<input id="searchLocation" name="searchLocation" type="text" placeholder="Enter an Address">
</div>
<!-- more code.... -->
<!-- footer -->
<script>
// initiate search area for autocomplete of places from Google Maps Street Addresses
function initAutocomplete() {
// define the search location box
var searchBox = $("#searchLocation")[0];
// initiate the autocomplete with the options on the search box
var autocomplete = new google.maps.places.Autocomplete(searchBox, options);
};
</script>
<script src="https://maps.googleapis.com/maps/api/js?libraries=places&region=ca&language=en&key= [YOUR_API_KEY]&callback=initAutocomplete" async defer>
</script>
Session Token Clarification
According to the Google Docs (all the way at the bottom of the page in a "warning text")...
https://developers.google.com/maps/documentation/javascript/places-autocomplete
Warning: Be sure to pass a unique session token for each new session.
Using the same token for more than one session will result in each
request being billed individually. Note that the Autocomplete Widget
handles sessions automatically (you don't need to do anything
additional). Now, lets dive into the basic Javascript example.
This may be a bit confusing, only because you are trying to get into the nomenclature of the google docs and what these things and possibly because you were doing it one way and suddenly your boss or client or someone else ask you to use Sessions/Session Tokens instead.
If you do new google.maps.places.Autocomplete() instead of AutocompleteServices or something along those lines, you are using the "Autocomplete Widget".
The autocomplete widget will handle the session / session tokens itself.
Prove it!
I trust Google and the Google Developer Documentation regarding their own products and how/what they do, but there was plenty of confusion on the web about this. So I wanted proof, besides getting a bill after a month. FYI - the billing in Google Clound Platform Console (https://console.cloud.google.com/google/maps-apis?pli=1) will show you this almost right away and will corroborate the stuff below and above.
Looking at the Network activity in Dev Tools, we see that there is a few calls to the Autocomplete Service.
I did not use the "AutocompleteService" function here, this is a basic example setup the same way I have listed above for the Callback with using the "Autocomplete Widget" via new google.maps.places.Autocomplete().
When you go to the page, you can see the call to library first, first screenshot of Dev Tools below.
Next I did a search, which worked. I started to type in a basic address of 1990, you can see 4 request made to the API after that. 1 request/call for each character I type in. Each of these request names begin with "AutocompleteServices.GetPredictions...", even though I DID NOT USE AutocompleteServices in my code. On the back-end, the "Autocomplete Widget" is using the "AutocompeleteServices" and doing all the UI/UX work for you as well as the functionality and the session and tokens. These are in the Dev tools screenshots #1-#3 below.
Then you see a 5th request call to "PlaceServces.GetPlaceDetails", which was when I selected from the dropdown. This is the dev tools screenshot #4 below.
Looking at the headers, for each of these requests, I see a few things.
The first thing, circled at the bottom is a "token". I know I just said and the Google Documentation just said that the "Autocomplete Widget" handles the session token, and a keen eye will see that this "token" value is different for every request. It is also not in 'version 4 UUID' format, which is recommended by Google, but that is a different topic. This "token" in the header, IS NOT the session token. This is buried in documentation that is 5 years old for a previous version of this on Google Docs, but this is not used anymore, and is auto-set for each request and is not the Session token.
SO, "where is the session token then?" you might ask.
The other items in the header that start with 1s, 2s and so on are different session variables that are passed. In this case, the session token is '20s' followed by the version 4 UUID auto created by the "Autocomplete Widget". A keen eye will also notice that for the 3 screenshots below they are the same. I have a fifth screenshot after the page refreshes that shows that the Session Token has changed. You can see in my code that I did not specify those things, but because I am using the "Autocomplete Widget" they are done for me.

I'm afraid token parameter that you can see in the request is not the places autocomplete session token. This parameter was there even before Google Maps platform changes. I don't know how Google manages handle autocomplete session tokens, probably they handle them server side.
The documentation states that autocomplete widget has automatic session tokens implemented.
Autocomplete (Web Service and JavaScript) and the JavaScript Autocomplete widget have been updated to use session-based billing.
Note: No code changes are required for the JavaScript Autocomplete widget, as the widget manages sessions for you automatically.
source: https://cloud.google.com/maps-platform/user-guide/pricing-changes/
To be sure I would suggest checking you billing reports in developer console.
https://console.cloud.google.com/billing/unbilledinvoice?project=YOUR_PROJECT_ID&authuser=1
In this report you will see which method was used for billing of places autocomplete requests. Have a look at rows where Product column is Places API and the Resource column will show which method was used for billing Autocomplete - Per Request or Autocomplete - Per Session.
If you use Place autocomplete widget with automatic sessions, but this report shows only usage of Autocomplete - Per Request reach out to Google maps technical support via https://console.cloud.google.com/google/maps-apis/support

If you use google.maps.places.Autocomplete(input, options); it will add session token automatically.
In your pluker example there is no Autocomplete but new google.maps.places.SearchBox

Programmatically get Gmail storage usage

How can I get the storage usage of Gmail programmatically? This value appears in two places, but I cannot find an API for it.
The main goal is to automatically log Gmail storage usage to a Google Sheet and graph the storage usage trend.
What I've tried:
DriveApp.getStorageUsed() does not include Gmail usage.
Reports API (gmail_used_quota_in_mb) is not available for consumer Gmail accounts (only G Suite)
Scraping www.google.com/settings/u/0/storage is not possible because a log-in is required, even when spoofing cookies (probably detects a new device/IP address).

Not the programmatic answer I am looking for, but this is my manual work-around until I discover an API method:
Examine the source code for google.com/settings/u/0/storage
Find exact bytes of usage embedded in JSON:
...
,[["Drive",["24004828","0.02 GB"]
]
,["Gmail",["4859735562","4.52 GB"]
]
,["Google Photos",["0","0 GB"]
...
Log that (4,859,735,562) into a Google Sheet, along with the current date/time
Using precise measurements of bytes and seconds, it should only take a few measurements to get an accurate trend. (And I don't need to scale this for multiple accounts.)

You can use Google Drive V2 API.
About: get returns quotaBytesByService[] where you can filter service by name (e.g. GMAIL, in your case)
If you want to use the API, you have to enable the advance API option and use it.
Note that the version 3 does not give you the service breakdown.

Display Google Analytics data on my web site?

I'm trying to figure out a way to display data collected from Google Analytics on my web site. I'm using NopCommerce, and I want to display this information/statistics in a view in the Admin Section.
There might be many ways to achieve this, and after searching the web I found some examples using JavaScript, but I couldn't find a good tutorial for this.
I have also looked into integrating Google Analytics with C#, and I found this example: http://biasecurities.com/2012/02/using-the-google-analytics-api-with-asp-net-mvc/#comment-1310
A demo project can be downloaded from GitHub here: https://github.com/jgeurts/Analytics-Example
However the demo project doesn't seem to work as the google URL (https://www.google.com/analytics/feeds/accounts/default) is no longer in use.
As I'm using a MVC application it would be preferable to make this happen by applying the Google Analytics logic within a Controller and displaying it in a view. Or something like that.
Google provides a query tool to experiment with here, so it shouldn't be to hard extracting data from Google Analytics and display the data on the website: https://ga-dev-tools.appspot.com/explorer/
Has anyone been able to successfully display Google Analytics data on their website?

In case some one else is having the same problem here's what I did and it pretty much answers the question.
1.
Here is the basic code for a API client that access data from Google Analytics via your Google Service Account. https://developers.google.com/api-client-library/dotnet/guide/aaa_oauth#service_account
In order to make this application work you need to have several things ready before you start coding.
*Google Analytics Account - once registered a "tracker" code is generated for you to put on each webpage you want to track. You may not see any statistics right away and it can take up to 24h before any statistics are shown in the Google Analytics Dashboard.
An OAuth Authorisation (API-Key) with CLIENT_ID, CLIENT SECRET and EMAIL ADRESS (This is not your normal email but a service account email that is created for you when you make an OAuth Authorisation).
console.developers.google.com/
A serverkey, can also be created here: console.developers.google.com/.
You can also create a browser key, haven't bothered with that though and don't know what it does.
Finally you need a certificate key. Your application will only be able to access your Google Analytics account by using the key and credentials. The key is an encrypted p.12 file. You can find the key in https://code.google.com/apis/console/.
Here is a guide for the key: http://www.pimcore.org/wiki/display/PIMCORE/Setup+Google+Analytics+Reporting+with+OAuth2+Service+Accounts+(since+1.4.6)
2.
Now that you have all keys and credentials you need it is time to start looking at the code I linked in "1". Here is the basic for it again: https://developers.google.com/api-client-library/dotnet/guide/aaa_oauth#service_account
Create a console application and implement the code above.
Note: your not making a "Google Plus service" so you have to change those parts for "AnalyticsService". Go to manage nuget and install packages:
Google Apis Core Library
Google Apis Client Library
Google Apis Auth Client Library
Google Apis Analytics.v3 Library
Google GData Client (This provides properties used to query data, metrics, demensions etc)
Google GData Extensions Library
Analytics
Might forgot something but here are the namespaces I use:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Security.Cryptography.X509Certificates;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Services;
using Google.Apis.Analytics.v3;
3
Finally, here's is some of my code. Note I'm creating a new Analytics as supposed to "new ServiceAccountCredentials" as in the code from Google. That's the main difference:
Retrieve data from Google Analytics API with .NET, multiple metrics?
With this I'm able to access and query data from Google Analytics account. The best part is that you don't have to log in to Google for this as the key and credentials gives you access to the account data directly.
I will migrate this code to MVC now I might make an update later on for how to implement this Analytics client in Mvc.

This document explains how to get Google Access tokens and use them to get Google Analytics data to be displayed in our websites.
Example: A live example is available in
https://newtonjoshua.com
Note: Use the same Gmail account for all the below steps.
STEP 1: Set Up Google Analytics
Follow the below steps to set up Google Analytics on your website
Sign in to your Analytics account.
Select the Admin tab.
Select an account from the drop-down menu in the ACCOUNT column.
Select a property from the drop-down menu in the PROPERTY column.
Under PROPERTY, click Tracking Info -> Tracking Code.
To collect data, you must copy and paste the Analytics tracking code
into the source code on every web page you wish to track.
Once you have the JavaScript tracking code snippet for your
property, copy the snippet exactly without editing it.
Paste your tracking code snippet (unaltered, in its entirety) before
the closing </head> tag on every web page on your site you wish to
track.
Once you have successfully installed Analytics tracking, it may take
up to 24 hours for data such as traffic referral information, user
characteristics, and browsing information to appear in your reports
Refer to:
https://support.google.com/analytics/answer/1008080?hl=en
https://analytics.google.com
STEP 2: Get Tokens
Google Project:
To create a Google Cloud Platform project, open the Google Developers Console (https://console.developers.google.com) and click Create Project.
Enable OAuth 2.0 API access:
Your application will need to access user data and contact other Google services on your behalf. Use OAuth 2.0 to grant your app API access.
To enable that, you need a client ID:
Open the Google API Console Credentials page (https://console.developers.google.com/apis/credentials).
From the project, drop-down and select your project.
Select Create credentials and choose OAuth client ID.
Under Application type, select Web application, enter a Name and
set the Restrictions by entering JavaScript origins, Redirect URIs to point the website where you are planning to display the data, and then click Create.
Make note of the OAuth 2.0 client_id and client_secret. You will need them to configure the UI.
Get Authorization code:
Enter in browser:
https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/analytics.readonly&response_type=code&client_id={{
client_id}}&redirect_uri={{redirect_uri }}
&approval_prompt=force&access_type=offline
You will get redirected to
{{redirect_uri }}?code=={{authorization_code}}#
Get Refresh Token:
Send a POST request, possibly via a REST console to
https://www.googleapis.com/oauth2/v3/token?code={{authorization_code}}
&client_id={{client_id}}&client_secret={{client_secret}}
&redirect_uri={{redirect_uri }}&grant_type=authorization_code
You will get a JSON response with
{"refresh_token": refresh_token}
You can use the refresh token to get access token to access to Google APIs.
Get the Access Token:
Send a POST request to,
https://www.googleapis.com/oauth2/v3/token?client_id={{client_id}}
&client_secret={{client_id}}
&grant_type=refresh_token&refresh_token={{refresh_token}}
You will get a JSON with access_token in the response.
{access_token: {{access_token}}}
Example:
var access_token = '';
function getAccessToken(){
$.post('https://www.googleapis.com/oauth2/v3/token', {
client_id: {{client_id}},
client_secret: {{client_secret}},
grant_type: 'refresh_token',
refresh_token: {{refresh_token}}
}, function (data, status) {
if (status === 'success') {
access_token = data.access_token;
// Do something eith the access_token
}
else {
console.error(status);
}
});
}
Check Token validity:
Send a POST request to,
https://www.googleapis.com/oauth2/v1/tokeninfo?access_token={{access_token}}
Example:
function checkValidity() {
$.post('https://www.googleapis.com/oauth2/v1/tokeninfo', {
access_token:{{access_token}}
}).done(function (data, status) {
if (status === 'success') {
console.debug(data.expires_in);
var check = false;
check = data.hasOwnProperty('expires_in');
if (check) {
// Token is valid
}
if (!check) {
getAccessToken();
}
}
else {
console.debug(status);
}
})
.fail(function (data) {
console.error(data);
getAccessToken();
});
}
Step 3: Fetch Data
Embed API:
The GA Embed API is a JavaScript library that allows you to easily create and embed your GA dashboard on your website in a matter of minutes.
Refer to https://developers.google.com/analytics/devguides/reporting/embed/v1/getting-started.
Query Explorer:
Visit Embed API Query Explorer and authorize
https://ga-dev-tools.appspot.com/query-explorer/
Select the view for which you want to fetch the data.
Select the required metrics and dimensions.
Example:
Get Country Data (I want to know the number of users accessing my website from each country).
To get that data, select the metrics as 'users' and the dimensions as 'country'.
Click on Run Query.
You will find the analytics data for the query displayed in a table.
Copy the API Query URI. And add access_token={{access_token}} to the URI.
Example:
https://www.googleapis.com/analytics/v3/data/ga?ids={{ids}}&start-date=2015-07-01&end-date=today&metrics=ga%3Ausers&dimensions=ga%3Acountry&access_token={{access_token}}
Send POST request to the URIs to get the data in your browser.
Example:
function gaGetCountry() {
$.get('https://www.googleapis.com/analytics/v3/data/ga?' +
'ids={{ids}}' +
'start-date=2015-07-01&' +
'end-date=today&' +
'metrics=ga%3Ausers&' +
'dimensions=ga%3Acountry&' +
'sort=ga%3Ausers&' +
'filters=ga%3Ausers%3E10&' +
'max-results=50' +
'&access_token=' + {{access_token}},
function (data, status) {
if (status === 'success') {
// Display the Data
drawRegionsMap(data.rows);
} else {
console.debug(status);
}
});
}
Step 4: Display Data
Now we have gathered the data. Finally we have to display them on our website.
"Display live data on your site" is the title of Google Charts. And that is what we are going to do.
Refer to https://developers.google.com/chart/.
The following example will draw a GeoChart in the div with id='countryChart'.
// Draw country chart
function drawRegionsMap(data) {
var head = data[0];
head[0] = 'Country';
head[1] = 'Users';
for (var i = 1; i < data.length; i++) {
var d = data[i];
d[1] = Number(d[1]);
}
var chartData = google.visualization.arrayToDataTable(data);
var options = {
title: 'My Website is viewed from,',
domain: '{{Country Code eg: IN for India}}',
tooltip: {
textStyle: {
color: 'navy'
},
showColorCode: true
},
legend: {
textStyle: {
color: 'navy',
fontSize: 12
}
},
colorAxis: {
colors: ['#00FFFF', '#0000FF']
}
};
var chart = new google.visualization.GeoChart(document.getElementById('countryChart'));
chart.draw(chartData, options);
}
Refer to https://newtonjoshua.com to view the above example in action.

I'd recommend to use the new Google APIs Client Library for .NET (currently in beta). Information about the Analytics API can be found here. Note that the Client Library for .NET (google-api-dotnet-client) supersedes the .NET library for the Google Data API (google-gdata).
Unfortunately, there is no sample code available yet from Google (see this issue) but this question on SO should help.
If you don't want to login every time you access Analytics data, you can use OAuth 2.0 authorization with offline access. You have to grant initial access to your web application, though. This requires you to login once but you can use a refresh token later on.

To date, the easiest solution is to create a report of your Google Analytics data in Google Data Studio (free, native connector to GA), and share that report to embed in an <iframe>
See details on https://support.google.com/datastudio/answer/7450249?hl=en
You get
flexible vizualisation
security and control on the data being shared
0 code nor maintenance
result is being cached for better performance

I spent a couple of days trawling the Internet to get some sample ASP.NET code with no luck. I used Koffe14's method for authentication, and also Linda Lawton's excellent ASP.NET analytics example.
I've posted the code on my website. It's not MVC, but it might help other people who need to get data from Google analytics into an ASP.NET web page using the v3 Google API.

Check out embeddedanalytics.com (disclaimer - I work with them).
This is a simple yet powerful solution geared for people that don't want deal with learning the GA API and then having to link it to visualization (e.g. charts/graphs).
Basically define your charts and embed a snippet of code where you want the chart to show. We also support mechanisms so that it is easy to integrate into a custom built CMS or other web portal.