We Keep Coding

Find and replace based on criteria with Google Apps Scripts Code

I have two Google Sheets housed in the same Spreadsheet object, for reference I'll refer to them as Data and Change-Log.
Data Sheet
ID Country Attribute Value
1 USA X 100
2 RUS X 77
3 MEX Y 32
4 GER Z 111
...
Change-Log Sheet
Country Attribute Value
USA X 84
GER Z 97
Updated Data Sheet
ID Country Attribute Value
1 USA X 84
2 RUS X 77
3 MEX Y 32
4 GER Z 97
...
Currently I am pulling into the Data sheet via an API, which is cleared and updated monthly.
Ideally I would like to write some sort of helper function that can query the Data sheet for entries shared between the two sheets and overwrite values in the Data sheet with the corresponding value from the Change-Log.
In the example above, I would want to query where the Country and Indicator variables are the same, then compare the Value variable and give preference to the Change-Log.Value entry by overwriting this value into the Data sheet where appropriate.

We have a very little information on how your sheets look and what are you trying to do.
You can do something like this:
function test() {
const ss = SpreadsheetApp.getActiveSpreadsheet();
const dataSheet = ss.getSheetByName("Data");
const dataRange = dataSheet.getRange(2, 1, dataSheet.getLastRow() - 1, 4);
const dataValues = dataRange.getValues(); // Get at once, faster execution
const changeLogSheet = ss.getSheetByName("Change-Log");
const replacements = changeLogSheet.getRange(2, 1, changeLogSheet.getLastRow() - 1, 3).getValues(); // Faster
const newValues = [];
dataValues.forEach(row => {
const replacementValue = getReplacementValue_(row[1], row[2], replacements);
newValues.push([row[0], row[1], row[2], (replacementValue ? replacementValue : row[3])]);
});
dataRange.setValues(newValues); // Faster
}
function getReplacementValue_(country, attribute, replacements) {
const matchingRows = replacements.filter(row => row[0] == country && row[1] == attribute);
return matchingRows.length > 0 ? matchingRows[0][2] : null;
}
I'm not sure if you want to overwrite all the data each time, or start at some row.
Note 1: Stack Overflow community will be able to help you more if your question is in the form of "here is my goal and code so far, these are the errors and problems". If you ask "how to do something" we won't be able to help you that much.
Note 2: If some answer helps you, upvote/accept it. If not, edit your question to provide more information. In this case it would be good if you can make a copy of your sheet (if data is sensitive then replace with mock data) and share it with anyone on the internet and post a link in your question. Something like "Data" sheet, "Change Log" sheet and "Desired Data" sheet.

Advice on how to optimise Google AppScript code [duplicate]

I've just written my first google apps scripts, ported from VBA, which formats a column of customer order information (thanks to you all of your direction).
Description:
The code identifies state codes by their - prefix, then combines the following first name with a last name (if it exists). It then writes "Order complete" where the last name would have been. Finally, it inserts a necessary blank cell if there is no gap between the orders (see image below).
Problem:
The issue is processing time. It cannot handle longer columns of data. I am warned that
Method Range.getValue is heavily used by the script.
Existing Optimizations:
Per the responses to this question, I've tried to keep as many variables outside the loop as possible, and also improved my if statements. #MuhammadGelbana suggests calling the Range.getValue method just once and moving around with its value...but I don't understand how this would/could work.
Code:
function format() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var s = ss.getActiveSheet();
var lastRow = s.getRange("A:A").getLastRow();
var row, range1, cellValue, dash, offset1, offset2, offset3;
//loop through all cells in column A
for (row = 0; row < lastRow; row++) {
range1 = s.getRange(row + 1, 1);
//if cell substring is number, skip it
//because substring cannot process numbers
cellValue = range1.getValue();
if (typeof cellValue === 'number') {continue;};
dash = cellValue.substring(0, 1);
offset1 = range1.offset(1, 0).getValue();
offset2 = range1.offset(2, 0).getValue();
offset3 = range1.offset(3, 0).getValue();
//if -, then merge offset cells 1 and 2
//and enter "Order complete" in offset cell 2.
if (dash === "-") {
range1.offset(1, 0).setValue(offset1 + " " + offset2);
//Translate
range1.offset(2, 0).setValue("Order complete");
};
//The real slow part...
//if - and offset 3 is not blank, then INSERT CELL
if (dash === "-" && offset3) {
//select from three rows down to last
//move selection one more row down (down 4 rows total)
s.getRange(row + 1, 1, lastRow).offset(3, 0).moveTo(range1.offset(4, 0));
};
};
}
Formatting Update:
For guidance on formatting the output with font or background colors, check this follow-up question here. Hopefully you can benefit from the advice these pros gave me :)

Issue:
Usage of .getValue() and .setValue() in a loop resulting in increased processing time.
Documentation excerpts:
Minimize calls to services:
Anything you can accomplish within Google Apps Script itself will be much faster than making calls that need to fetch data from Google's servers or an external server, such as requests to Spreadsheets, Docs, Sites, Translate, UrlFetch, and so on.
Look ahead caching:
Google Apps Script already has some built-in optimization, such as using look-ahead caching to retrieve what a script is likely to get and write caching to save what is likely to be set.
Minimize "number" of read/writes:
You can write scripts to take maximum advantage of the built-in caching, by minimizing the number of reads and writes.
Avoid alternating read/write:
Alternating read and write commands is slow
Use arrays:
To speed up a script, read all data into an array with one command, perform any operations on the data in the array, and write the data out with one command.
Slow script example:
/**
* Really Slow script example
* Get values from A1:D2
* Set values to A3:D4
*/
function slowScriptLikeVBA(){
const ss = SpreadsheetApp.getActive();
const sh = ss.getActiveSheet();
//get A1:D2 and set it 2 rows down
for(var row = 1; row <= 2; row++){
for(var col = 1; col <= 4; col++){
var sourceCellRange = sh.getRange(row, col, 1, 1);
var targetCellRange = sh.getRange(row + 2, col, 1, 1);
var sourceCellValue = sourceCellRange.getValue();//1 read call per loop
targetCellRange.setValue(sourceCellValue);//1 write call per loop
}
}
}
Notice that two calls are made per loop(Spreadsheet ss, Sheet sh and range calls are excluded. Only including the expensive get/set value calls). There are two loops; 8 read calls and 8 write calls are made in this example for a simple copy paste of 2x4 array.
In addition, Notice that read and write calls alternated making "look-ahead" caching ineffective.
Total calls to services: 16
Time taken: ~5+ seconds
Fast script example:
/**
* Fast script example
* Get values from A1:D2
* Set values to A3:D4
*/
function fastScript(){
const ss = SpreadsheetApp.getActive();
const sh = ss.getActiveSheet();
//get A1:D2 and set it 2 rows down
var sourceRange = sh.getRange("A1:D2");
var targetRange = sh.getRange("A3:D4");
var sourceValues = sourceRange.getValues();//1 read call in total
//modify `sourceValues` if needed
//sourceValues looks like this two dimensional array:
//[//outer array containing rows array
// ["A1","B1","C1",D1], //row1(inner) array containing column element values
// ["A2","B2","C2",D2],
//]
//#see https://stackoverflow.com/questions/63720612
targetRange.setValues(sourceValues);//1 write call in total
}
Total calls to services: 2
Time taken: ~0.2 seconds
References:
Best practices
What does the range method getValues() return and setValues() accept?

Using methods like .getValue() and .moveTo() can be very expensive on execution time. An alternative approach is to use a batch operation where you get all the column values and iterate across the data reshaping as required before writing to the sheet in one call. When you run your script you may have noticed the following warning:
The script uses a method which is considered expensive. Each
invocation generates a time consuming call to a remote server. That
may have critical impact on the execution time of the script,
especially on large data. If performance is an issue for the script,
you should consider using another method, e.g. Range.getValues().
Using .getValues() and .setValues() your script can be rewritten as:
function format() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var s = ss.getActiveSheet();
var lastRow = s.getLastRow(); // more efficient way to get last row
var row;
var data = s.getRange("A:A").getValues(); // gets a [][] of all values in the column
var output = []; // we are going to build a [][] to output result
//loop through all cells in column A
for (row = 0; row < lastRow; row++) {
var cellValue = data[row][0];
var dash = false;
if (typeof cellValue === 'string') {
dash = cellValue.substring(0, 1);
} else { // if a number copy to our output array
output.push([cellValue]);
}
// if a dash
if (dash === "-") {
var name = (data[(row+1)][0]+" "+data[(row+2)][0]).trim(); // build name
output.push([cellValue]); // add row -state
output.push([name]); // add row name
output.push(["Order complete"]); // row order complete
output.push([""]); // add blank row
row++; // jump an extra row to speed things up
}
}
s.clear(); // clear all existing data on sheet
// if you need other data in sheet then could
// s.deleteColumn(1);
// s.insertColumns(1);
// set the values we've made in our output [][] array
s.getRange(1, 1, output.length).setValues(output);
}
Testing your script with 20 rows of data revealed it took 4.415 seconds to execute, the above code completes in 0.019 seconds

Check if a column header title matches a string and if so then return the column index

I am trying to write a script for google sheets which returns the date in the next cell when the user enters 'y' in the current cell. I have a script which does this already, but the problem with my script is that the columns which it is evaluating is based on the column index, which means if our data set ever grows then these columns always have to stay in the same index which is creating a lot of organizational issues.
My question is..
Is it possible to look for the column header title rather than the column index in my code, and if so, what changes would I need to make?
function onEdit(e) {
if ([19].indexOf(e.range.columnStart) == -1 || ['y', 'Y'].indexOf(e.value) == -1) return;
e.range.offset(0, 1)
.setValue(Utilities.formatDate(new Date(), "GMT-5", "MM-dd-yyyy"))
}
This code currently looks at column index 19 and when either 'y' or 'Y' is entered into a cell in column index 19 it then outputs the date in the next cell in column 20.
How can I change the code to look for where the column header = 'Replied?' rather than index?

Goal:
If the following criteria is met:
Value is written into column 19 (S).
Header of column 19 (S) is 'Replied?'.
Value written is either 'Y' or 'y'.
Then write a date into the adjacent cell.
Code:
function onEdit(e) {
var sh = e.source.getActiveSheet();
var row = e.range.getRow();
var col = e.range.getColumn();
var value = e.value.toUpperCase();
var header = sh.getRange(1, col).getValue();
if (col === 19 && value === 'Y' && header === 'Replied?') {
sh.getRange(row, 20).setValue(Utilities.formatDate(new Date(), "GMT-5", "MM-dd-yyyy"))
}
}
Explanation:
I've based everything on the event objects passed to your onEdit trigger. For var value I have used toUpperCase() so that we don't have to check for either 'Y' OR 'y', only 'Y' alone. Also, instead of using range.offset I have just specified column 20 specifically in the getRange().setValue().
References:
Event Objects
String.toUpperCase()

One possible way to do this is to name the column/ cell in google sheets. See this website on how to.
Basically:
Open a spreadsheet in Google Sheets.
Select the cells you want to name.
Click Data and then Named ranges. A menu will open on the right.
Type the range name you want.
To change the range, click Spreadsheet Grid.
Select a range in the spreadsheet or type the new range into the text box, then - click Ok.
Click Done.
You can then refer to that named cell in google scripts by creating a custom function
function myGetRangeByName(n) { // just a wrapper
return SpreadsheetApp.getActiveSpreadsheet().getRangeByName(n).getA1Notation();
}
Then, in a cell on the spreadsheet:
myGetRangeByName("Names")

I'd do this.
function onEdit(e) {
var editedColumn = e.range.columnStart;
var sh = SpreadsheetApp.getActiveSpreadsheet();
var ss = sh.getSheetByName("This");//you only want onedits to the specific page
var data = ss.getDataRange().getValues();
var header = data[0][editedColumn];
if (header != "Replied") return;
if(e.value.toLowerCase() == "y"){
e.range.offset(0, 1)
.setValue(Utilities.formatDate(new Date(), "GMT-5", "MM-dd-yyyy"));}
}
You could also consider using a checkbox, that might be faster for your users.

Hide all but today's columns on four sheets (Google Spreadsheets script)

Goal: I'm trying to create a behavior tracker for four classes in Google Spreadsheets. The tracker has nine sheets: Class7A, Class7B, Class8A, Class8B, and Mon-Fri summary sheets. The goal was for each ClassXX sheet to have behavior tracking information for an entire week, but for the default view to show only the current day's information.
Attempts: During initial workup (with only the Class7A sheet created), I got this to work using a modification of the script found here (Thank you Jacob Jan Tuinstra!): Optimize Google Script for Hiding Columns
I modified it to check the value in the third row of each column (which held a 1 for Monday, 2 for Tuesday, etc), and if it did not match the numerical equivalent for the day of the week (var d = new Date(); var n = d.getDay();), then it would hide that column. This process was somewhat slow - I'm assuming because of the iterating through each column - but it worked.
Quite excited, I went ahead and added the rest of the sheets, and tried again - but the code as written, seems to affect only the current sheet. I tried modifying it by replacing var sheet = ss.getSheets()[0]; with for script that iterated through the columns, until i>4 (I've since lost that piece of code), with no luck.
Deciding to go back and try adapting the original version of the script to instead explicitly run multiple times for each named sheet, I found the that script no longer seems to work at all. I get various version of "cannot find XX function in sheet" or "cannot find XX function in Range."
Source: A shared version (with student info scrubbed) can be found here: https://docs.google.com/spreadsheets/d/1OMq4a4_Gh_xyNk_IRy-mwJn5Hq36RXmdAzTzx7dGii0/edit?usp=sharing (editing is on).
Stretch Goal: Ultimately, I need to get this to reliably show only the current day's columns (either through preset ranges (same for each sheet), or the 1-5 values), and I need it to do so for all four ClassXX sheets, but not the summary pages (and preferably more quickly than the iterations). If necessary, I can remove the summary pages and set them up externally, but that's not my first preference. I would deeply appreciate any help with this; so far my attempts have seemed to only take me backwards.
Thanks!
Current code:
function onOpen() {
// get active spreadsheet
var ss = SpreadsheetApp.getActiveSpreadsheet();
// create menu
var menu = [
{name: "Show Today Only", functionName: "hideColumn"},
{name: "Show All Days", functionName: "showColumn"},
{name: "Clear Week - WARNING will delete all data", functionName: "clearWeek"}
];
// add to menu
ss.addMenu("Show Days", menu);
}
var d = new Date();
var n = d.getDay();
function hideColumn() {
// get active spreadsheet
var ss = SpreadsheetApp.getActiveSpreadsheet();
// get first sheet
var sheet = ss.getSheets()[0];
// get data
var data = sheet.getDataRange();
// get number of columns
var lastCol = data.getLastColumn()+1;
Logger.log(lastCol);
// itterate through columns
for(var i=1; i<lastCol; i++) {
if(data.getCell(2, i).getValue() != n) {
sheet.hideColumns(i);
}
}
}
function showColumn() {
// get active spreadsheet
var ss = SpreadsheetApp.getActiveSpreadsheet();
// get first sheet
var sheet = ss.getSheets()[0];
// get data
var data = sheet.getDataRange();
// get number of columns
var lastCol = data.getLastColumn();
// show all columns
sheet.showColumns(1, lastCol);
}

I cannot recreate the problem of the script not working at all, it's working fine for Class7A so that part is working fine.
So let's look at the two other problems:
Applying this to all Sheets
Speeding up the script
First let's create some globals we use in both functions
var d = new Date();
var n = d.getDay();
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheetNames = ss.getSheets().map(function(sheet) {return sheet.getName();});
var classSheets = sheetNames.filter(function(sheetName) {return sheetName.match("Class")});
Now we can iterate over classSheets and get the sheet by name and hide columns in each.
However hiding each individual column is very slow.
The sheet is built very structured, every week has 12 columns (except for friday which doesn't have the grey bar), so we can just calculate the ranges we want to hide.
function hideColumn() {
classSheets.map(function(sheetName){
var sheet = ss.getSheetByName(sheetName);
if (n == 1) {
// Hide everything after the first three columns + Monday
sheet.hideColumns(3 + 11, 12 * 4);
} else if (n == 5) {
// Hide everything to the left except the leftmost three columns
sheet.hideColumns(3, 4 * 12);
} else {
// Hide everything left of the current day
sheet.hideColumns(3, (n - 1) * 12);
// Hide everything after the current day
sheet.hideColumns(3 + n * 12, (5 - n) * 12 - 1);
}
});
}
Lastly we can shorten showColumn
function showColumn() {
classSheets.map(function(sheetName){
var sheet = ss.getSheetByName(sheetName);
var lastCol = sheet.getLastColumn();
sheet.showColumns(1, lastCol);
});
}

Google Spreadsheet App Script will not wait for results to load

I have a small script and what I'm trying to do is to write one value from 'Sheet 1' to 'Sheet 2'. Wait for the results to load and compare the cells to see if it is above 10% or not. I have some =importhtml functions in the spreadsheet and it takes along time to load. I've tried sleep, utilities sleep, and flush. None have been working, maybe because I might be putting it in the wrong area..
function compareCells() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var listSheet = ss.getSheetByName('Stocks');
var dataSheet = ss.getSheetByName('Summary');
var listSheetLastRow = listSheet.getLastRow();
var currRow = 1;
for (i = 1; i <= listSheetLastRow; i++) {
if (listSheet.getRange(1, 3).getValue() == 'asd') {
var ticker = listSheet.getRange(currRow, 1).getValue();
dataSheet.getRange(5, 4).setValue(ticker);
var value1 = dataSheet.getRange(15, 4).getValue();
var value2 = dataSheet.getRange(22, 4).getValue();
SpreadsheetApp.flush();
if (value1 > 0.10 && value2 > 0.10) {
listSheet.getRange(currRow, 8).setValue('True');
listSheet.getRange(currRow, 9).setValue(value1);
listSheet.getRange(currRow, 10).setValue(value2);
} else {
listSheet.getRange(currRow, 8).setValue('False');
}
} else {
Browser.msgBox('Script aborted');
return null;
}
currRow++;
}
}

If it is not important that you use the =IMPORTHTML() function in your sheet, the easiest way to do this will be to use UrlFetchApp within Apps Script. Getting the data this way will cause your script to block until the HTML response is returned. You can also create a time-based trigger so your data is always fresh, and the user will not have to wait for the URL fetch when looking at your sheet.
Once you get the HTML response, you can do all of the same processing you'd do in Sheet1 within your script. If that won't work because you have complex processing in Sheet1, you can:
use UrlFetchpApp.fetch('http://sample.com/data.html') to retrieve your data
write the data to Sheet1
call SpreadsheetApp.flush() to force the write and whatever subsequent processing
proceed as per your example above
By handling these steps sequentially in your script you guarantee that your later steps don't happen before the data is present.

I had a similar problem but came up with a solution which uses a while loop which forces the script to wait until at least 1 extra column or 1 extra row has been added. So for this to work the formula needs to add data to at least one extra cell other than the one containing the formula, and it needs to extend the sheet's data range (number of rows or columns), for example by adding the formula to the end of the sheet, which looks like what you are doing. Every 0.5 seconds for 10 seconds it checks if extra cells have been added.
dataSheet.getRange(5, 4).setValue(ticker);
var wait = 0;
var timebetween = 500;
var timeout = 10000;
var lastRow = dataSheet.getLastRow();
var lastColumn = dataSheet.getLastColumn();
while (dataSheet.getLastColumn() <= lastColumn && dataSheet.getLastRow() <= lastRow){
Utilities.sleep(timebetween);
wait += timebetween;
if (wait >= timeout){
Logger.log('ERROR: Source data for ' + ticker + ' still empty after ' + timeout.toString() + ' seconds.');
throw new Error('Source data for ' + ticker + ' still empty after ' + timeout.toString() + ' seconds.');
}
}

In case if you are getting these two values (
var value1 = dataSheet.getRange(15, 4).getValue();
var value2 = dataSheet.getRange(22, 4).getValue();
) after the =importhtml call, you have to add sleep function before these two lines of code.
You also can have a loop until you get some values into the range from =importhtml call and add some sleep in the loop. Also note that as of April 2014 the limitation of script runtime is 6 minutes.
I also found this link which might be helpful.
Hope that helps!