I'm a bit of newbie at coding, especially Javascript/Google-script language. I've created the code below, and it works, but now that I've got a working code I'd like to see how I can optimize it. It seems to me that all of the getValue() calls are a major performance hit, and I've never really been good at optimizing loops. Anyone know a better way to accomplish the same as this code?
What it does: Checks each spreadsheet in one of my folders to see if it needs to have the rest of the script run. If true, it opens that sheet and counts the number of rows that have data, using that to limit the amount of rows it checks in the loop. It then looks for any row marked for push and copies that range to another spreadsheet in my drive. It then continues to the next file in the folder and does the same.
Here's my code:
function myVupdate() {
try {
var folder = DriveApp.getFolderById("123abc"),
files = folder.getFiles();
while (files.hasNext()) {
var file = files.next(),
sss = SpreadsheetApp.open(file);
SpreadsheetApp.setActiveSpreadsheet(sss);
//Work orders update
var ss = sss.getSheetByName("Sheet2"),
refresh = ss.getRange("W3").getValue();
if (refresh == 0) {continue};
var avals = ss.getRange("D5:D").getValues(),
count = avals.filter(String).length,
rows = count + 5
var val = ss.getDataRange().getValues();
for (var row=5; row < rows; row++) {
var cell = ss.getDataRange().getCell(row, 23).getValue();
if (cell == 0) {
var cells = [["v" + "WO-" + val[row-1][3] + "_" + val[row-1][2],val[row-1][13],val[row-1][14],val[row-1][15],new Date()]];
var tss = SpreadsheetApp.openById("target_spreadsheet"),
ts = tss.getSheetByName("Sheet5");
ts.insertRowBefore(2);
var last_hmy = ts.getRange(3,1).getValue();
ts.getRange(2,1).setValue(last_hmy+1);
ts.getRange(2,2,cells.length,cells[0].length).setValues(cells);
ts.getRange(2,7).setValue(sss.getName());
ss.getRange(row,17).setValue(last_hmy+1);
ss.getRange(row,18,cells.length,cells[0].length).setValues(cells);
//Turnover update
var ss = sss.getSheetByName("Sheet1"),
avals = ss.getRange("D5:D").getValues(),
count = avals.filter(String).length,
rows = count + 5
var val = ss.getDataRange().getValues();
}
}
for (var row=5; row < rows; row++) {
var cell = ss.getDataRange().getCell(row, 24).getValue();
if (cell == 0) {
var cells = [["v" + val[row-1][3] + "_" + val[row-1][2],val[row-1][12],val[row-1][15],val[row-1][16],new Date()]];
var tss = SpreadsheetApp.openById("target_spreadsheet"),
ts = tss.getSheetByName("Sheet5");
ts.insertRowBefore(2);
var last_hmy = ts.getRange(3,1).getValue();
ts.getRange(2,1).setValue(last_hmy+1);
ts.getRange(2,2,cells.length,cells[0].length).setValues(cells);
ts.getRange(2,7).setValue(sss.getName());
ss.getRange(row,18).setValue(last_hmy+1);
ss.getRange(row,19,cells.length,cells[0].length).setValues(cells);
}
}
}
}
catch(e) {
// Browser.msgBox("An error occured. A log has been sent for review.");
var errorSheet = SpreadsheetApp.openById ("target_sheet").getSheetByName("Error Log"),
source = sss.getName();
lastRow = errorSheet.getLastRow();
var cell = errorSheet.getRange('A1');
cell.offset(lastRow, 0).setValue(e.message);
cell.offset(lastRow, 1).setValue(e.fileName);
cell.offset(lastRow, 2).setValue(e.lineNumber);
cell.offset(lastRow, 3).setValue(source);
cell.offset(lastRow, 4).setValue(new Date());
MailApp.sendEmail("my#email.com", "Error report - " + new Date(),
"\r\nSource: " + source + "\r\n"
+ "\r\nMessage: " + e.message
+ "\r\nFile: " + e.fileName
+ "\r\nLine: " + e.lineNumber
);
}
}
Hello and welcome to Stack Overflow,
first of all, you are correct. The more getValue(), or setValue() calls you do the worse the performance, read more on best practices here. Google recommends you batch these as much as possible. One thing that immediately springs to attention is the following:
var val = ss.getDataRange().getValues();
so now you have all the values on the sheet in a 2D array. That means that in the following bit
var ss = sss.getSheetByName("Sheet2"),
refresh = ss.getRange("W3").getValue();
if (refresh == 0) {continue};
var avals = ss.getRange("D5:D").getValues(),
count = avals.filter(String).length,
rows = count + 5
var val = ss.getDataRange().getValues();
for (var row=5; row < rows; row++) {
var cell = ss.getDataRange().getCell(row, 23).getValue();
every single getValue() or getValues() is no longer necessary. Instead, you know that refresh = val[2][22] because you need the 3rd row and 23rd column, as you already have the entire range that has data from that sheet.
Same with avals as all values in range D5:D are in vals[n][3], where n starts from 4. Remember, the array index starts from 0 (so first row and first column is vals[0][0].
So anywhere you are trying to use getValues() from the ss spreadsheet, you already have that data. What you can also do, is manipulate the array you have, so you always change the values only in that array. Once you are done with it, you use ss.getDataRange().setValues(vals) to push the entire array back to the same range (you can just store the range in a variable like datRange = ss.getDataRange() and then do datRange.setValues(vals).
You will just need to work with a separate data array for any other sheet. I did not go into detail for the rest of the code as the same ideas go throughout. Since you already grab everything with getValues() there is no longer any reason to use getValue() for any cell within that range.
Related
I'm trying to write a simple script to clear the values from a specific Google Sheets range [H29:H] if the background color of the cell = #ffff00. While the script doesn't return any errors, it isn't making any changes to cells that meet those criteria (or anywhere else in the sheet). Any guidance as to where I am going wrong?
function resetCells() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName('Test Sheet');
var rangeData = sheet.getDataRange();
var lastRow = rangeData.getLastRow();
var searchRange = sheet.getRange('H29:H');
var rangeColors = searchRange.getBackgrounds();
for ( i = 29 ; i < lastRow - 1; i++){
if(rangeColors[i][8] === '#ffff00'){
sheet.getRange(i,8).clearContent();
};
};
}
When I saw your script, I thought that the index of rangeColors might not be corresponding to i in the for a loop. I thought that this might be the reason for your issue. In this case, how about the following modification?
Modified script:
From:
for ( i = 29 ; i < lastRow - 1; i++){
if(rangeColors[i][8] === '#ffff00'){
sheet.getRange(i,8).clearContent();
};
};
To:
for (i = 0; i < rangeColors.length; i++) {
if (rangeColors[i][0] === '#ffff00') {
sheet.getRange(i + 29, 8).clearContent();
}
}
or, in your situation, when sheet.getRange(i + 29, 8).clearContent() is used at outside of the loop, the process cost might be able to be reduced a little. In this case, please modify as follows.
var rangeList = rangeColors.reduce((ar, [h], i) => {
if (h === '#ffff00') ar.push("H" + (i + 29));
return ar;
}, []);
sheet.getRangeList(rangeList).clearContent();
In this modification, when the background color of the cell of column "H" is #ffff00, the cell content is clear.
Reference:
Class RangeList
What you did wrong
In rangeColors[i][8] you were assuming rangeColors related to the cells in the spreadsheet. But they actually relate to the searchRange you made. So rangeColors[i][8]should be rangeColors[i][0]. Where i starts at 0.
To make it "extra fun" for you rangeColors is an array which is zero-based indexed. And when you are working with a range you can retreive the first cell with range.getCell(1, 1), not range.getCell(0, 0).
Because I got a little carried away I wrote this solution ( may be an interesting extra to read )
I don't think mapping an array of arrays (rangeColors) to a range is a great way to do this; It's hard to keep track on what your code is doing.
What I did was:
Create a range to check
Generate column and row numbers for all the cells in the range. Note that these are relative to the range, not the stylesheet. So range.getCell(1, 1) is the top left cell in the range. This is the (main) mistake you made in your code.
Do whatever you want with every individual cell in the range.
code:
resetCells( '#ffff00' );
function resetCells( color ) {
const app= SpreadsheetApp.getActiveSpreadsheet();
const sheet = app.getSheetByName('Sheet1');
const range = sheet.getRange('F4:F50');
for( let row = 1; row < range.getNumRows() + 1; row++ ) {
for( let column = 1; column < range.getNumColumns() + 1; column++ ) {
const cell = range.getCell( row, column );
if( cell.getBackground() === color )
cell.clearContent();
}
}
}
Disclaimer: I never used this google-app-script before. So I may have done something wrong performance-wise; iterating a large amount of cells takes a little long;
https://i.stack.imgur.com/7VAJk.png
i want to copy data from "dB" sheet A5:A29 and paste to correct column.
so i use the script to find the correct column.
there range B2:CX2 have 0(not-correct) or 1(correct) value, so i use 'for' & 'if'
BUT!! It's too delay!!
i use console.time() and i get 25909ms(timecheck2 value) !!!
please help me.....
here is my code
function save(){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName('dB');
console.time("timecheck1");
//find last row
var copyrangeO = sheet.getRange(5,1,25,1).getValues();
var lastrowO = copyrangeO.filter(String).length;
var copyrange = sheet.getRange(5,1,lastrowO,1);
console.timeEnd("timecheck1");
//my dB data start "B2".
var cv = 1;
//find correct value(1). B2 ~ CX2 (#100)
console.time("timecheck2");
for (var i=2; i<101;i++){
if(sheet.getRange(2,i).getValue()===1){
cv = i;
}
}
console.timeEnd("timecheck2");
//if data isn't correct, cv===1. so error msg print.
console.time("timecheck3");
if(cv ===1){
Browser.msgBox("ERROR")
}else {
//data copy and paste.
var columnToCheck = sheet.getRange(4,cv,1000).getValues();
var lastrow = getLastRowSpecial(columnToCheck);
var pasterange = sheet.getRange(lastrow+4,cv);
copyrange.copyTo(pasterange, SpreadsheetApp.CopyPasteType.PASTE_VALUES, false);
Browser.msgBox(lastrowO + " saved!");
}
console.timeEnd("timecheck3");
}
Issue:
If I understand your situation correctly, you want to find the cell in B2:CX2 in which the value is 1, but the script is taking too much time for this.
The problem here is that you are using getRange and getValue in a loop (sheet.getRange(2,i).getValue()===1). This greatly increases the amount of calls to the Sheets service, which slows down your script, as you can see at Minimize calls to other services.
Solution:
In that case, I'd suggest doing the following:
Get the values from all columns at once using getValues().
Use findIndex to get the column index for which value is 1.
In order to do that, replace this:
var cv = 1;
//find correct value(1). B2 ~ CX2 (#100)
console.time("timecheck2");
for (var i=2; i<101;i++){
if(sheet.getRange(2,i).getValue()===1){
cv = i;
}
}
With this:
var ROW_INDEX = 2;
var FIRST_COLUMN = 2; // Column B
var LAST_COLUMN = 102; // Column CX
var columnValues = sheet.getRange(ROW_INDEX, FIRST_COLUMN, 1, LAST_COLUMN-FIRST_COLUMN+1).getValues()[0];
var cv = columnValues.findIndex(columnValue => columnValue === 1) + FIRST_COLUMN;
Note:
If there's no cell in the range with value 1, findIndex returns -1 which, added to FIRST_COLUMN, results in 1. That's appropriate for your current script, but won't work if the FIRST_COLUMN stops being 2, so be careful with this (either change the condition if(cv ===1){ to something less strict, or don't assign the resulting value to cv if findIndex returns -1).
The function will spend most of its time in the for loop because it repeats the Range.getValue() call many times. You can speed things up quite a bit by getting all values with one Range.getValues() call, like this:
let cv = 1;
console.time("timecheck2");
sheet.getRange('B2:B100').getValues().flat()
.some((value, index) => (cv = 2 + index) && value === 1);
console.timeEnd("timecheck2");
Note that this is not a cleanest way of finding cv, but it should help illustrate why you have a performance issue. You may want to do a complete rewrite of the code, using declarative instead of imperative style.
Try this:
I don't know what you're doing in the save because to did not supply the helper function code.
function save(){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sh = ss.getSheetByName('dB');
var vs0 = sh.getRange(5,1,25,1).getValues();
var lr0 = vs0.filter(String).length;
var crg = sh.getRange(5,1,lr0,1);
var cv = 1;
const vs1 = sh.getRange(2,2,1,99).getValues().forEach((c,i) => {
if(c == 1)cv = i + 2
})
if(cv == 1){
Browser.msgBox("ERROR")
}else {
var vs2 = sh.getRange(4,cv,1000).getValues();
var lastrow = getLastRowSpecial(vs2);
var drg = sh.getRange(lastrow+4,cv);
crg.copyTo(drg, SpreadsheetApp.CopyPasteType.PASTE_VALUES, false);
Browser.msgBox(lr0 + " saved!");
}
}
There are two ways that i am able to add an auto increment column. By auto-increment, i mean that if column B has a value, column A will be incremented by a numeric value that increments based on the previous rows value.
The first way of doing this is simple, which is to paste a formula like the one below in my first column:
=IF(ISBLANK(B1),,IF(ISNUMBER(A1),A1,0)+1)
The second way i have done this is via a GA script. What i found however is performance using a GA script is much slower and error prone. For example if i pasted values quickly in the cells b1 to b10 in that order, it will at times reset the count and start at 1 again for some rows. This is because the values for the previous rows have not yet been calculated. I assume that this is because the GA scripts are probably run asynchronously and in parallel. My question is..is there a way to make sure each time a change happens, the execution of this script is queued and executed in order?
OR, is there a way i should write this script to optimize it?
function auto_increment_col() {
ID_COL = 1;
var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
//only increment column 1 for sheets in this list
var auto_inc_sheets = SpreadsheetApp.getActiveSpreadsheet().getRangeByName("auto_inc_sheets").getValues();
auto_inc_sheets = auto_inc_sheets.map(function(row) {return row[0];});
var is_auto_inc_sheet = auto_inc_sheets.indexOf(spreadsheet.getSheetName()) != -1;
if (!is_auto_inc_sheet) return;
var worksheet = spreadsheet.getActiveSheet();
var last_row = worksheet.getLastRow();
var last_inc_val = worksheet.getRange(last_row, ID_COL).getValue();
//if auto_inc column is blank and the column next to auto_inc column (col B) is not blank, then assume its a new row and increment col A
var is_new_row = last_inc_val == "" && worksheet.getRange(last_row, ID_COL+1).getValue() != "";
Logger.log("new_row:" + is_new_row + ", last_inc_val:" + last_inc_val );
if (is_new_row) {
var prev_inc_val = worksheet.getRange(last_row-1, ID_COL).getValue();
worksheet.getRange(last_row, ID_COL).setValue(prev_inc_val+1);
}
}
There is my vision of auto increment https://github.com/contributorpw/google-apps-script-snippets/tree/master/snippets/spreadsheet_autoincrement
The main function of this is
/**
*
* #param {GoogleAppsScript.Spreadsheet.Sheet} sheet
*/
function autoincrement_(sheet) {
var data = sheet.getDataRange().getValues();
if (data.length < 2) return;
var indexCol = data[0].indexOf('autoincrement');
if (indexCol < 0) return;
var increment = data.map(function(row) {
return row[indexCol];
});
var lastIncrement = Math.max.apply(
null,
increment.filter(function(e) {
return isNumeric(e);
})
);
lastIncrement = isNumeric(lastIncrement) ? lastIncrement : 0;
var newIncrement = data
.map(function(row) {
if (row[indexCol] !== '') return [row[indexCol]];
if (row.join('').length > 0) return [++lastIncrement];
return [''];
})
.slice(1);
sheet.getRange(2, indexCol + 1, newIncrement.length).setValues(newIncrement);
}
But you have to open the snippet for details because this doesn't work without locks.
I have this script in Google Spreadsheet and it fetches all the rows marked "READY" just fine, then it sets the value in column "W"(23) to "SENT", and then I am trying to avoid fetching duplicates, by marking the column as "SENT" but then when I run the code again, it ignores the "SENT" that it just pasted? What is wrong here?
var ss = SpreadsheetApp.openById("12y85GmJ94s6k3213j2nGK8rFr0GOfd_Emfk8WHu_MUQ");
var stitchSheet = ss.getSheetByName("Sheet8");
var orderSheet = ss.getSheetByName("Sheet1");
var SENT = "SENT";
function getOrders() {
var range = orderSheet.getDataRange();
var orders = range.getValues();
for (var i = 1; i < orders.length; i++) {
var row = orders[i];
var status = row[1];
var order = row[4];
var name = row[5];
var system = row[22];
if(system != SENT){
if(status.toString() === 'READY'){
orderSheet.getRange(i,23).setValue(SENT);
stitchSheet.appendRow([order,name]);
}
}
}
}
Your code is fine, so there must be a logical error somewhere. I've noticed that you made var i in the for loop 1. I do not know if this is intentional or not, but the index of arrays pretty much always starts with 0 in most programming languages, which means that you'll start at row 2 of your sheet, not row 1.
Finding logical errors
To find logical errors you need to learn how to use the debugger console in the script editor.
Put breakpoints on the lines I mark with a star below:
var ss = SpreadsheetApp.openById("12y85GmJ94s6k3213j2nGK8rFr0GOfd_Emfk8WHu_MUQ");
var stitchSheet = ss.getSheetByName("Sheet8");
var orderSheet = ss.getSheetByName("Sheet1");
var SENT = "SENT";
function getOrders() {
var range = orderSheet.getDataRange();
* var orders = range.getValues();
for (var i = 1; i < orders.length; i++) {
* var row = orders[i];
var status = row[1];
var order = row[4];
var name = row[5];
* var system = row[22];
if(system != SENT){
if(status.toString() === 'READY'){
orderSheet.getRange(i,23).setValue(SENT);
stitchSheet.appendRow([order,name]);
}
}
}
}
Start the debugger and it will stop at the first breakpoint. Inspect what value range has. Expanding This should let you find orderSheet and SENT (since they are outside the function and should be in the scope). If not you've got a problem here.
Go to the next breakpoint and inspect orders, it should have an array of arrays now. You can inspect that you've got the right values, if not, skip forward to the next breakpoint and look at what row is.
I modified the code this way... it gathers data from all the sheets and finds only the rows that have data, BUT now I am having a problem modifying the range with each pass so that it is equal to the number of rows that do have value (found with (values[row][0] != '')). I have put a ??? in the spot where I am trying to have a variable height.
function getAllData() {
var folder = DocsList.getFolderById("folderid");
var contents = folder.getFiles();
Logger.log("file length: " + contents.length);
var file;
var data;
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Base")
sheet.clearContents();
var numOfFiles = contents.length;
for (var i = 0; i < numOfFiles; i++) {
file = contents[i];
Logger.log("count: " + i);
var theFileType = file.getFileType();
Logger.log("theFileType: " + theFileType);
if (theFileType==DocsList.FileType.SPREADSHEET) {
var sheet2 = SpreadsheetApp.open(file).getSheetByName("Sheet 1");
var lastLine = sheet2.getLastRow();
var values = sheet2.getRange('A3:J').getValues();
var formulas = sheet2.getRange('A3:J').getFormulas();
var data = [];
for(var row = 0 ; row < (values).length ; row++){
var lastrow = sheet.getLastRow()+1;
if (values[row][0] != '') {
for(var col = 0 ; col < formulas[row].length ; col++){
if(formulas[row][col] != '')
{values[row][col] = formulas[row][col]};
data.push(values[row]);}
if(data.length > 0)
sheet.getRange(lastrow, 1, ???, data[0].length).setValues(data);
}
}
};
}}
You are using getValue() as opposed to getValues() (With a letter "s" on the end)
var onecell = posheet.getRange('B4').getValue();
The documentation states:
getValue() - Returns the value of the top-left cell in the range.
The parameter for getRange() is kind of tricky and not well documented.
For example this:
getRange(2, 3, 6, 4)
gets a range from C2 to G8. Figure that out. The first number is the number 2, which is for the row 2. The second number is 3, for the third column (which is C). The third and fourth numbers are relative to the first two numbers.
Also, you are using: appendRow([array]) which uses an array for the parameter. So you must make sure that the data is in the form of an array, or use something else.
Here is the link for getValues:
Google Documentation - getValues
The example is this code:
// The code below will get the values for the range C2:G8
// in the active spreadsheet. Note that this will be a javascript array.
var values = SpreadsheetApp.getActiveSheet().getRange(2, 3, 6, 4).getValues();
Logger.log(values[0][0]);
Here is code that seems to work:
function getAllData() {
var folder = DocsList.getFolderById("Your file ID");
var contents = folder.getFiles();
Logger.log("file length: " + contents.length);
var file;
var data;
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet1")
sheet.clearContents();
sheet.appendRow(["Value from Sheet One", "Range of values from Sheet Two"]);
var numOfFiles = contents.length;
for (var i = 0; i < numOfFiles; i++) {
file = contents[i];
Logger.log("count: " + i);
//Reset to null on every iteration
var onecell = null;
var theRange = null;
var theFileType = file.getFileType();
Logger.log("theFileType: " + theFileType);
if (theFileType==DocsList.FileType.SPREADSHEET) {
var sheet1 = SpreadsheetApp.open(file).getSheetByName("Sheet1");
var sheet2 = SpreadsheetApp.open(file).getSheetByName("Sheet2");
// The code below will get the values for the range A3:A9
// in the active spreadsheet. Note that this will be a javascript array.
onecell = sheet1.getRange('B4').getValue();
theRange = sheet2.getRange(1,3,1,6).getValues();
Logger.log('onecell: ' + onecell);
Logger.log('onecell[0][0]: ' + onecell[0][0]);
Logger.log('theRange: ' + theRange)
Logger.log('theRange[0][0]: ' + theRange[0][0])
var multipleValues = [theRange[0][0], theRange[0][1], theRange[0][2], theRange[0][3], theRange[0][4]];
Logger.log('multipleValues: ' + multipleValues);
sheet.appendRow([onecell, "'" + multipleValues]);
};
}
}
In the first column, it only enters one value into the sheet cell. In the second column, the cell gets multiple values put into it from the row. In other words, and entire rows values, and combined and put into one cell. I think that's what you want from the code.
If you try to put an array into a spreadsheet cell, instead of showing the array of values as text, it shows something like an object. So I put a quote in front of the values so the cell formatting would default to text.