I'm using Google Apps Script to write a script to edit Google Sheets for a Mailing List. I'd like it to run through all rows and delete any rows with 'BOUNCED' 'ERROR' or 'NO_RECIPIENT' in a specific cell.
The problem I'm having is the For Loop uses brackets [ ] to designate the rows and columns, which indexes the first row at 0. The deleteRows() action uses curved parenthesis, which indexes the first row at 1. For this reason, I'm having trouble deleting the correct row.
If I program deleteRow(i), it deletes the row following the one being tested by the For loop. If I program deleteRow(i+1), it deletes the correct row the first time, but subsequently deletes the following row. See my code below:
function cleanUp() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getActiveSheet();
var data = sheet.getDataRange().getValues();
for ( var i = 1; i < 30; i++) {
if (data[i][9] === 'ERROR' || data[i][9] === 'BOUNCED' || data[i][9] === 'NO_RECIPIENT') {
sheet.deleteRow(i+1);
}
}
}
Once a row is deleted the rows below it change it's position. One way to avoid this problem is to do the loop in reverse order.
In other words, instead of
for(var i = 1; i < 30; i++)
Use
for(var i = 29 ; i > 0; i--)
Related
I manage Google Sheet lists that sometimes exceed 10,000 rows. For sheets with rows up to around 5,000, the remove duplicates function noted below works finely. But for anything above 5,000, I receive the 'Exceeded maximum execution time' error. I would be grateful for some instruction on how to make the code more efficient such that it could run smoothly even for sheets with 10k+ rows.
function removeDuplicates() {
var sheet = SpreadsheetApp.getActiveSheet();
var data = sheet.getDataRange().getValues();
var newData = new Array();
for(i in data){
var row = data[i];
var duplicate = false;
for(j in newData){
if(row.join() == newData[j].join()){
duplicate = true;
}
}
if(!duplicate){
newData.push(row);
}
}
sheet.clearContents();
sheet.getRange(1, 1, newData.length, newData[0].length).setValues(newData);
}
There are a couple of things that are making your code slow. Let's look at your two for loops:
for (i in data) {
var row = data[i];
var duplicate = false;
for (j in newData){
if (row.join() == newData[j].join()) {
duplicate = true;
}
}
if (!duplicate) {
newData.push(row);
}
}
On the face of it, you're doing the right things: For every row in the original data, check if the new data already has a matching row. If it doesn't, add the row to the new data. In the process, however, you're doing a lot of extra work.
Consider, for example, the fact that at any given time, a row in data will have no more than one matching row in newData. But in your inner for loop, after you find that one match, it still continues checking the rest of the rows in newData. The solution to this would be to add a break; after duplicate = true; to stop iterating.
Consider also that for any given j, the value of newData[j].join() will always be the same. Suppose you have 100 rows in data, and no duplicates (the worst case). By the time your function finishes, you'll have calculated newData[0].join() 99 times, newData[1].join() 98 times... all in all you'll have done almost 5,000 calculations to get the same 99 values. A solution to this is memoization, whereby you store the result of a calculation in order to avoid doing the same calculation again later.
Even if you make those two changes, though, your code's time complexity is still O(n²). If you have 100 rows of data, in the worst case the inner loop will run 4,950 times. For 10,000 rows that number is around 50 million.
However, we can do this is O(n) time instead, if we get rid of the inner loop and reformulate the outer loop like so:
var seen = {};
for (var i in data) {
var row = data[i];
var key = row.join();
if (key in seen) {
continue;
}
seen[key] = true;
newData.push(row);
}
Here, instead of checking every row of newData for a row matching row in every iteration, we store every row we've seen so far as a key in the object seen. Then in each iteration we just have to check if seen has a key matching row, an operation we can do in nearly constant time, or O(1).1
As a complete function, here's what it looks like:
function removeDuplicates_() {
const startTime = new Date();
const sheet = SpreadsheetApp.getActiveSheet();
const data = sheet.getDataRange().getValues();
const numRows = data.length;
const newData = [];
const seen = {};
for (var i = 0, row, key; i < numRows && (row = data[i]); i++) {
key = JSON.stringify(row);
if (key in seen) {
continue;
}
seen[key] = true;
newData.push(row);
}
sheet.clearContents();
sheet.getRange(1, 1, newData.length, newData[0].length).setValues(newData);
// Show summary
const secs = (new Date() - startTime) / 1000;
SpreadsheetApp.getActiveSpreadsheet().toast(
Utilities.formatString('Processed %d rows in %.2f seconds (%.1f rows/sec); %d deleted',
numRows, secs, numRows / secs, numRows - newData.length),
'Remove duplicates', -1);
}
function onOpen() {
SpreadsheetApp.getActive().addMenu('Scripts', [
{ name: 'Remove duplicates', functionName: 'removeDuplicates_' }
]);
}
You'll see that instead of using row.join() this code uses JSON.stringify(row), because row.join() is fragile (['a,b', 'c'].join() == ['a', 'b,c'].join(), for example). JSON.stringify isn't free, but it's a good compromise for our purposes.
In my tests this processes a simple spreadsheet with 50,000 rows and 2 columns in a little over 8 seconds, or around 6,000 rows per second.
I am trying to find an empty cell in a specific column in google sheets. I am familiar with getLastRow(), but it returns the last row in whole sheet. I want to get the first empty cell in a specific column. So, I used a for loop and if. But I don't know why it is not working. The problem is that for loop does not return anything.
I am getting 10 rows of the column H (position 8) in the sheet test (line2). first 5 rows already have content. Data will be added to this cell later using another code. so, I want to be able to find the first empty cell in this column to be able to put the new data.
var sheettest = SpreadsheetApp.getActive().getSheetByName("test");
var columntest = sheettest.getRange(4, 8, 10).getValues();
for(var i=0; columntest[i]=="" ; i++){
if (columntest[i] ==""){
var notationofemptycell = sheettest.getRange(i,8).getA1Notation();
return notationofemptycell
}
}
As I said, I want to find the first empty cell in that column. I defined the for loop to go on as long as the cell is not empty. then if the cell is empty, it should return the A1 notation of that cell.
It seems the for loop does go on until it find the empty cell, because I get no definition for the var "notationofemptycell" in debugging.
This may be faster:
function getFirstEmptyCellIn_A_Column() {
var rng,sh,values,x;
/* Ive tested the code for speed using many different ways to do this and using array.some
is the fastest way - when array.some finds the first true statement it stops iterating -
*/
sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('test');
rng = sh.getRange('A:A');//This is the fastest - Its faster than getting the last row and
//getting a specific range that goes only to the last row
values = rng.getValues(); // get all the data in the column - This is a 2D array
x = 0;
values.some(function(ca,i){
//Logger.log(i)
//Logger.log(ca[0])
x = i;//Set the value every time - its faster than first testing for a reason to set the value
return ca[0] == "";//The first time that this is true it stops looping
});
Logger.log('x: ' + x)//x is the index of the value in the array - which is one less than
//the row number
return x + 1;
}
You need to loop through the length of columntest. Try this:
function myFunction() {
var sheettest = SpreadsheetApp.getActive().getSheetByName("test");
var lr=sheettest.getLastRow()
var columntest = sheettest.getRange(4, 8, lr).getValues();
for(var i=0; i<columntest.length ; i++){
if (columntest[i] ==""){
var notationofemptycell = sheettest.getActiveCell().getA1Notation();
Logger.log( notationofemptycell)
break;//stop when first blank cell on column H os found.
}
}
}
I'm working in Google apps script and seem to have screwed up one of my for loops. I'm sure that I am missing something trivial here, but I can't seem to spot it.
Code Snippet:
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheets = ss.getSheets();
var lastRow = sheets[3].getLastRow();
var zw = sheets[3].getRange(2, 1, lastRow - 1, 26).getValues();
for (var j = 0; j < zw.length; ++j) {
if (zw[j][9] === 'Yes') {
var masterEmail = [];
var firstLetterLastName = [];
var first2Letter = [];
var masterEmail.push(zw[j][22]);
var firstLetterLastName.push(zw[j][1].charAt(0).toLowerCase());
var first2Letter.push(zw[j][1].charAt(0).toLowerCase() + zw[j][1].charAt(1).toLowerCase());
//The rest of the function follows...
}
}
What's Not Working:
The for loop doesn't increment. When running the code in a debugger, var j stays at a value of 0.0, and the rest of the function only runs based of off the values in the 0 position of zw.
What I need it to do (AKA - How I thought I had written it:)
The ZW variable is holding a 2 dimensional array of cell values from a Google sheet. I'm looping through that, checking the 9th value of each array entry for a string of "Yes" and then running the rest of the function (for each column with a "Yes") if the condition is true.
I thought I had this working before, but recently had to restructure and optimize some things. Now I'm starting to think I may need to rethink things and use a different loop method. Can anyone educate me?
Edit: Here's a bit more context as requested:
function menuItem1() {
var ui = SpreadsheetApp.getUi();
var response = ui.alert('Are you sure you want to send emails?', ui.ButtonSet.YES_NO);
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheets = ss.getSheets();
var lastRow = sheets[3].getLastRow();
var zw = sheets[3].getRange(2, 1, lastRow - 1, 26).getValues();
if (response === ui.Button.YES) {
for (var j = 0; j < zw.length; j++) {
if (zw[j][9] === 'Yes') {
var firstLetterLastName = [];
firstLetterLastName.push(zw[j][1].charAt(0).toLowerCase());
//Other Stuff....
}
}
}
}
I have a menu item attached to a simple onOpen, that calls menuItem1(). Calling the function prompts the user with a warning that they are about to send emails, then goes about getting data to assign email addresses based on the contents of the sheets. firstLetterLastName is an example.
I'm still not getting the loop to function, is it because I have it between two if statements? (Here is a link to the sheet)
Indeed it is quite trivial. You have mixed up your increment. You wrote
for (var j = 0; j < zw.length; ++j)
which means that you do 1 + i (and we know that at the start i = 0 which means your value will always be 1) instead of using the usual
for (var j = 0; j < zw.length; j++)
which would mean that you do i + 1 and update i, so you will get the expected 0 + 1 1 + 1 etc
EDIT:
First, I recommend instead of something like
if (responseMir === ui.Button.YES) {
// Your For loop
doing
if (responseMir !== ui.Button.YES) {
return
}
and in a similar fashion in the for loop
if (zw[j][9] !== 'Yes') {
break
}
It mostly helps increase readability by not including large blocks of code under a single if, when all you want to do is to stop execution.
Your for loop gets broken because of the mistake here:
teacherEmailMir.push(selValsMir[j][7]);
So your loop will go over once. However on the next itteration, you try to push selValsMir[1][7] which does not exist. Note that each itteration you have var selValsMir = []; inside the loop, which means that for every j selValsMir will always be an empty array. So with the following line
selValsMir.push([zw[j][0], zw[j][1], zw[j][2], zw[j][3], zw[j][4], zw[j][5], zw[j][7], zw[j][22], zw[j][23], zw[j][24]]);
your array will always have selValsMir.lenght = 1 and selValsMir[0].length = 10. So obviously trying to access anything from selValsMir[1] will throw you an error and stop the script right there.
I also recommend looking over the if statements that look at the first and first 2 letters of the name as I believe you can accomplish the same with less code. Always try to streamline. Consider using switch() where you end up using a lot of else if
I'm writing a script that's going to look through a monthly report and create sheets for each store for a company we do work for and copy data for each to the new sheets. Currently the issue I'm running into is that we have two days of data and 171 lines is taking my script 369.261 seconds to run and it is failing to finish.
function menuItem1() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet1 = ss.getSheetByName("All Stores");
var data = sheet1.getDataRange().getValues();
var CurStore;
var stores = [];
var target_sheet;
var last_row;
var source_range
var target_range;
var first_row = sheet1.getRange("A" + 1 +":I" + 1);
//assign first store number into initial index of array
CurStore = data[1][6].toString();
//add 0 to the string so that all store numbers are four digits.
while (CurStore.length < 4) {CurStore = "0" + CurStore;}
stores[0] = CurStore;
// traverse through every row and add all unique store numbers to the array
for (var row = 2; row <= data.length; row++) {
CurStore = data[row-1][6].toString();
while (CurStore.length < 4) {
CurStore = "0" + CurStore;
}
if (stores.indexOf(CurStore) == -1) {
stores.push(CurStore.toString());
}
}
// sort the store numbers into numerical order
stores.sort();
// traverse through the stores array, creating a sheet for each store, set the master sheet as the active so we can copy values, insert first row (this is for column labels), traverse though every row and when the unique store is found,
// we take the whole row and paste it onto it's newly created sheet
// at the end push a notification to the user letting them know the report is finished.
for (var i = stores.length -1; i >= 0; i--) {
ss.insertSheet(stores[i].toString());
ss.setActiveSheet(sheet1);
target_sheet = ss.getSheetByName(stores[i].toString());
last_row = target_sheet.getLastRow();
target_range = target_sheet.getRange("A"+(last_row+1)+":G"+(last_row+1));
first_row.copyTo(target_range);
for (var row = 2; row <= data.length; row++) {
CurStore = data[row-1][6].toString();
while (CurStore.length < 4) {
CurStore = "0" + CurStore;
}
if (stores[i] == CurStore) {
source_range = sheet1.getRange("A" + row +":I" + row);
last_row = target_sheet.getLastRow();
target_range = target_sheet.getRange("A"+(last_row+1)+":G"+(last_row+1));
source_range.copyTo(target_range);
}
}
for (var j = 1; j <= 9; j++) {
target_sheet.autoResizeColumn(j);
}
}
Browser.msgBox("The report has been finished.");
}
Any help would be greatly appreciated as I'm still relatively new at using this, and I'm sure there are plenty of ways to speed this up, if not, I'll end up finding a way to break down the function to divide up the execution. If need be, I can also provide some sample data if need be.
Thanks in advance.
The problem is calling SpreadsheepApp lib related methods like getRange() in each iteration. As stated here:
Using JavaScript operations within your script is considerably faster
than calling other services. Anything you can accomplish within Google
Apps Script itself will be much faster than making calls that need to
fetch data from Google's servers or an external server, such as
requests to Spreadsheets, Docs, Sites, Translate, UrlFetch, and so on.
Your scripts will run faster if you can find ways to minimize the
calls the scripts make to those services.
I ran into the same situation and, instead of doing something like for(i=0;i<data.length;i++), I ended up dividing the data.length into 3 separate functions and ran them manually each time one of them ended.
Same as you, I had a complex report to automate and this was the only solution.
I have two sheets, one is updated from time to time, one is the backup which it may contain some the same values of the first one, the first one needs to be updated in some columns with recorded data already saved in the backup, I have written this script which is working fine some times but other times completely wrong (!) is taking right value form wrong row! second sheet is sorted descendant way so that inedxOf will match the right value for sure as first one.
var M = data.length;
for ( var r = 0; r < M ;r++){
var L = datiArchivio.length;
var row = data[r];
var nave = row[4];
var imo1 = data [r][23]
if ( imo1 == "" && nave!= ""){
for (var j = 0; j < L;j++){
var roww = datiArchivio[j];
var IMO = roww[23];
var naveArchivio = roww[4];
var GT = roww[25];
if ( IMO == ""){continue;} else {
if ( naveArchivio == nave) {
row.splice(23,1,IMO);
row.splice(25,1,GT);
}
}
}
}
}
I guess you meant something like this, but it is not working, in the archivio sheet there could be matching row with empty value at the reference column as well as fulfil values, it is ordered so that it should match first one which fullfil the condition, hence I added roww[23]=="" condition in order to skip that j row. it is challenging, keep on work on the logic.
This last version is working, and also quickly, I was stucked in the old "rubbish in rubbish out".