I'm trying to find a faster way to copy specific rows of a sheet to different sheets. Iterating through them as done in code below takes too much time it leads to timeout.
Some information about origin sheet:
Has already a blocked header
Has around 5000 rows
Column A has a header "Project"
Sheet is sorted by Column A
Goal is to copy range of all rows for each project from origin sheet to a blank sheet that is named from specific project - so all rows that has in column e.g. "ProjectA" in column A are in a sheet called "ProjectA".
Here is a code that is working, but it is using very slow iteration, so I'm waiting around 20 minutes or even get a timeout when I'm processing it:
var sheet = SpreadsheetApp.getActiveSheet();
var columnRoom = sheet.getRange("A:A").getValues();
var rows = SpreadsheetApp.getActiveSheet().getDataRange().getValues();
var header = rows[0];
var completedRooms = []
var last = columnRoom[1][0]
for (var i = 1; i < columnRoom.length; i++) {
if (!completedRooms.includes(columnRoom[i][0])) {
if (columnRoom[i][0] === "") {
var currentSheet = SpreadsheetApp.getActiveSpreadsheet().insertSheet("No Room");
} else {
var currentSheet = SpreadsheetApp.getActiveSpreadsheet().insertSheet(columnRoom[i][0]);
}
currentSheet.appendRow(header);
currentSheet.appendRow(rows[i]);
completedRooms.push(columnRoom[i][0])
last = columnRoom[i][0]
} else if (last == columnRoom[i][0]) {
var currentSheet = SpreadsheetApp.getActiveSpreadsheet()
currentSheet.appendRow(rows[i]);
}
}
Is there a way to do it faster? I'm thinking about appending specific rows to a range and use copyTo but I can't arrange it, maybe use map function?
The number of times you call insertSheet() and appendRow() functions is what makes your script slow.
For the appendRow() problem, instead of calling this function multiple times, send your data in a single call. For example, on lines 14-15 instead of calling appendRow(header) + appendRow(rows[i]) just do one call type sheet.setValues(header + rows[i]).
In general, I see 2 options to optimize your script:
1. GAS allows you to run 30 simultaneous executions, so you can adapt your script to run multiple executions, where each execution works a specific range of rows, this way instead of having a 20 min execution to process 100 rows, you have 10 simultaneous executions processing 10 rows and lasting only 2 min.
2. All your sheets have the same values, the only difference is their name and the row they reference data, so why not get the data with formula and only use the script to change the names of the sheets?
Related
This question already has answers here:
Deleting rows in google sheets using Google Apps Script
(10 answers)
Closed 2 years ago.
i want to delete multiple rows of data on condition if column B in that row is equal to CID parameter. but after I run it, the deleted data is incorrect. Sometimes the deleted data is located one row above or below the specified row in the for row loop
here is my code :
if (e.parameter.func == "DeleteRec") {
var rg=ss.getSheetByName("MAIN").getDataRange().getValues();
var lr= ss.getSheetByName("MAIN").getLastRow();
for(var row=1;row<=lr;row++){
var cid=rg[row].toString().split(',');
if(cid[1]==e.parameter.CID){
ss.getSheetByName("MAIN").deleteRow(row-1);
var data=200;
}
}
return ContentService.createTextOutput(data).setMimeType(ContentService.MimeType.TEXT);
}
Explanation:
You need to delete the rows from bottom-up or backwards because after you delete one row you change the structure of the spreadsheet file and therefore rg does not match the new structure of the sheet.
Keep in mind that array indexes in JavaScript start from 0. When variable row is 1 you want to delete row+1 in the sheet, because row=1 is getting the second element of your array.
Calling getSheetByName in a for loop will increase the performance cost of your script. Instead, define the sheet variable before the loop so you can use it every time you need it.
Since you are using getDataRange the length of rg is equal to what getLastRow returns. Therefore lr is not needed.
Not sure why you iteratively assign a fixed value to a variable var data=200 but I kept that part since it is not part of your question.
Solution:
Replace the relevant part with this code:
const sheet = ss.getSheetByName("MAIN");
const rg = sheet.getDataRange().getValues();
for (var row = rg.length - 1; row>0; row--) {
var cid=rg[row].toString().split(',');
if(cid[1]==e.parameter.CID){
sheet.deleteRow(row+1);
var data=200;
}
}
I have a function that adds an auto-expanding formula to some Header row cell
In the next line of code, I get the Display Values and then post them back to the sheet
I am concerned that I will be getting the values in the range of the auto-expanding formulas before they have finished expanding
Will this r.getDisplayValues(); get the values before the auto-expansion has finished? thereby getting values with blank data that should have data
I have tested various scenarios, but this is not definitive
Also, I have not been able to find anything in searching on this
Thank you
function setFormulasAE_n() {
var ss =SpreadsheetApp.getActive();
var sheet =ss.getSheetByName('Elements');
var LC = sheet.getLastColumn();
var LR = sheet.getLastRow();
//Auto-expanding Formulas to be added
//Two dim array with 1 row
var formulas = [[
"=ArrayFormula({\"ig_TagsHistorical\";iferror(vlookup(INDIRECT(\"Elements!A2:A\"&counta(Elements!$AJ$1:$AJ)), \'HelperElements_(ignore)\'!$A$2:$G, {5}, 0))})",
"=ArrayFormula({\"Additional Networks\";iferror(vlookup(INDIRECT(\"Elements!A2:A\"&counta(Elements!$AJ$1:$AJ)), \'Helper_(ignore)\'!$A$2:$D, {4}, 0))})",
]];
//Add auto-expanding formulas to Cell(s)
var cell = sheet.getRange(1,LC+1,1,formulas[0].length);
cell.setFormulas(formulas);
//Get range and post back Display Values
var r = sheet.getRange(1,LC+1,LR,formulas[0].length);
var v = r.getDisplayValues();
r.setValues(v)
}
The setFormulas() function you have used in the script is synchronous which essentially means that the code following the instruction won't be executed until the functions finishes the execution.
Therefore, r.getDisplayValues() will always get the values after the auto-expansion.
What you could do to make sure that the values you get are the expected ones is to use the flush() function after the cell.setFormulas(formulas) line of code. What flush() does is basically applying all the pending Spreadsheet changes - more specifically the formulas you need to set.
Furthermore, you can check these links since they might be of help to you:
SpreadsheetApp Class - flush();
Range Class - setFormulas().
Some background: We have a shared Google Sheet to track our openings, screenings, and other events at a movie theater. We have a main tab ("Master") that contains all of our events and the details that go with them, and a tab for archiving ("Archive").
I would like to write a script within Google Sheets to detect events & screenings that are from yesterday and earlier based on the date (in column E), take the full row(s) (events) that meet that criteria, copy & paste them to the separate "Archive" tab, and then delete the row(s) from the "Master" tab.
Anything to point me in the right direction would be super helpful. I found a few similar responses to this but they're specific to Excel/VBA and I'm not familiar with that (or much Javascript, for that matter).
I suggested that you do some tutorials to familiarise yourself with how to write scripts.
In this answer, I will flesh out the steps that your code needs to address. You will find many existing topics on the same or similar question. This is merely in order to enable you to better search for the elements of code that you need. Consider that this may be just one way of achieving your outcome.
You have one spreadsheet with two sheets and you will refer to both sheets at different stages. getSheetByName(name) will enable you to create a re-usable variable for a sheet.
You will need to find the bottom row in each sheet. getLastRow() will help.
You want to find rows in "Master" for dates, so you need to get ALL values for "Master".
You'll start by defining the range - use getRange(row, column, numRows, numColumns), though this is just one of 5 ways to define a range.
Having defined the range you'll need the values in "Master" so that you can access the date field. Use getValues() in conjunction with the range that defined. FWIW, note how this is in plural because there are lots of values. If you just wanted a single cell, you'd use getValue().
You'll want to loop through the rows in "Master" and find those rows that have a date prior to today. The "Removing Duplicate Rows in a Spreadsheet" tutorial shows one way of looping, and you can read up on basic JavaScript "Loops and iteration".
In your scenario, there is a 'hitch' with looping. If one adopts the "usual" process, then one will loop from the first row to the last. However, you are deleting a row from "Master" and, as each row is deleted, the row numbers of the remaining rows will/may change; so the "usual" process won't do. What you need to do is two things: first) loop from the bottom of the range; this will ensure that the row numbers of remaining rows will never change; second) sort the data so that the oldest dates are at the bottom. So... now you will loop from the bottom to the top, and you will evaluate all the oldest dates without any risk that when you encounter a date greater than "today", there will be NO risk of further rows with a date less than "today". Of course, once the code is complete, you can always re-sort the data on "Master" back to any order that you might wish.
You need to compare the date in the row in "Master" with today's date and then build a if...else statement so that you can define what to do depending on the result. Comparing dates is sometimes easier said than done. This topic is relevant Checking if one date is greater than the other using Google Script and you can search on other topics for "Google Sheets Script date comparison".
When you find a date less than today, you want to copy the details of that row to "Archive". This is a two part process first) to gather there the data from the row on "Master", and second) to "copy" that data to "Archive". Gathering the data will have been covered in the tutorials. There are many options for copying the data to "Archive". You could append a row and use setValues to update the new values. An alternative is to accumulate the additional "Archive" data and add it to the "Archive" after the loops have been completed.
When you find a date less than today, you want to delete the row from "Master". There's a command for that: deleteRow(rowPosition).
You can process your function manually, on demand, or you may prefer it to be automated as a time-driven installable trigger. The option is yours.
There are many ways that you can combine these elements.
In preparing the summary above, I had to make sure that I was providing accurate and complete advice. So the following is but one approach to achieving your goal. It should be noted that my test data assumes that columns A and C are formatted for date and time respectively.
function so5710086103() {
// set up spreadsheet and sheets
var ss = SpreadsheetApp.getActiveSpreadsheet();
var master = ss.getSheetByName("Master");
var archive = ss.getSheetByName("Archive");
// get the last row and column of Master
var masterLR = master.getLastRow();
var masterLC = master.getLastColumn();
// get the last row and column of Archive
var archiveLR = archive.getLastRow();
var archiveLC = archive.getLastColumn();
//Logger.log("DEBUG: Last Row - Master = "+masterLR+", and Archive = "+archiveLR);
//Logger.log("DEBUG: Last Column - Master = "+masterLC+", and Archive = "+archiveLC);
// create a range, sort it and get the data from "Master"
var masterRange = master.getRange(2, 1, masterLR - 1, masterLC);
// sort master based on date
masterRange.sort({
column: 1,
ascending: false
});
// Logger.log("DEBUG: Master range = "+masterRange.getA1Notation());
var masterData = masterRange.getValues();
//Logger.log("DEBUG: Length of Master data = "+masterData.length);
// create a range and get the data from "Archive"
var archiveRange = archive.getRange(1, 1, archiveLR, archiveLC);
var archiveData = archiveRange.getValues();
// create a formatted date for today
var formattedToday = Utilities.formatDate(new(Date), 'GMT+10',
'dd MMMM yyyy');
// loop through the rows
// from bottom to top
for (var i = (+masterLR - 2); i > 0; i--) {
// convert cell dates to comparable format
var DBdate = Utilities.formatDate(masterData[i][0], 'GMT+10',
'dd MMMM yyyy');
var DBtime = Utilities.formatDate(masterData[i][2], 'GMT+10',
'hh:mm a');
//Logger.log("DEBUG: i = "+i+", DBdate = "+DBdate+", Today = "+formattedToday);
// clear the temporary row array
var archivecells = [];
if (DBdate < formattedToday) {
// the table date is less than today, so archive the data
// Logger.log("DEBUG: i = "+i+", DBdate = "+DBdate+", Today = "+formattedToday+" - DB value is less than Today. ACTION: Archive this row");
// copy the row cells to temporary row array
archivecells.push(DBdate);
archivecells.push(masterData[i][1]);
archivecells.push(DBtime);
archivecells.push(masterData[i][3]);
archivecells.push(masterData[i][4]);
// copy the temporary row array to archivedata
archiveData.push(archivecells);
// delete the Master Row
master.deleteRow(i + 2);
} else {
// the table date is NOT less than today, so do nothing
// Logger.log("DEBUG: i = "+i+", DBdate = "+DBdate+", Today = "+formattedToday+" - DB value is NOT less than Today. ACTION: Do nothing");
}
// update the accumulated data to Archive.
archive.getRange(1, 1, archiveData.length, archiveLC).setValues(
archiveData);
}
}
getSheetByName(name)
getLastRow()
getRange(row, column, numRows, numColumns)
getValues()
Loops and iteration
Javascript if...else
deleteRow(rowPosition)
Master - Before
Master - After
Archive - After
I feel like I'm going about this in all the wrong way. I'm trying to automate some of my workload here. I'm cleaning up spreadsheets with 4 columns (A-E), 2000+ rows. Column B contains website URLs, column D contains the URL's business name, generated from another source.
Sometimes the tool doesn't grab the name correctly or the name is missing, so it populates the missing entries in column D with "------" (6 hyphens). I've been trying to make a function that takes an input cell, checks if the contents of the cell are "------", and if it is the function changes the contents of the input cell to the contents of the cell two columns to the left (which is generally a website url). This is what I've come up with.
function replaceMissing(input) {
var sheet = SpreadsheetApp.getActiveSheet();
//sets active range to the input cell
var cell = sheet.getRange('"' + input + '"');
//gets cell to fill input cell
var urlCell = sheet.getRange(cell.getRow(), cell.getColumn() - 2);
//gets contents of input cell as String
var data = cell.getValue();
//gets contents of urlCell as String
var data2 = cell.getValue();
//checks if input cell should be replaced
if (data === "------") {
//set current cell's value to the value of the cell 2 columns to the left
cell.setValue(data2);
}
}
When I attempt to use my function in my sheet, the cell is returning the error
Error Range not found (line 4).
I'm assuming, based on similar questions people have asked, that this is how you use the A1 notation of the function with an argument. However, that doesn't seem to be the case, so I'm stuck. I also don't think my solution is very good period.
1) It's somewhat ambiguous in GAS documentation, but custom functions have quite a few limitations. They are better suited for scenarios where you need to perform a simple calculation and return a string or a number type value to the cell. While custom functions can call some GAS services, this practice is strongly discouraged by Google.
If you check the docs for the list of supported services, you'll notice that they support only some 'get' methods for Spreadsheet service, but not 'set' methods https://developers.google.com/apps-script/guides/sheets/functions
That means you can't call cell.setValue() in the context of a custom function. It makes sense if you think about it - your spreadsheet can contain 1000s of rows, each with its own custom function making multiple calls to the server. In JavaScript, every function call creates its own execution context, so things could get ugly very quickly.
2) For better performance, use batch operations and don't alternate between read / write actions. Instead, read all the data you need for processing into variables and leave the spreadsheet alone. After processing your data, perform a single write action to update values in the target range. There's no need to go cell by cell when you can get the entire range using GAS.
Google Apps Script - best practices
https://developers.google.com/apps-script/guides/support/best-practices
Below is a quick code example that runs onOpen and onEdit. If you need more flexibility in terms of when to run the script, look into dynamically-created triggers https://developers.google.com/apps-script/reference/script/script-app
Because your spreadsheets have lots of rows, you may hit the execution quota anyway - by using triggers you can work around the limitation.
Finally, if a cell containing '----' is a rare occurrence, it might be better to create another array variable with new values and row numbers to update than updating the entire range.
Personally, I think the single range update action would still be quicker, but you could try both approaches and see which one works best.
function onOpen(){
test();
}
function onEdit() {
test();
}
function test() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName('yourSheetName');
//range to replace values in
var range = sheet.getRange(2, 4, sheet.getLastRow() - 1, 1);
//range to get new values from
var lookupRange = range.offset(0, -2);
//2d array of values from the target range
var values = range.getValues();
//2d array of values from the source range
var lookupValues = lookupRange.getValues();
//looping through the values array and checking if array element meets our condition
for (var i=0; i < values.length; i++) {
values[i][0] = (values[i][0] == '------') ? lookupValues[i][0] : values[i][0];
}
// one method call to update the range
range.setValues(values);
}
Goal: I am trying to pull data from a website and use it to create a big table. I can tell that I'm very close to getting this to work, but I've reached a roadblock.
Background:
I have a google sheet with three pages. (1) Titled "tickers" is a list of every ticker in the S&P 500, in rows A1-A500. (2) Titled actionField is just a blank page used during the script. (3) Titled resultField will hold the results. The website I am pulling from is (http://www.reuters.com/finance/stocks/companyOfficers?symbol=V) Though, I want the script to work (with minor modification) for any data accessible through importHtml.
Script:
The script I currently have is as follows:
function populateData() {
var googleSheet = SpreadsheetApp.getActive();
// Reading Section
var sheet = googleSheet.getSheetByName('tickers');
var tickerArray = sheet.getDataRange().getValues();
var arrayLength = tickerArray.length;
var blankSyntaxA = 'ImportHtml("http://www.reuters.com/finance/stocks/companyOfficers?symbol=';
var blankSyntaxB = '", "table", 1)';
// Writing Section
for (var i = 0; i < arrayLength; i++)
{
var sheet = googleSheet.getSheetByName('actionField');
var liveSyntax = blankSyntaxA+tickerArray[i][0]+blankSyntaxB;
sheet.getRange('A1').setFormula(liveSyntax);
Utilities.sleep(5000);
var importedData = sheet.getDataRange().getValues();
var sheet = googleSheet.getSheetByName('resultField');
sheet.appendRow(importedData)
}
}
This successfully grabs the ticker from the tickers page. Calls importHtml. Copies the data. And appends SOMETHING to the right page. It loops through and does this for each item in the ticker list.
However, the data being appended is as follows:
[Ljava.lang.Object;#42782e7c
[Ljava.lang.Object;#2de9f184
[Ljava.lang.Object;#4b86a4d0
That displays across many columns, for as many rows as there are iterations in the loop.
How do I successfully append the data?
(And any advice on improving this script?)
The appendRow method is not suitable here. As it only appends one row, its argument is expected to be a 1D array of values.
What you get from getValues is normally a 2D array of values, like [[a,b], [c,d]]. Even if it's just one row, getValues will return [[a,b]]. The only exception is a single-cell range, for which you get just the value in that cell. It's never a 1D array.
If just one row is needed, use, e.g., appendRow(importedData[0]).
Otherwise, insert the required number of rows and assign the 2D array of values to them.
var sheet = googleSheet.getSheetByName('resultField');
var lastRow = sheet.getLastRow();
sheet.insertRowsAfter(lastRow, importedData.length);
sheet.getRange(lastRow + 1, 1, importedData.length, importedData[0].length)
.setValues(importedData);