We Keep Coding

Service Spreadsheet Time Out - Optimizing Macro script?

Context
I have a g-sheet that acts as a sort of "master sheet" in which everything pours into from a bunch of other outside spreadsheets that are all consistently being live updated throughout the day.
Each outside spreadsheet I connect, routes to its own tab within our master spreadsheet through importrange function
All those tabs then route to one master tab using row ID #'s - so that everyone can just work from that tab.
The Problem
In this master tab where everything lands, I have a macro sorting the rows to bring the most recent rows to the top, among other things to keep the data clean. As I connect more sheets over time, I add to the number in the macro to accommodate new rows.
Macro a couple days ago started throwing "Service Spreadsheet timed out while accessing document with id..." then the id is the id # of the master tab itself.
Know there is probably a lot smoother way to have this done without using a large bandwidth macro in place, but optimizing the script to best fit the use-case is far out of my experience level. The macro I have in place is as follows:
function MasterSormat2() {
var spreadsheet = SpreadsheetApp.getActive();
spreadsheet.getRange('D1').activate();
var criteria = SpreadsheetApp.newFilterCriteria()
.setHiddenValues([''])
.build();
spreadsheet.getActiveSheet().getFilter().setColumnFilterCriteria(4, criteria);
criteria = SpreadsheetApp.newFilterCriteria()
.build();
spreadsheet.getActiveSheet().getFilter().setColumnFilterCriteria(4, criteria);
criteria = SpreadsheetApp.newFilterCriteria()
.setHiddenValues([''])
.build();
spreadsheet.getActiveSheet().getFilter().setColumnFilterCriteria(4, criteria);
spreadsheet.getRange('A1:AP11001').activate();
spreadsheet.getActiveRange().offset(1, 0, spreadsheet.getActiveRange().getNumRows() - 1).sort({column: 4, ascending: false});
spreadsheet.getRange('A:AM').activate();
spreadsheet.getActiveRangeList().setFontFamily('Calibri')
.setHorizontalAlignment('left');
spreadsheet.getRange('P:S').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
spreadsheet.getRange('U:U').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
spreadsheet.getRange('AA:AG').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
spreadsheet.getRange('AL:AL').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right')
.setNumberFormat('"$"#,##0.00');
spreadsheet.getRange('D4').activate();
};
Can anyone possibly point me in the right direction here when it comes to improving this?
Thanks for any help that you can provide here, I look forward to learning further

Tl;Dr:
If you are able to record the macro again, consider to use Go to range or the name box instead of using your mouse to move from one place to anohter in your spreadsheet as each click on a sheet and range adds a statement that activates the corresponding sheet / range. The .activate() methods usually are a lot slower that other options.
The alternative to record the macro again is to remove statements like spreadsheet.getRange(harcoded_ref).activate() and replace statements like spreadsheet.getActiveRangeList() by, i.e.,
spreadsheet.getRange(hardcoded_ref),
spreadsheet.getRangeList(array_of_refs),
etc.
For this, you require some writing-code-skills, JavaScript knowledge and Spreadsheet Service (Class SpreadsheetApp) knowledge. In order to be able achive the best performance possible in Google Apps Script you should consider to use the Advanced Sheets Service, more specifically the spreadsheet.batchUpdate method.
Go to range
Try the following keyboard shortcuts
Windows
Mac
Ctrl + Alt + .Ctrl + Alt + ,
⌘ + Option + .⌘ + Option + ,
If the above keyboard shortcuts don't work for you or you are on a different operative system, to open the Go to side panel, click in the Help menu, then on the search box type Go to, then select Go to range. This will open a "side panel", type the cell reference, then press enter or click on >.
Help menu
Side panel
Name box
Writing code skills and JavaScript knowledge
I suggest you to spend some time on learning the JavaScript basics in order to be able to understand the recorded macro and adapt it to your needs.
Let say that you learned about primitives, objects, properties, classes, methods, literals and variables and understand the very basics of method chaining as the macro recorder used it a lot.
One of the things that you might find that will help to optimize the recorded macros is by assigning objects to variables. I.E. assign Class Sheet object corresponding to the active sheet to the variable named sheet:
var sheet = spreadsheet.getActiveSheet();
Then replace all the spreadsheet.getActiveSheet() by sheet.
In order to improve the performance of your recorded macro, also you should replace
spreadsheet.getRange(something).activate();
by
var rangeSomething = spreadsheet.getRange(something);
then replace the spreadsheet.getRange(something).chain1 before the following spreadsheet.getRange(something).activate(); by rangeSomething.chain1
If you find multiple likes like
spreadsheet.getRange('P:S').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
replace these lines by something like this:
var rangeList = spreadsheet.getRangeList(['P:S','U:U','AA:AG','AL:AL']);
rangeList.setHorizontalAlignment('right');
Related
GOOGLE script 'copyTo values only' does not work when the source is a function (e.g. NOW())

Google Sheets Scripts: Can you get a range from a rangeId?

My google sheet has a cell on sheet1 that contains a link to a cell on sheet2. In my function, I am able to get the link url, but cannot figure out how to get a range from the rangeId:
var link = generatorSheet.getRange(currRow, 2)
var linkUrl = link.getRichTextValue().getLinkUrl()
Logger.log(linkUrl) // linkUrl = "rangeid=1843553975"
I've tried using getRangeByName and various other functions but keep getting a null value back, not a Range object.
Thanks in advance!
Edit: My overall goal in this is to iterate over each row in sheet1, where each cell in column 2 links to a cell in sheet2. I need to take the value from the cell in sheet2 and copy it into sheet3. In sheet1, there's a check box in column 1 of each row, so that's what I'm using to determine whether or not the linked to value will be copied. I'll have a button to kick off my function and populate sheet3, and it has to assume these links are already in place - they were done by hand prior

When you create an hyperlink to a range using the user interface, you are facing this issue. I think you may have to change the way of designing the hyperlink and try to define it by the formula
=hyperlink("#gid=123456789&range=A2","go to ...")
and then you will retrieve the range by
Logger.log(linkUrl.match(/(?<=range=).*/g))

For documentation purposes,
This is a url hash fragment:
#rangeid=1843553975
The id seems to be created, when inserting a link to a range using the user interface. This is distinctly different from a namedRange When clicked, it's appended to the url in the browser, i.e.,https://docs.google.com/spreadsheets/id/edit#rangeid=1843553975. Once appended, through onpopstate javascript event, the range linked to the id is highlighted in the browser/app.
NamedRanges has a similar workflow. It also provides a rangeid=<10 digit ID>. But, it also has a name attached to it. But even in this case, the rangeid is not retrievable, though Sheets API provides a obfuscated range id.
There was a feature request made to Google, but it was made obsolete, because of lack of response on the part of the requestor:
https://issuetracker.google.com/issues/162810351
You may create a new similar issue there with a link to this answer. Once created, link the issue here.
Other related trackers:
https://issuetracker.google.com/issues/129841094
https://issuetracker.google.com/issues/134986436

JS/GAS: For loop to speed up copy from spreadsheet cell, replace document placeholder with value?

I'm using Google App Script. I have a spreadsheet with questions on it for energy auditors about buildings they visit. The auditor is asked to put their answers to the spreadsheet's questions in certain cells. Then they can use a script I wrote to generate a more formal looking Google Document report. The report is generated via these steps: Each cell the auditor inputs an answer into is a defined range. For instance, let's say Cell B10 is defined as "buildingAddress" in spreadsheet. The auditor is asked to put the building address in that cell - let's say he inputs "55 Sample Drive, Portland". When the auditor clicks to generate a Document report, the script runs these lines:
var buildingAddress = sheet.getRangeByName('buildingAddress').getValue();
copyBody.replaceText("<buildingAddress>", buildingAddress);
The place holder in my (Document file) report template is <buildingAddress>. So the code finds this in the report template and replaces it with "55 Sample Drive, Portland" - the value the auditor entered into the spreadsheet cell.
Unfortunately, there are A LOT of such cell values I need to pull from the spreadsheet and push to a placeholder in the report document. They all fit the structure of this:
var buildingAddress = sheet.getRangeByName('buildingAddress').getValue();
copyBody.replaceText("<buildingAddress>", buildingAddress);
So, I'm wondering, can I achieve the same result but use a lot less code by using an array and for loop??? Let's say the array looks like this:
var array = ["buildingAddress", "buildingOwner", "auditorName"];
How do I set up a for loop???
Thank you!!!!!

a loop will not give you gains its exactly the same except cleaner code.
debug it and see where the slow parts are (see execution transcript or log at key steps).
For example if its slow to get a range by name, and all those named ranges are contiguous, instead make a single named range for those cells. get the range (will return array) and get the values from there. this makes a single "get range" call instead of the N you have now.
from your "how to write a loop" question, seems you are just beginning programming. Id suggest a tutorial and more practicing as stackoverflow assumes you know those basics.

Thanks! If anyone's interested, here's what worked for me. I made an array of string objects. Each string was same text as a defined range in my spreadsheet. Then I used this for loop:
for(var i = 0; i < simpleCopyReplaceArray.length; i++){
var definedRangeCellName = simpleCopyReplaceArray[i];
var cellValue = ss.getRangeByName(definedRangeCellName).getValue();
var placeHolder = "<" + definedRangeCellName + ">";
if( cellValue != ""){
copyBody.replaceText(placeHolder, cellValue);
}else{
copyBody.replaceText(placeHolder, "");}
}

Updating Page Items with Respect to Filters in Oracle APEX Interactive Report

I've recently started using Interactive Reports in my Oracle APEX application. Previously, all pages in the application used Classic Reports. The Interactive Report in my new page works great, but, now, I'd like to add a summary box/table above the Interactive Report on the same page that displays the summed values of some of the columns in the Interactive Report. In other words, if my Interactive Report displays 3 distinct manager names, 2 distinct office locations, and 5 different employees, my summary box would contain one row and three columns with the numbers, 3, 2, and 5, respectively.
So far, I have made this work by creating the summary box as a Classic Report that counts distinct values for each column in the same table that my Interactive Report pulls from. The problem arises when I try to filter my interactive report. Obviously, the classic report doesn't refresh based on the interactive report filters, but I don't know how I could link the two so that the classic report responds to the filters from the interactive report. Based on my research, there are ways to reference the value in the Interactive Report's search box using javascript/jquery. If possible, I'd like to reference the value from the interactive table's filter with javascript or jquery in order to refresh the summary box each time a new filter is applied. Does anyone know how to do this?

Don't do javascript parsing on the filters. It's a bad idea - just think on how you would implement this? There's massive amounts of coding to be done and plenty of ajax. And with apex 5 literally around the corner, where does it leave you when the APIs and markup are about to change drastically?
Don't just give in to a requirement either. First make sure how feasible it is technically. And if it's not, make sure you make it abundantly clear what the implications are in regard of time consumption. What is the real value to be had by having these distinct value counts? Maybe there is another way to achieve what they want? Maybe this is nothing more than an attempted solution, and not the core of the real problem. Stuff to think about...
Having said that, here are 2 options:
First method: Count Distinct Aggregates on Interactive reports
You can add these to the IR through the Actions button.
Note though, that this aggregate will be THE LAST ROW! In the example I've posted here, reducing the rows per page to 5 would push the aggregate row to the pagination set 3!
Second Method: APEX_IR and DBMS_SQL
You could use the apex_ir API to retrieve the IR's query and then use that to do a count.
(Apex 4.2) APEX_IR.GET_REPORT
(Apex 5.0) APEX_IR.GET_REPORT
Some pointers:
Retrieve the region ID by querying apex_application_page_regions
Make sure your source query DOES NOT contain #...# substitution strings. (such as #OWNER#.)
Then get the report SQL, rewrite it, and execute it. Eg:
DECLARE
l_report apex_ir.t_report;
l_query varchar2(32767);
l_statement varchar2(32000);
l_cursor integer;
l_rows number;
l_deptno number;
l_mgr number;
BEGIN
l_report := APEX_IR.GET_REPORT (
p_page_id => 30,
p_region_id => 63612660707108658284,
p_report_id => null);
l_query := l_report.sql_query;
sys.htp.prn('Statement = '||l_report.sql_query);
for i in 1..l_report.binds.count
loop
sys.htp.prn(i||'. '||l_report.binds(i).name||' = '||l_report.binds(i).value);
end loop;
l_statement := 'select count (distinct deptno), count(distinct mgr) from ('||l_report.sql_query||')';
sys.htp.prn('statement rewrite: '||l_statement);
l_cursor := dbms_sql.open_cursor;
dbms_sql.parse(l_cursor, l_statement, dbms_sql.native);
for i in 1..l_report.binds.count
loop
dbms_sql.bind_variable(l_cursor, l_report.binds(i).name, l_report.binds(i).value);
end loop;
dbms_sql.define_column(l_cursor, 1, l_deptno);
dbms_sql.define_column(l_cursor, 2, l_mgr);
l_rows := dbms_sql.execute_and_fetch(l_cursor);
dbms_sql.column_value(l_cursor, 1, l_deptno);
dbms_sql.column_value(l_cursor, 2, l_mgr);
dbms_sql.close_cursor(l_cursor);
sys.htp.prn('Distinct deptno: '||l_deptno);
sys.htp.prn('Distinct mgr: '||l_mgr);
EXCEPTION WHEN OTHERS THEN
IF DBMS_SQL.IS_OPEN(l_cursor) THEN
DBMS_SQL.CLOSE_CURSOR(l_cursor);
END IF;
RAISE;
END;
I threw together the sample code from apex_ir.get_report and dbms_sql .
Oracle 11gR2 DBMS_SQL reference
Some serious caveats though: the column list is tricky. If a user has control of all columns and can remove some, those columns will disappear from the select list. Eg in my sample, letting the user hide the DEPTNO column would crash the entire code, because I'd still be doing a count of this column even though it will be gone from the inner query. You could block this by not letting the user control this, or by first parsing the statement etc...
Good luck.

Tallying instances of unique values in an HTML table

I'm looking to count the instances of each unique value in an HTML table and return the results in a table of it's own. The table is generated from a user's text input. So for example the users input might look like this:
Report 46 Bob Marley 4/20/2013 Summary: I shot the sheriff Case #32 User Error
Report 50 Billy The Kid 7/14/2013 Summary: I'm just a boy in a grown up world Case #33 User Experience
Report 51 Oscar The Grouch 10/10/2013 Summary: Refuse, reuse, recycle Case #33 User Experience
where the large spaces are tabs.
Which would return:
<table>
<tr>
<td>Bob Marley</td><td>46</td><td>4/20/2013</td><td>Case #32</td><td>User Error</td>
</tr>
<tr>
<td>Billy The Kid</td><td>50</td><td>4/20/2013</td><td>Case #33</td><td>User Experience</td>
</tr>
<tr>
<td>Oscar The Grouch</td><td>51</td><td>10/10/2013</td><td>Case #33</td><td>User Experience</td>
</tr>
</table>
What I need to do is 1) tally the number of reports, 2) tally the number of times each case number appears, 2) and tally the number of times each category appears and then display it on the next page like such:
Number of reports:
3
Cases:
Case #33 - 2
Case #32 - 1
Categories:
User Experience - 2
User Error - 1
I'm looking for any suggestions on how I might approach this problem. I'm using and learning Javascript/HTML (and jQuery), but would be open to using PHP, SQL, etc. if those tools are more appropriate.
I was thinking of passing the table values to an array and then utilizing a for loop and regexes to count unique values, but I'm not sure if that's the best approach.
EDIT
One more detail that I didn't explicitly state is that I have access to the user input data (i.e. tab-separated text) prior to it being turned into a table. So, if it would be easier to tally the values in question prior to converting it into a table, then please let me know.

As far as PHP, you can store the table HTML into a string and load it into a DOM parser.
http://simplehtmldom.sourceforge.net/
This is what we use for most of our projects involving page scraping, though it would work just as well for parsing your HTML from a string using their function:
$html = str_get_html($yourHtmlString);
Then you can loop through each tr, and from there you can look into each td to add to your tallies.
i.e. to get the category of the third row, you would use:
$html->find("table", 0)->find("tr", 2)->find("td", 4)->plaintext;
You can loop through the table like:
$reportCount = 0;
$reportCases = array();
foreach ($html->find("table", 0)->find("tr") as $tableRow) {
$reportCount++;
$reportCases[] = $tableRow->find("td", 1)->plaintext;
}
etc. though of course also storing all your other necessary data, then formatting it into your table output as needed.