Context
I have a g-sheet that acts as a sort of "master sheet" in which everything pours into from a bunch of other outside spreadsheets that are all consistently being live updated throughout the day.
Each outside spreadsheet I connect, routes to its own tab within our master spreadsheet through importrange function
All those tabs then route to one master tab using row ID #'s - so that everyone can just work from that tab.
The Problem
In this master tab where everything lands, I have a macro sorting the rows to bring the most recent rows to the top, among other things to keep the data clean. As I connect more sheets over time, I add to the number in the macro to accommodate new rows.
Macro a couple days ago started throwing "Service Spreadsheet timed out while accessing document with id..." then the id is the id # of the master tab itself.
Know there is probably a lot smoother way to have this done without using a large bandwidth macro in place, but optimizing the script to best fit the use-case is far out of my experience level. The macro I have in place is as follows:
function MasterSormat2() {
var spreadsheet = SpreadsheetApp.getActive();
spreadsheet.getRange('D1').activate();
var criteria = SpreadsheetApp.newFilterCriteria()
.setHiddenValues([''])
.build();
spreadsheet.getActiveSheet().getFilter().setColumnFilterCriteria(4, criteria);
criteria = SpreadsheetApp.newFilterCriteria()
.build();
spreadsheet.getActiveSheet().getFilter().setColumnFilterCriteria(4, criteria);
criteria = SpreadsheetApp.newFilterCriteria()
.setHiddenValues([''])
.build();
spreadsheet.getActiveSheet().getFilter().setColumnFilterCriteria(4, criteria);
spreadsheet.getRange('A1:AP11001').activate();
spreadsheet.getActiveRange().offset(1, 0, spreadsheet.getActiveRange().getNumRows() - 1).sort({column: 4, ascending: false});
spreadsheet.getRange('A:AM').activate();
spreadsheet.getActiveRangeList().setFontFamily('Calibri')
.setHorizontalAlignment('left');
spreadsheet.getRange('P:S').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
spreadsheet.getRange('U:U').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
spreadsheet.getRange('AA:AG').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
spreadsheet.getRange('AL:AL').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right')
.setNumberFormat('"$"#,##0.00');
spreadsheet.getRange('D4').activate();
};
Can anyone possibly point me in the right direction here when it comes to improving this?
Thanks for any help that you can provide here, I look forward to learning further
Tl;Dr:
If you are able to record the macro again, consider to use Go to range or the name box instead of using your mouse to move from one place to anohter in your spreadsheet as each click on a sheet and range adds a statement that activates the corresponding sheet / range. The .activate() methods usually are a lot slower that other options.
The alternative to record the macro again is to remove statements like spreadsheet.getRange(harcoded_ref).activate() and replace statements like spreadsheet.getActiveRangeList() by, i.e.,
spreadsheet.getRange(hardcoded_ref),
spreadsheet.getRangeList(array_of_refs),
etc.
For this, you require some writing-code-skills, JavaScript knowledge and Spreadsheet Service (Class SpreadsheetApp) knowledge. In order to be able achive the best performance possible in Google Apps Script you should consider to use the Advanced Sheets Service, more specifically the spreadsheet.batchUpdate method.
Go to range
Try the following keyboard shortcuts
Windows
Mac
Ctrl + Alt + .Ctrl + Alt + ,
⌘ + Option + .⌘ + Option + ,
If the above keyboard shortcuts don't work for you or you are on a different operative system, to open the Go to side panel, click in the Help menu, then on the search box type Go to, then select Go to range. This will open a "side panel", type the cell reference, then press enter or click on >.
Help menu
Side panel
Name box
Writing code skills and JavaScript knowledge
I suggest you to spend some time on learning the JavaScript basics in order to be able to understand the recorded macro and adapt it to your needs.
Let say that you learned about primitives, objects, properties, classes, methods, literals and variables and understand the very basics of method chaining as the macro recorder used it a lot.
One of the things that you might find that will help to optimize the recorded macros is by assigning objects to variables. I.E. assign Class Sheet object corresponding to the active sheet to the variable named sheet:
var sheet = spreadsheet.getActiveSheet();
Then replace all the spreadsheet.getActiveSheet() by sheet.
In order to improve the performance of your recorded macro, also you should replace
spreadsheet.getRange(something).activate();
by
var rangeSomething = spreadsheet.getRange(something);
then replace the spreadsheet.getRange(something).chain1 before the following spreadsheet.getRange(something).activate(); by rangeSomething.chain1
If you find multiple likes like
spreadsheet.getRange('P:S').activate();
spreadsheet.getActiveRangeList().setHorizontalAlignment('right');
replace these lines by something like this:
var rangeList = spreadsheet.getRangeList(['P:S','U:U','AA:AG','AL:AL']);
rangeList.setHorizontalAlignment('right');
Related
GOOGLE script 'copyTo values only' does not work when the source is a function (e.g. NOW())
Related
My google sheet has a cell on sheet1 that contains a link to a cell on sheet2. In my function, I am able to get the link url, but cannot figure out how to get a range from the rangeId:
var link = generatorSheet.getRange(currRow, 2)
var linkUrl = link.getRichTextValue().getLinkUrl()
Logger.log(linkUrl) // linkUrl = "rangeid=1843553975"
I've tried using getRangeByName and various other functions but keep getting a null value back, not a Range object.
Thanks in advance!
Edit: My overall goal in this is to iterate over each row in sheet1, where each cell in column 2 links to a cell in sheet2. I need to take the value from the cell in sheet2 and copy it into sheet3. In sheet1, there's a check box in column 1 of each row, so that's what I'm using to determine whether or not the linked to value will be copied. I'll have a button to kick off my function and populate sheet3, and it has to assume these links are already in place - they were done by hand prior
When you create an hyperlink to a range using the user interface, you are facing this issue. I think you may have to change the way of designing the hyperlink and try to define it by the formula
=hyperlink("#gid=123456789&range=A2","go to ...")
and then you will retrieve the range by
Logger.log(linkUrl.match(/(?<=range=).*/g))
For documentation purposes,
This is a url hash fragment:
#rangeid=1843553975
The id seems to be created, when inserting a link to a range using the user interface. This is distinctly different from a namedRange When clicked, it's appended to the url in the browser, i.e.,https://docs.google.com/spreadsheets/id/edit#rangeid=1843553975. Once appended, through onpopstate javascript event, the range linked to the id is highlighted in the browser/app.
NamedRanges has a similar workflow. It also provides a rangeid=<10 digit ID>. But, it also has a name attached to it. But even in this case, the rangeid is not retrievable, though Sheets API provides a obfuscated range id.
There was a feature request made to Google, but it was made obsolete, because of lack of response on the part of the requestor:
https://issuetracker.google.com/issues/162810351
You may create a new similar issue there with a link to this answer. Once created, link the issue here.
Other related trackers:
https://issuetracker.google.com/issues/129841094
https://issuetracker.google.com/issues/134986436
I'm making a volunteers' self-scheduler using Google Sheets. On one tab, the user enters a location and the time of their shift, formatted like so:
NYC: 3/18/18, 11-2pm; LA: 3/19/18, 7-8pm; HOME: 3/19/18, 8-10pm;
SANF: 3/20/18, 9-11AM;
and on another tab, I have made a table that reads that information and displays the volunteer's name and info under the corresponding cells for that scheduling shift and location.
=IFERROR(TRIM(SUBSTITUTE(CONCATENATE(ARRAYFORMULA(QUERY({Shifts!$A$2:$A, Shifts!$B$2:$B, Vols!$C$2:$C, Vols!$D$2:$D, Vols!$G$2:$G}, "select Col1, Col2, Col4, Col5 where Col3 matches '.*(NYC: "&C$2&", "&C$3&").*'", 0) &"|")),"|",char(10))))
There's a row for every major location. In the query above, C$2 and C$3 are the date and time. However, I would like to make a "misc" row to display volunteer shifts in locations that aren't common enough to have their own rows. This row would exclude all the big locations.
I wrote a regex that almost expresses what I want, but unfortunately it appears that Google Sheets does not seem to support lookarounds yet. Is there a reasonably concise way to recreate this without a lookbehind? I'm also happy to find a more elegant solution to the problem!
/.*(?<!NYC: |LA: |SANF:)3\/18\/18, 11-2pm.*/
Scenario
I have web interface (in a large web application) that allows a user to make a connection between two very large lists.
List A - 40,000+ items
List B - 1,000+ items
List C - Contains a list of items in b that are connected to the selected item in list A
The Code
Here is a rough jsfiddle of the current behavior minus the ajax update of the database.
Here is the primary functionality (only here because stack overflow requires a code snippet for jsfiddle links).
$('.name-listb input').add('.name-listc input').click(function (e) {
var lista_id = $('.name-lista input:checked').val();
var listb_id = $(this).val();
var operation = $(this).prop('checked') ? 'create' : 'delete';
var $listb = $('.name-listb .checkbox-list');
var $listc = $('.name-listc .checkbox-list');
if (operation == 'create') {
$listb.find('input[value=' + listb_id + ']').prop('checked', true);
// Ajax request to add checked item.
$new_item = $listb.find('input[value=' + listb_id + ']').parents('.option-group').clone();
$listc.append($new_item);
} else if (operation == 'delete') {
console.log('hello list delete');
$listb.find('input[value=' + listb_id + ']').prop('checked', false);
// Ajax request to remove checked item.
$listc.find('input[value=' + listb_id + ']').parents('.option-group').remove();
}
});
The Problem
The requirements do not allow for me to use an auto complete field or pager. But the current page takes way too long to load (between 1 and 5sec depending on caching). Also the JS behaviors are attached to all 40k+ items which will cause problems on lower performance computers (Tested on a newish $200 consumer special and the computer was crippled by the JS). There is also (not on JS fiddle but the final product) a filter that filters the list down based on text input.
The Question
What is a good strategy for handling this scenario?
My Idea
My first thought was to create a sort of document view architecture. A JavaScript list that adds items to the top and bottom as the user scrolls and dumps items off the other end when the list reaches a certain size. To filter I would dump the whole list and obtain a new list of filtered items like an auto-complete but it would be able to scroll and add items using ajax. But this is very complicated. I was hoping someone might have a better idea or a jquery plugin that already uses this approach.
Update
Lista is 70K Fixed
Listb is User generated and will span between 1k-70k.
That said just optimizing the JS with the excellent feedback of using delegates (which will make life 10x more awesome), won't be enough. Still need to limit the visible list.
Your Ideas?
I've encountered this issue on numerous projects before and one solution that's both easy to implement and well performing is using something like Infinity.js.
To summarize shortly, Infinity, like many other "infinite scroll" libraries, allows you to render only a small part of the actual list that should be visible (or should be visible soon), thus reducing the strain on the browser tremendously. You can see a simple live demo over here, check the first link for the API reference.
This is not impossible to do, the trick is NOT to load all that stuff onto the DOM because it will wreck any Java Script Engine.
The answer is using the d3js base library, which is the king amongst sorting extremely large data on client side, whether it is tabular or graphical. When I first saw it it had 4 examples, now there are pages and pages.
This is one of the first examples provided by d3, crossfilter.
The dataset if 5.3megabytes! And it filters data in milliseconds, and it promises to sort millions of rows without a performance loss.
I'm using Google App Script. I have a spreadsheet with questions on it for energy auditors about buildings they visit. The auditor is asked to put their answers to the spreadsheet's questions in certain cells. Then they can use a script I wrote to generate a more formal looking Google Document report. The report is generated via these steps: Each cell the auditor inputs an answer into is a defined range. For instance, let's say Cell B10 is defined as "buildingAddress" in spreadsheet. The auditor is asked to put the building address in that cell - let's say he inputs "55 Sample Drive, Portland". When the auditor clicks to generate a Document report, the script runs these lines:
var buildingAddress = sheet.getRangeByName('buildingAddress').getValue();
copyBody.replaceText("<buildingAddress>", buildingAddress);
The place holder in my (Document file) report template is <buildingAddress>. So the code finds this in the report template and replaces it with "55 Sample Drive, Portland" - the value the auditor entered into the spreadsheet cell.
Unfortunately, there are A LOT of such cell values I need to pull from the spreadsheet and push to a placeholder in the report document. They all fit the structure of this:
var buildingAddress = sheet.getRangeByName('buildingAddress').getValue();
copyBody.replaceText("<buildingAddress>", buildingAddress);
So, I'm wondering, can I achieve the same result but use a lot less code by using an array and for loop??? Let's say the array looks like this:
var array = ["buildingAddress", "buildingOwner", "auditorName"];
How do I set up a for loop???
Thank you!!!!!
a loop will not give you gains its exactly the same except cleaner code.
debug it and see where the slow parts are (see execution transcript or log at key steps).
For example if its slow to get a range by name, and all those named ranges are contiguous, instead make a single named range for those cells. get the range (will return array) and get the values from there. this makes a single "get range" call instead of the N you have now.
from your "how to write a loop" question, seems you are just beginning programming. Id suggest a tutorial and more practicing as stackoverflow assumes you know those basics.
Thanks! If anyone's interested, here's what worked for me. I made an array of string objects. Each string was same text as a defined range in my spreadsheet. Then I used this for loop:
for(var i = 0; i < simpleCopyReplaceArray.length; i++){
var definedRangeCellName = simpleCopyReplaceArray[i];
var cellValue = ss.getRangeByName(definedRangeCellName).getValue();
var placeHolder = "<" + definedRangeCellName + ">";
if( cellValue != ""){
copyBody.replaceText(placeHolder, cellValue);
}else{
copyBody.replaceText(placeHolder, "");}
}
I've recently started using Interactive Reports in my Oracle APEX application. Previously, all pages in the application used Classic Reports. The Interactive Report in my new page works great, but, now, I'd like to add a summary box/table above the Interactive Report on the same page that displays the summed values of some of the columns in the Interactive Report. In other words, if my Interactive Report displays 3 distinct manager names, 2 distinct office locations, and 5 different employees, my summary box would contain one row and three columns with the numbers, 3, 2, and 5, respectively.
So far, I have made this work by creating the summary box as a Classic Report that counts distinct values for each column in the same table that my Interactive Report pulls from. The problem arises when I try to filter my interactive report. Obviously, the classic report doesn't refresh based on the interactive report filters, but I don't know how I could link the two so that the classic report responds to the filters from the interactive report. Based on my research, there are ways to reference the value in the Interactive Report's search box using javascript/jquery. If possible, I'd like to reference the value from the interactive table's filter with javascript or jquery in order to refresh the summary box each time a new filter is applied. Does anyone know how to do this?
Don't do javascript parsing on the filters. It's a bad idea - just think on how you would implement this? There's massive amounts of coding to be done and plenty of ajax. And with apex 5 literally around the corner, where does it leave you when the APIs and markup are about to change drastically?
Don't just give in to a requirement either. First make sure how feasible it is technically. And if it's not, make sure you make it abundantly clear what the implications are in regard of time consumption. What is the real value to be had by having these distinct value counts? Maybe there is another way to achieve what they want? Maybe this is nothing more than an attempted solution, and not the core of the real problem. Stuff to think about...
Having said that, here are 2 options:
First method: Count Distinct Aggregates on Interactive reports
You can add these to the IR through the Actions button.
Note though, that this aggregate will be THE LAST ROW! In the example I've posted here, reducing the rows per page to 5 would push the aggregate row to the pagination set 3!
Second Method: APEX_IR and DBMS_SQL
You could use the apex_ir API to retrieve the IR's query and then use that to do a count.
(Apex 4.2) APEX_IR.GET_REPORT
(Apex 5.0) APEX_IR.GET_REPORT
Some pointers:
Retrieve the region ID by querying apex_application_page_regions
Make sure your source query DOES NOT contain #...# substitution strings. (such as #OWNER#.)
Then get the report SQL, rewrite it, and execute it. Eg:
DECLARE
l_report apex_ir.t_report;
l_query varchar2(32767);
l_statement varchar2(32000);
l_cursor integer;
l_rows number;
l_deptno number;
l_mgr number;
BEGIN
l_report := APEX_IR.GET_REPORT (
p_page_id => 30,
p_region_id => 63612660707108658284,
p_report_id => null);
l_query := l_report.sql_query;
sys.htp.prn('Statement = '||l_report.sql_query);
for i in 1..l_report.binds.count
loop
sys.htp.prn(i||'. '||l_report.binds(i).name||' = '||l_report.binds(i).value);
end loop;
l_statement := 'select count (distinct deptno), count(distinct mgr) from ('||l_report.sql_query||')';
sys.htp.prn('statement rewrite: '||l_statement);
l_cursor := dbms_sql.open_cursor;
dbms_sql.parse(l_cursor, l_statement, dbms_sql.native);
for i in 1..l_report.binds.count
loop
dbms_sql.bind_variable(l_cursor, l_report.binds(i).name, l_report.binds(i).value);
end loop;
dbms_sql.define_column(l_cursor, 1, l_deptno);
dbms_sql.define_column(l_cursor, 2, l_mgr);
l_rows := dbms_sql.execute_and_fetch(l_cursor);
dbms_sql.column_value(l_cursor, 1, l_deptno);
dbms_sql.column_value(l_cursor, 2, l_mgr);
dbms_sql.close_cursor(l_cursor);
sys.htp.prn('Distinct deptno: '||l_deptno);
sys.htp.prn('Distinct mgr: '||l_mgr);
EXCEPTION WHEN OTHERS THEN
IF DBMS_SQL.IS_OPEN(l_cursor) THEN
DBMS_SQL.CLOSE_CURSOR(l_cursor);
END IF;
RAISE;
END;
I threw together the sample code from apex_ir.get_report and dbms_sql .
Oracle 11gR2 DBMS_SQL reference
Some serious caveats though: the column list is tricky. If a user has control of all columns and can remove some, those columns will disappear from the select list. Eg in my sample, letting the user hide the DEPTNO column would crash the entire code, because I'd still be doing a count of this column even though it will be gone from the inner query. You could block this by not letting the user control this, or by first parsing the statement etc...
Good luck.