I had this problem which Cooper helped me to solve it (thanks again for that), but now I'm struggling with a different one. The following script will count how many times a client code will appear on another Spreadsheet using as a second condition yesterday date.
function countSheets()
{
var vA = appSh();
var td = Utilities.formatDate(subDaysFromDate(new Date(),2), Session.getScriptTimeZone(), "dd/MM/yyyy");
var mbs=getAllSheets();
//var s='';
for (var i=2;i<vA.length;i++)
{
var d = Utilities.formatDate(new Date(vA[i][12]), Session.getScriptTimeZone(), "dd/MM/yyyy");
for(var key in mbs)
{
if(vA[i][0]==key && d==td)
{
mbs[key]+=1;
}
}
}
return mbs;
}
Then I have the below code which will search in the main spreadsheet (a table) a string and when was found will return row number, also will search for the date yesterday and return the column number. Based on these information I'll get the range where I need to paste the count result from the first script.
function runScript()
{
var ss=SpreadsheetApp.openById('ID');
var mbs=countSheets();
for(var key in mbs)
{
var sh=ss.getSheetByName(key);
var rg=sh.getDataRange();
var vA=rg.getValues();
for(var i=0;i<vA.length;i++)
{
if(vA[i][1]=='Total Number of Applications')
{
var nr=i;
break;//by terminating as soon as we find a match we should get improved performance. Which is something you cant do in a map.
}
}
if(typeof(nr)!='undefined')//If we don't find a match this is undefined
{
var today=subDaysFromDate(new Date(),2).setHours(0,0,0,0);
for(var i=0;i<vA[3].length;i++)
{
if(vA[3][i])//Some cells in this range have no contents
{
if(today.valueOf()==new Date(vA[3][i]).valueOf())
{
sh.getRange(nr+1,i+1,1,1).setValue(Number(mbs[key]));
}
}
}
}
}
return sh;
}
PROBLEM: I have 24 rows on the main Spreadsheet. So I will need to write the same script 24 times. As example, I need to count Total Number of Applications, Total Number of Calls, Number of Live Adverts and so on. If I do this it will exceed execution time since each script takes on average 25 seconds to run.
I did some researches on this website and internet and read about storing values and re-use them over and over. At the moment my script will have to go every time through the same file and count for each condition.
Q1: Is there any chance to create another array that contain all those strings from the second script?
Q2: How to use PropertiesService or anything else to store data and don't have to run over and over getValues() ? I've read Google Documentation but couldn't understand that much from it.
I hope it all make sense and can fix this problem.
My best regards,
Thank you!
My Approach to your Problem
You probably should write it for a couple of rows and then look at the two of them and see what is unique to each one. What is unique about each one is what you have to figure out how to store or access via an external function call.
The issue of time may require that you run these functions separately. I have a dialog which I use to load databases which does exactly that. It loads 800 lines and waits for 10 seconds then loads another 800 lines and wait for ten seconds and keeps doing that until there are no more lines. True it takes about 10 minutes to do this but I can be doing something else while it's working so I don't really care how long it takes. I do care about minimizing my impact to the Google Server though and so I don't run something like this just for fun.
By the way the 10 second delay is external to the gs function.
Related
I am currently running a timer based script (once a week) that copies a couple of values from one range (source) to another (target).
The source range is the result of a query based on a import. Thus it goes through multiple calculation steps before displaying the proper result.
Usually this should be an easy task - just add a sleep timer to be sure and be done with it.
However, the script copies the data based on an unfinished calculation.
Here's how I approached the this:
I setup a cell that checks if the data was correctly copied (trigger cell)
Now, I could just run a timer on sunday every hour or so, checking if the trigger is true or false. But I was wondering if there is a way to do this with a loop.
I am fairly new to javascript so I'm not very confident with implementing possible solutions from the web.
It seems that loops can only be broken based on values within the loop. A do/while (while 'trigger' == false) loop for example just loops for 5 Minutes until it times out. I think a function call with the function calling itself should do the trick, but I couldn't figure out how to do this properly. My version just seems to run once and break - even when I change the triggervalue manually to force a loop.
function looper(){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("example");
var trigger = sheet.getRange("a1").getValue();
var source = sheet.getRange("b1:j1").getValues();
var target = sheet.getRange("b2:j2");
Utilities.sleep(5000)
if (trigger == false){
target.setValues(source);
looper;
}
}
In JavaScript, to call a function you need to use parentheses. So for the last line, try doing this: looper(); instead.
I have written a very simple code on my Google Sheets file. This is the purpose:
Save some cells values from StaticSheet (all the Copyxxx) that need to be copied in DynamicSheet.
Get the value of one specific cell inserted by the user manually.
Enter a While loop useful only to increase an indicator and get the number of the row where I want to copy those values previously saved.
Copy those values on this row but different columns.
The problem is that it seems that most of the time it does not even run the script after I told it to do so.
What is funny is that sometimes it works, super slowly, but it works for like a couple of minutes. And after it stops working again.
Could you please tell me what am I missing here please?
function Copy_Static_on_Dynamic() {
var app = SpreadsheetApp;
var ss = app.openById("xxxxyy--------yyzzzz")
var StaticSheet = ss.getSheetByName("DEAT Price");
var DynamicSheet = ss.getSheetByName("DEAT Price + TEST");
var CopySKU = StaticSheet.getRange(5,1,40);
var CopyPrices = StaticSheet.getRange(5,3,40,4);
var CopyUsage = StaticSheet.getRange(5,8,40);
var Week_1 = StaticSheet.getRange(2,4).getValues();
var i = 1;
Logger.clear();
while(DynamicSheet.getRange(i,3).getValues() != Week_1)
{
Logger.log(i);
i+=1;
}
CopySKU.copyTo(DynamicSheet.getRange(i,4,40));
CopyPrices.copyTo(DynamicSheet.getRange(i,6,40,4));
CopyUsage.copyTo(DynamicSheet.getRange(i,11,40));
}
If you see the "Preparing for Execution" message in the Apps Script editor, you can reload the browser window and run the function again. The program will likely go away.
So I think I have solved it.
As Serge insas was saying I had my script running on the background, I found it out on the "Execution" section, where you can also interrupt them.
After I discover it I kept testing, and I saw that the while loop needed almost 2 seconds to check the condition every time, making the script incredibly long.
So instead of:
while(DynamicSheet.getRange(i,3).getValues() != Week_1)
... I have created a variable declared previously such as:
var WeekLOOP = DynamicSheet.getRange(i,3).getValues();
while(WeekLOOP != Week_1) { --- }
... and now the script needs few milliseconds to run the condition. I don't have enough technical knowledge to say if this was the only issue, but is what apparently solved my problem.
Thanks to all for the support! Will come back if I need any further help :)
As was mentioned by Amit Agarwal, to solve the error message mentioned on the question, refresh the web browser tab.
Regarding the code,
On
var Week_1 = StaticSheet.getRange(2,4).getValues();
and
DynamicSheet.getRange(i,3).getValues()
instead of getValues you should use getValue because your code are referring to single cell cells otherwise you will be getting 2D arrays instead of scalar values.
The use of while should be made very carefully to avoid functions running endlessly. You could add some "safeguard" like the following
var max_iterations = 100 // Edit this
while(DynamicSheet.getRange(i,3).getValue() != Week_1 && i <= max_iterations) {
I keep getting this red lightbulb message in the sheets script editor every time it runs. Is there something wrong with the code? It seems to work and doesn't take very long to execute.
Method Range.getValue is heavily used by the script
The script uses a method which is considered expensive. Each invocation generates a time consuming call to a remote server. That may have critical impact on the execution time of the script, especially on large data. If performance is an issue for the script, you should consider using another method, e.g. Range.getValues()
function Merge() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName('Expense Index');
var lastrow=sheet.getLastRow();
var range = sheet.getRange("A1:F"+(lastrow+1));
var k=3;
for (var i = 4; i <=lastrow+1; i++) {
var val1=range.getCell(i-1, 1).getValue();
var val2=range.getCell(i, 1).getValue();
if(val1!==val2){
if(!(sheet.getRange('A'+k+':A'+(i-1)).isPartOfMerge())){
sheet.getRange('A'+k+':A'+(i-1)).mergeVertically();
range.getCell((i-1), 6).setValue('=SUM(E'+k+':E'+(i-1)+')');
}
k=i;
}
}
sheet.getRange("A1:F"+(lastrow+1)).setHorizontalAlignment('center');
sheet.getRange("A1:F"+(lastrow+1)).setVerticalAlignment('middle')
}
Google Apps Script has best practices recommended by the development team. Making repeat range calls is expensive in operation. It's much better to batch your range data and then iterate the Object that is returned.
Right now, you're requesting 2 Objects on each loop. It's more efficient to get all the data as a single object. So, change:
var range = sheet.getRange("A1:F"+(lastrow+1));
to
// range.getValues() returns a 2D array you can loop through in one call.
var range = sheet.getRange("A1:F"+(lastrow+1)).getValues();
and loop through the object using bracket notation.
I am developping an embedded website contained within a windows CE device.
See it as if it was the web configuration interface of your router.
Everything is contained within a really small footprint of memory, the entire website is less than 500KB with every script, html, css and icons.
We have to assume that the user that is going to 'browse' into that interface does not have access to the internet (LAN only) so no 'online' solution here.
I am looking for a solution so the user choose his timezone and the code will get all the DST/STD times and dates for the next 10-20 years at least and downloaded them to the device that will run autonomously after that and at specific dates, will change its time to DST/STD by itself. Current application is custom (not windowce api related) and needs the DST/STD date pairs.
iana.org is maintaining a db for like every location in the world. I also saw that moment-timezone (javascript interface) is 'packaging' this data in a very compact package 25kb zipped.
I need to know if it is possible to:
'Extract' a main tz list from this DB so the user can choose its own tz. I looked at their doc but example didn't work:
var itsTimeZones = moment.tz.names();
2- Extract the next 10-20 years of DST/STD dates/times for a chosen zone ? I haven't saw any documentation anywhere on this topic. But since it's kind of the purpose of such a database, i would say it must be burried in there somewhere, need a way to dig it out.
3- If moment timezone is not the right track to go, anybody has a solution that will fullfill that requirement ?
Thanxs
My solution for extracting DST list for a given zone from moment-timezone.
1. "theTz" is a numerical index from a change on a select in the webpage.
2. select is populated with "List".
var itsTz =
{
"List" : moment.tz.names(), // Get all zone 'Names'
"Transitions" : [],
};
function TimeZoneChanged(theTz)
{
var myZone = moment.tz.zone(itsTz.List[theTz]); // Get zone by its name from List
var myTime = moment.tz(moment(), myZone.name); // Start today
var myResult;
// Build transition list according to new zone selected
itsTz.Transitions = [];
do
{
myResult = GetNextTransition(myZone, myTime);
if(myResult != null)
{
itsTz.Transitions.push(moment(myResult).format("YYYY/MM/DD"));
myTime = moment.tz(myResult, myZone.name); // Get next from date found
}
}while(myResult != null);
}
function GetNextTransition(theZone, theTime)
{
var myResult;
for(var i = 0; i < theZone.untils.length; i++)
{
if(theZone.untils[i] > theTime)
{
theZone.untils[i] == Infinity ? myResult = null : myResult = new moment(theZone.untils[i]).format();
return myResult;
}
}
}
Results for 'America/Toronto:
2017/03/12,
2017/11/05,
2018/03/11,
2018/11/04,
2019/03/10,
2019/11/03,
2020/03/08,
2020/11/01,
2021/03/14,
2021/11/07,
2022/03/13,
2022/11/06,
2023/03/12,
2023/11/05,
2024/03/10,
2024/11/03,
2025/03/09,
2025/11/02,
2026/03/08,
2026/11/01,
2027/03/14,
2027/11/07,
2028/03/12,
2028/11/05,
2029/03/11,
2029/11/04,
2030/03/10,
2030/11/03,
2031/03/09,
2031/11/02,
2032/03/14,
2032/11/07,
2033/03/13,
2033/11/06,
2034/03/12,
2034/11/05,
2035/03/11,
2035/11/04,
2036/03/09,
2036/11/02,
2037/03/08,
2037/11/01,
The following code is working but extremely slow. Up till the search function all goes well. First, the search function returns a sequence and not an array (why?!). Second, the array consists of nodes and I need URI's for the delete. And third, the deleteDocument function takes a string and not an array of URI's.
What would be the better way to do this? I need to delete year+ old documents.
Here I use xdmp.log in stead of document.delete just te be safe.
var now = new Date();
var yearBack = now.setDate(now.getDate() - 365);
var date = new Date(yearBack);
var b = cts.jsonPropertyRangeQuery("Dtm", "<", date);
var c = cts.search(b, ['unfiltered']).toArray();
for (i=0; i<fn.count(c); i++) {
xdmp.log(fn.documentUri(c[i]), "info");
};
Doing the same with cts.uris:
var now = new Date();
var yearBack = now.setDate(now.getDate() - 365);
var date = new Date(yearBack);
var b = cts.jsonPropertyRangeQuery("Dtm", "<", date);
var c = cts.uris("", [], b);
while (true) {
var uri = c.next();
if (uri.done == true){
break;
}
xdmp.log(uri.value, "info");
}
HTH!
Using toArray will work but is most likely were your slowness is. The cts.search() function returns an iterator. So All you have to do is loop over it and do your deleting until there is no more items in it. Also You might want to limit your search to 1,000 items. A transaction with a large number of deletes will take a while and might time out.
Here is an example of looping over the iterator
var now = new Date();
var yearBack = now.setDate(now.getDate() - 365);
var date = new Date(yearBack);
var b = cts.jsonPropertyRangeQuery("Dtm", "<", date);
var c = cts.search(b, ['unfiltered']);
while (true) {
var doc = c.next();
if (doc.done == true){
break;
}
xdmp.log(fn.documentUri(doc), "info");
}
here is an example if you wanted to limit to the first 1,000.
fn.subsequence(cts.search(b, ['unfiltered']), 1, 1000);
Several things to consider.
1) If you are searching for the purpose of deleting or anything that doesnt require the document body, using a search that returns URIs instead of nodes can be much faster. If that isnt convenient then getting the URI as close to the search expression can achieve similar results. You want to avoid having the server have to fetch and expand the document just to get the URI to delete it.
2) While there is full coverage in the JavaScript API's for all MarkLogic features, the JavaScript API's are based on the same underlying functions that the XQuery API's use. Its useful to understand that, and take a look at the equivalent XQuery API docs to get the big picture. For example Arrays vs Iterators - If the JS search API's returned Arrays it could be a huge performance problem because the underlying code is based on 'lazy evaluation' of sequences. For example a search could return 1 million rows but if you only look at the first one the server can often avoid accessing the remaining 999,999,999 documents. Similarly, as you iterate only the in scope referenced data needs to be in available. If they had to be put into an array then all results would have to be pre-fetched and put put in memory upfront.
3) Always keep in mind that operations which return lists of things may only be bounded by how big your database is. That is why cts.search() and other functions have built in 'pagination'. You should code for that from the start.
By reading the users guides you can get a better understanding of not only how to do something, but how to do it efficiently - or even at all - once your database becomes larger than memory. In general its a good idea to always code for paginated results - it is a lot more efficient and your code will still work just as well after you add 100 docs or a million.
4) take a look at xdmp.nodeUrl https://docs.marklogic.com/xdmp.nodeUri,
This function, unlike fn.documentUri(), will work on any node even if its not document node. If you can put this right next to the search instead of next to the delete then the system can optimize much better. The examples in the JavaScript guide are a good start https://docs.marklogic.com/guide/getting-started/javascript#chapter
In your case I suggest something like this to experiment with both pagination and extracting the URIs without having to expand the documents ..
var uris = []
for (var result of fn.subsequence(cts.search( ... ), 1 , 100 )
uris.push(xdmp.nodeUri(result))
for( i in uris )
xdmp.log( uris[i] )