I am trying to generate a unique number when a form is submitted. I have simplified my script to the following for testing.
function onFormSubmit(event) {
// Get a script lock, because we're about to modify a shared resource.
var lock = LockService.getScriptLock();
// Wait for up to 30 seconds for other processes to finish.
lock.waitLock(30000);
var ticketNumber = Number(ScriptProperties.getProperty('lastTicketNumber')) + 1;
ScriptProperties.setProperty('lastTicketNumber', ticketNumber);
targetCell = event.range.offset(0, event.range.getNumColumns(), 1, 1);
targetCell.setValue(ticketNumber);
SpreadsheetApp.flush();
// Release the lock so that other processes can continue.
lock.releaseLock();
};
I find that if I submit two forms within a second of each other I get the same ticketnumber.
Any help would be appreciated.
Preface: You are using the deprecated ScriptProperties service, this has been replaced by the Properties Service. You probably want to change this first.
In the past I have received the same results when trying to utilize the project properties in rapid succession. It's almost as if the old value hangs around in whatever caching apps script uses for script properties for a few seconds.
I would recommend utilizing the Cache Service to suppliment for scripts that need to reflect changes immediately.
Modified Code:
function onFormSubmit(event) {
// Get a script lock, because we're about to modify a shared resource.
var lock = LockService.getScriptLock();
// Wait for up to 30 seconds for other processes to finish.
lock.waitLock(30000);
var scriptCache = CacheService.getScriptCache();
var scriptProperties = PropertiesService.getScriptProperties();
var cachedTicketNumber = scriptCache.get('lastTicketNumber');
var ticketNumber;
if(cachedTicketNumber !== null){
ticketNumber = Number(cachedTicketNumber) + 1;
} else {
//Cache has expired/does not exist, fall back to properties service
ticketNumber = Number(scriptProperties.getProperty('lastTicketNumber')) + 1;
}
//Set properties service, and cache values of the ticket number
scriptProperties.setProperty('lastTicketNumber', ticketNumber);
scriptCache.put('lastTicketNumber', ticketNumber, 21600); //21600 seconds = 6 hours, if you want it to last that long
targetCell = event.range.offset(0, event.range.getNumColumns(), 1, 1);
targetCell.setValue(ticketNumber);
SpreadsheetApp.flush();
// Release the lock so that other processes can continue.
lock.releaseLock();
};
Related
I wonder if someone could help me.
I have a non-professional development license for a reverse geocoder within a javascript program where I am only allowed to do two requests
per second otherwise I get a 429 error. I have 3 sets of co-ordinates I wish to feed into the reverse geocoder and I get the first two
processed correctly but after that I get an error and the third one isn't processed. I thought that if I used the SetTimeout function either in the for
loop or in one of the lower level functions this would delay the requests enough to be able to process all 3 addresses but no matter where I
place the SetTimeout function it continues to get the 429 error. When I log the time to the console, I can see that the three calls to the
reverse geocoder happen at the same time. Can anyone suggest where I can place the timeout to slow down the requests enough?
Thanks (last attempted version of code below)
for (let i = 0; i < mapMarkers.length; i++){
// use a reverse geocode function to build a display address for each of the co-ordinates chosen
SetTimeout(reverseGeocode(mapMarkers[i].getLatLng()), 1000);
};
function reverseGeocode(coords){
var today = new Date();
var time = today.getHours() + ":" + today.getMinutes() + ":" + today.getSeconds();
console.log("Into reverse geocoder " + time);
let REVERSEURL = `https:....`
let TOKEN = '.....'
let url = `${REVERSEURL}key=${TOKEN}&lat=${coords.lat}&lon=${coords.lng}`;
//do a reverse geocoding call
getData(url, coords);
}
async function getData(url,coords) {
try {
const data = await getRequest(url);
// create a display address with the first three elements containing something in the address dictionary
let address = createAddressString(data.address, 3) +
" ( " + coords.lat.toFixed(5) + ", " + coords.lng.toFixed(5) + " ) ";
// Insert a div containing the address just built
$("#addresses-container").append("<div class='request-page'>"+address+'</div>');
} catch(e) {
console.log(e);
}
}
async function getRequest(url) {
const res = await fetch(url);
if (res.ok) {
return res.json();
} else {
throw new Error("Bad response");
}
}
Your current logic is invoking the reverseGeocode() method immediately, as you pass the response from that function call to the timeout. You need to provide a function reference instead.
Even if you correct that issue, then you would instead delay all the requests by 1 second, but they would still get fired at the same time.
To stagger them you can use the index of the iteration to multiply the delay. For example, the following logic will fire 1 request every 250ms. This delay can be amended depending on what the rate limit is of your API provider. Also note that SetTimeout() needs to be setTimeout()
for (let i = 0; i < mapMarkers.length; i++) {
setTimeout(() => reverseGeocode(mapMarkers[i].getLatLng()), 250 * i);
}
Aside from the problem, it would be worth checking if the API can accept multiple lookups in a single request, which will alleviate the issue. Failing that, I'd suggest finding an alternative provider which allows more than 3 requests per N period.
I have a web socket that receives data from a web socket server every 100 to 200ms, ( I have tried both with a shared web worker as well as all in the main.js file),
When new JSON data arrives my main.js runs filter_json_run_all(json_data) which updates Tabulator.js & Dygraph.js Tables & Graphs with some custom color coding based on if values are increasing or decreasing
1) web socket json data ( every 100ms or less) -> 2) run function filter_json_run_all(json_data) (takes 150 to 200ms) -> 3) repeat 1 & 2 forever
Quickly the timestamp of the incoming json data gets delayed versus the actual time (json_time 15:30:12 vs actual time: 15:31:30) since the filter_json_run_all is causing a backlog in operations.
So it causes users on different PC's to have websocket sync issues, based on when they opened or refreshed the website.
This is only caused by the long filter_json_run_all() function, otherwise if all I did was console.log(json_data) they would be perfectly in sync.
Please I would be very very grateful if anyone has any ideas how I can prevent this sort of blocking / backlog of incoming JSON websocket data caused by a slow running javascript
function :)
I tried using a shared web worker which works but it doesn't get around the delay in main.js blocked by filter_json_run_all(), I dont thing I can put filter_json_run_all() since all the graph & table objects are defined in main & also I have callbacks for when I click on a table to update a value manually (Bi directional web socket)
If you have any ideas or tips at all I will be very grateful :)
worker.js:
const connectedPorts = [];
// Create socket instance.
var socket = new WebSocket(
'ws://'
+ 'ip:port'
+ '/ws/'
);
// Send initial package on open.
socket.addEventListener('open', () => {
const package = JSON.stringify({
"time": 123456,
"channel": "futures.tickers",
"event": "subscribe",
"payload": ["BTC_USD", "ETH_USD"]
});
socket.send(package);
});
// Send data from socket to all open tabs.
socket.addEventListener('message', ({ data }) => {
const package = JSON.parse(data);
connectedPorts.forEach(port => port.postMessage(package));
});
/**
* When a new thread is connected to the shared worker,
* start listening for messages from the new thread.
*/
self.addEventListener('connect', ({ ports }) => {
const port = ports[0];
// Add this new port to the list of connected ports.
connectedPorts.push(port);
/**
* Receive data from main thread and determine which
* actions it should take based on the received data.
*/
port.addEventListener('message', ({ data }) => {
const { action, value } = data;
// Send message to socket.
if (action === 'send') {
socket.send(JSON.stringify(value));
// Remove port from connected ports list.
} else if (action === 'unload') {
const index = connectedPorts.indexOf(port);
connectedPorts.splice(index, 1);
}
});
Main.js This is only part of filter_json_run_all which continues on for about 6 or 7 Tabulator & Dygraph objects. I wante to give an idea of some of the operations called with SetTimeout() etc
function filter_json_run_all(json_str){
const startTime = performance.now();
const data_in_array = json_str //JSON.parse(json_str.data);
// if ('DATETIME' in data_in_array){
// var milliseconds = (new Date()).getTime() - Date.parse(data_in_array['DATETIME']);
// console.log("milliseconds: " + milliseconds);
// }
if (summary in data_in_array){
if("DATETIME" in data_in_array){
var time_str = data_in_array["DATETIME"];
element_time.innerHTML = time_str;
}
// summary Data
const summary_array = data_in_array[summary];
var old_sum_arr_krw = [];
var old_sum_arr_irn = [];
var old_sum_arr_ntn = [];
var old_sum_arr_ccn = [];
var old_sum_arr_ihn = [];
var old_sum_arr_ppn = [];
var filtered_array_krw_summary = filterByProperty_summary(summary_array, "KWN")
old_sum_arr_krw.unshift(Table_summary_krw.getData());
Table_summary_krw.replaceData(filtered_array_krw_summary);
//Colour table
color_table(filtered_array_krw_summary, old_sum_arr_krw, Table_summary_krw);
var filtered_array_irn_summary = filterByProperty_summary(summary_array, "IRN")
old_sum_arr_irn.unshift(Table_summary_inr.getData());
Table_summary_inr.replaceData(filtered_array_irn_summary);
//Colour table
color_table(filtered_array_irn_summary, old_sum_arr_irn, Table_summary_inr);
var filtered_array_ntn_summary = filterByProperty_summary(summary_array, "NTN")
old_sum_arr_ntn.unshift(Table_summary_twd.getData());
Table_summary_twd.replaceData(filtered_array_ntn_summary);
//Colour table
color_table(filtered_array_ntn_summary, old_sum_arr_ntn, Table_summary_twd);
// remove formatting on fwds curves
setTimeout(() => {g_fwd_curve_krw.updateOptions({
'file': dataFwdKRW,
'labels': ['Time', 'Bid', 'Ask'],
strokeWidth: 1,
}); }, 200);
setTimeout(() => {g_fwd_curve_inr.updateOptions({
'file': dataFwdINR,
'labels': ['Time', 'Bid', 'Ask'],
strokeWidth: 1,
}); }, 200);
// remove_colors //([askTable_krw, askTable_inr, askTable_twd, askTable_cny, askTable_idr, askTable_php])
setTimeout(() => { askTable_krw.getRows().forEach(function (item, index) {
row = item.getCells();
row.forEach(function (value_tmp){value_tmp.getElement().style.backgroundColor = '';}
)}); }, 200);
setTimeout(() => { askTable_inr.getRows().forEach(function (item, index) {
row = item.getCells();
row.forEach(function (value_tmp){value_tmp.getElement().style.backgroundColor = '';}
)}); }, 200);
color_table Function
function color_table(new_arr, old_array, table_obj){
// If length is not equal
if(new_arr.length!=old_array[0].length)
console.log("Diff length");
else
{
// Comparing each element of array
for(var i=0;i<new_arr.length;i++)
//iterate old dict dict
for (const [key, value] of Object.entries(old_array[0][i])) {
if(value == new_arr[i][key])
{}
else{
// console.log("Different element");
if(key!="TENOR")
// console.log(table_obj)
table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'yellow';
if(key!="TIME")
if(value < new_arr[i][key])
//green going up
//text_to_speech(new_arr[i]['CCY'] + ' ' +new_arr[i]['TENOR']+ ' getting bid')
table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'Chartreuse';
if(key!="TIME")
if(value > new_arr[i][key])
//red going down
table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'Crimson';
}
}
}
}
Potential fudge / solution, thanks Aaron :):
function limiter(fn, wait){
let isCalled = false,
calls = [];
let caller = function(){
if (calls.length && !isCalled){
isCalled = true;
if (calls.length >2){
calls.splice(0,calls.length-1)
//remove zero' upto n-1 function calls from array/ queue
}
calls.shift().call();
setTimeout(function(){
isCalled = false;
caller();
}, wait);
}
};
return function(){
calls.push(fn.bind(this, ...arguments));
// let args = Array.prototype.slice.call(arguments);
// calls.push(fn.bind.apply(fn, [this].concat(args)));
caller();
};
}
This is then defined as a constant for a web worker to call:
const filter_json_run_allLimited = limiter(data => { filter_json_run_all(data); }, 300); // 300ms for examples
Web worker calls the limited function when new web socket data arrives:
// Event to listen for incoming data from the worker and update the DOM.
webSocketWorker.port.addEventListener('message', ({ data }) => {
// Limited function
filter_json_run_allLimited(data);
});
Please if anyone knows how websites like tradingview or real time high performance data streaming sites allow for low latency visualisation updates, please may you comment, reply below :)
I'm reticent to take a stab at answering this for real without knowing what's going on in color_table. My hunch, based on the behavior you're describing is that filter_json_run_all is being forced to wait on a congested DOM manipulation/render pipeline as HTML is being updated to achieve the color-coding for your updated table elements.
I see you're already taking some measures to prevent some of these DOM manipulations from blocking this function's execution (via setTimeout). If color_table isn't already employing a similar strategy, that'd be the first thing I'd focus on refactoring to unclog things here.
It might also be worth throwing these DOM updates for processed events into a simple queue, so that if slow browser behavior creates a rendering backlog, the function actually responsible for invoking pending DOM manipulations can elect to skip outdated render operations to keep the UI acceptably snappy.
Edit: a basic queueing system might involve the following components:
The queue, itself (this can be a simple array, it just needs to be accessible to both of the components below).
A queue appender, which runs during filter_json_run_all, simply adding objects to the end of the queue representing each DOM manipulation job you plan to complete using color_table or one of your setTimeout` callbacks. These objects should contain the operation to performed (i.e: the function definition, uninvoked), and the parameters for that operation (i.e: the arguments you're passing into each function).
A queue runner, which runs on its own interval, and invokes pending DOM manipulation tasks from the front of the queue, removing them as it goes. Since this operation has access to all of the objects in the queue, it can also take steps to optimize/combine similar operations to minimize the amount of repainting it's asking the browser to do before subsequent code can be executed. For example, if you've got several color_table operations that coloring the same cell multiple times, you can simply perform this operation once with the data from the last color_table item in the queue involving that cell. Additionally, you can further optimize your interaction with the DOM by invoking the aggregated DOM manipulation operations, themselves, inside a requestAnimationFrame callback, which will ensure that scheduled reflows/repaints happen only when the browser is ready, and is preferable from a performance perspective to DOM manipulation queueing via setTimeout/setInterval.
I wrote a web app to make reservations for a concert. People can select from certain dates and can choose with how many people they come by selecting an amount of seats.
I use a spreadsheet to gather all the reservations. I wrote down the last 2 functions of the process. If I press the button to order the seats, 2 functions activate: validate() on the frontend and getLastCheck() on the backend. This is a last check whether the asked amounts of seats are still available. If so, the data is written to the spreadsheet.
I tested the script a few times with 4 other colleagues and we simultaneously tried to book 3 seats on the same date. Since there were only 10 seats left, 2 of us should get the message that the "seats are not booked". Sometimes it worked fine, other times only 1 of us received the message "seats are not booked" and the other 4 people (1 too many!) could book their seats. In that case we exceeded the maximum capacity.
I presume that the belated updating from the spreadsheet (which results in a wrong evaluation) is caused by the time of traffic from and to the spreadsheet. Is there a way to solve this wrong evaluation when simultaneously submitting the data?
Frontend function:
function validate() {
var info = {};
info.firstName = document.getElementById("first-name").value;
info.lastName = document.getElementById("last-name").value;
info.mail = document.getElementById("email").value.trim();
info.date = document.getElementById("select-date").value;
info.seats = document.getElementById("select-seats").value;
google.script.run.withSuccessHandler(function(result){
console.log(result);
}).getLastCheck(info);
}
backend function:
function getLastCheck(info) {
var listAll = wsRsrv.getRange(2, 5, lastRowRsrv, 2).getValues();
var dates = listAll.map(function(element){ return element[0]; });
var seats = listAll.map(function(element){ return element[1]; });
var sum = 0;
var diff = maxPerDate - info.seats;
for (var i = 0; i<listAll.length; i++) {
if (info.date == dates[i]) { sum += Number(seats[i]); }
}
if (sum <= diff) {
wsRsrv.appendRow([new Date(), info.lastName, info.firstName, info.mail, info.date, info.seats]);
return "seats are booked";
} else {
return "seats are not booked";
}
}
I tested it out and it seems to work right.
function lockedFunction() {
var active_spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
// BEGIN - start lock here
const lock = LockService.getScriptLock();
try {
lock.waitLock(5000); // wait 5 seconds for others' use of the code section and lock to stop and then proceed
} catch (e) {
console.log('Could not obtain lock after 5 seconds.');
return "Error: Server busy try again later... Sorry :("
}
// note: if return is run in the catch block above the following will not run as the function will be exited
const active_sheet = active_spreadsheet.getActiveSheet()
const new_start_row = active_sheet.getLastRow() + 1;
// Run the function
const data = {
firstName: 'David',
lastName: 'Salomon',
mail: 'mail#example.com',
date: new Date(),
seats: 10
}
getLastCheck(data)
SpreadsheetApp.flush(); // applies all pending spreadsheet changes
lock.releaseLock();
// END - end lock here
return;
}
You can check this other thread to have more ideas on how to implement it but as mentioned in the comments, you can check the Google documentation
I'm making use of google spread sheets and Apps Script to collect some financial data, there is one function that takes some time because it processes a lot of information.
because of this, google shows me that exceeded the maximun execution time, but even having a for loop that writes on the file each loop it wont do it until the function finishes.
can someone give me an idea of how to write the spreadsheet while its executing?
thanks.
Leaving here an example script.
function myFunction() {
var sheet = SpreadsheetApp.getActiveSpreadsheet();
var rangeSheet = activeSheet.getRange(1, 1, activeSheet.getLastRow() - 1, 1).getValues();
// after cleared out i have an array of values so i do:
rangeSheet.forEach(async (el, idx) => {
let result = await anotherFunction(el) // <--- this is the function taking around 2 minutes to complete
sheet.getRange(`$B${idx + 1}`).setValue(`${result}
})
}
This will give you some boost in performance:
function myFunction() {
var sheet = SpreadsheetApp.getActiveSpreadsheet();
var rangeSheet = activeSheet.getRange(1, 1, activeSheet.getLastRow() - 1, 1).getValues();
var data = [];
rangeSheet.forEach(async (el, idx) => {
let result = await anotherFunction(el) // <--- this is the function taking around 2 minutes to complete
data.push([result]);
})
sheet.getRange(1,2,data.length,data[0].length).setValues(data);
}
Read Best Practices for more info.
However, if anotherFunction takes too much time, then I am afraid you need to redesign your logic. I would advice you to use UrlfetchApp.fetchAll() if you want to fetch multiple URLs.
References:
google apps script with UrlfetchApp.fetchAll() or with async/ await for multiple http requests?
Here is the scenario:
When my web app starts, I want to load data from several tables in local storage (using indexedDB). I delegate this work to a web worker. It will load each table in turn, and fire a message with the data as it loads each one. On the main thread, a listener will receive the message and store the data in a cache.
But let's say the user presses a button to view the data for a specific table. The app calls a function that checks the cache, and sees that the data for that table has not been loaded yet.
How does this function wait until the data for that table has been cached so that it can return the data? Even more important, what if the table is scheduled to be loaded last? How can this function send a message to the web worker to prioritize loading that specific table so that its data will available as soon as possible?
What is a general pattern for a clean solution to this pre-emptive scheduling problem? I would like to avoid polling if at all possible.
The Worker may use an asynchronous queue that contains all the tables to be loaded and is sorted after a certain priority, so you can priorize certain tables and they get sorted to the front of the table. As you havent shown a real implementation here is a more generalized version:
class AsyncPriorityQueue {
constructor(task){
this.task = task;
this.queue = [];
}
push(element, priority = 0){
const pos = this.queue.findIndex(el => el.priority < priority) + 1;
this.queue.splice(pos, 0, {element, priority});
if(this.running) return;
this.running = true;
this._run();
}
prioritize(element, priority = 10){
const pos = this.queue.findIndex(el => el.element === element);
if(pos != -1) this.queue.splice(pos, 1);
this.push(element, priority);
}
async _run(){
while(this.queue.length)
await this.task(this.queue.shift().element);
}
}
Note: If the task is not asynchronous you should use sth like setTimeout(next, 0) to allow the process messaging to interrupt it...
A sample implementation could be an image loader:
class ImageLoader extends AsyncPriorityQueue {
constructor(){
super(function task(url){
const img = new Image();
img.src = url;
return new Promise(res => img.onload = res);
});
}
}
const loader = new ImageLoader;
loader.push("a.jpg");
loader.push("b.jpg", 1); // a bit more important
// Oh, wait:
loader.prioritize("a.jpg");