I am trying to write a Google Apps script which has a client and server side component. The client side component displays a progress bar. The client calls server side functions (which are called asynchronously), whose progress has to be shown in the client side progress-bar. Now, what I want is to be able to update the client side progress bar based on feedback from the server side functions. Is this possible?
The complexity is created due the the fact that JS makes the server-side calls asynchronously and hence I cannot really have a loop on the client side calling the functions and updating the progress bar.
I could of course split up the execution of the server side function in multiple steps, call one by one from the client side, each time updating the status bar. But I'm wondering if there's a better solution. Is there a way to call a client side function from the server side, and have that update the progress bar based on the argument passed? Or is there a way to access the client side progress-bar object from server side and modify it?
The way I've handled this is to have a middleman (giving a shout out now to Romain Vialard for the idea) handle the progress: Firebase
The HTML/client side can connect to your Firebase account (they're free!) and "watch" for changes.
The client side code can update the database as it progresses through the code - those changes are immediately fed back to the HTML page via Firebase. With that, you can update a progress bar.
Romain has a small example/description here
The code I use:
//Connect to firebase
var fb = new Firebase("https://YOUR_DATABASE.firebaseio.com/");
//Grab the 'child' holding the progress info
var ref = fb.child('Progress');
//When the value changes
ref.on("value", function(data) {
if (data.val()) {
var perc = data.val() * 100;
document.getElementById("load").innerHTML = "<div class='determinate' style='width:" + perc + "%\'></div>";
}
});
On the client side, I use the Firebase library to update the progress:
var fb = FirebaseApp.getDatabaseByUrl("https://YOUR_DATABASE..firebaseio.com/");
var data = { "Progress": .25};
fb.updateData("/",data);
Rather than tying the work requests and progress updating together, I recommend you separate those two concerns.
On the server side, functions that are performing work at the request of the client should update a status store; this could be a ScriptProperty, for example. The work functions don't need to respond to the client until they have completed their work. The server should also have a function that can be called by the client to simply report the current progress.
When the client first calls the server to request work, it should also call the progress reporter. (Presumably, the first call will get a result of 0%.) The onSuccess handler for the status call can update whatever visual you're using to express progress, then call the server's progress reporter again, with itself as the success handler. This should be done with a delay, of course.
When progress reaches 100%, or the work is completed, the client's progress checker can be shut down.
Building on Jens' approach, you can use the CacheService as your data proxy, instead of an external service. The way that I've approached this is to have my "server" application generate an interim cache key which it returns to the "client" application's success callback. The client application then polls this cache key at an interval to see if a result has been returned into the cache by the server application.
The server application returns an interim cache key and contains some helper functions to simplify checking this on the client-side:
function someAsynchronousOperation() {
var interimCacheKey = createInterimCacheKey();
doSomethingComplicated(function(result) {
setCacheKey(interimCacheKey, result);
});
return interimCacheKey;
}
function createInterimCacheKey() {
return Utilities.getUuid();
}
function getCacheKey(cacheKey, returnEmpty) {
var cache = CacheService.getUserCache();
var result = cache.get(cacheKey);
if(result !== null || returnEmpty) {
return result;
}
}
function setCacheKey(cacheKey, value) {
var cache = CacheService.getUserCache();
return cache.put(cacheKey, value);
}
Note that by default getCacheKey doesn't return. This is so that google.script.run's successHandler doesn't get invoked until the cache entry returns non-null.
In the client application (in which I'm using Angular), you call off to the asynchronous operation in the server, and wait for its result:
google.script.run.withSuccessHandler(function(interimCacheKey) {
var interimCacheCheck = $interval(function() {
google.script.run.withSuccessHandler(function(result) {
$interval.cancel(interimCacheCheck);
handleSomeAsynchronousOperation(result);
}).getCacheKey(interimCacheKey, false);
}, 1000, 600); // Check result once per second for 10 minutes
}).someAsynchronousOperation();
Using this approach you could also report progress, and only cancel your check after the progress reaches 100%. You'd want to eliminate the interval expiry in that case.
Related
I am calling a Google Analytics API multiple times and load that data into a subscription. Now I want to create a progressbar to inform the user that data is being loaded and give a view on how long it is going to take.
I read that it's best to use publications to pass data from server to client. Is this true?
I created following publication on the server. What is does is following:
set the initial progressValue and the initial publication with id 1
keep looping if the progressValue is less than 100 and tell that the publication of 1 is changing.
Below this code I have an other publication running where progressValue is being set in steps in a loop.
When looking at the client only the last progressValue gets posted. Before this I receive a lot of empty arrays. So it's like:
[]
[]
[]
[]
...
Progress publication
What I want is that the client receives every change in progressValue instead of only the last one. How can I solve this?
If there are any better suggestions on how to create a subscription progressbar, these answers will also be accepted.
if (Meteor.isServer) {
let progressValue = 0;
Meteor.publish('progress', function() {
const self = this;
let lastProgressValue = 0;
const id = 1;
self.added('progress', id, {
progress: progressValue,
total: 100,
});
while (progressValue < 100) {
self.changed('progress', id, {
progress: progressValue,
total: 100,
});
}
self.ready();
});
...
Hmm... so, a couple of things here.
I read that it's best to use publications to pass data from server to
client. Is this true?
This is the whole point of Meteor, using ddp. Means that data is sent to the client automagically from the server. So, the bulk of the work to manipulate data is actually handled client side using minimongo.
Have a look at this article for a good discussion of the 'automagic' part...
http://richsilv.github.io/meteor/meteor-low-level-publications/
How do you do progress?
You don't want to try handle the incrementing on the server side. Instead, you want to get a simple count of the server, perhaps using a reactive aggregate (see my answer here How to reactively aggregate mongodb in meteor) and send that to the client. So, server does a count as a publication and tells the client '57' coming.
Then as your normal data publication, you send the 57 records to the client. ON THE CLIENT, you now basically do the same sum as you did on the server, but as only some of the 57 data records have been received by the client, you effectively get a progress counter by dividing client received by the servers message of total to be sent.
Summary
On the SERVER - 2 publications, 1 reactive aggregate for the count of the records to be sent and 1 as the normal data being sent
On the CLIENT - function to count the records in the local minimongo collection - collection.find({}).count() - will do the trick. This will increment as each record is received by the client from the server.
Progress is as simple as count on client divided by server sent count to be delivered.
Hi guys I have a problem that i don't really have idea how to solve. it's also a bit strange :/
Basically I have created this Lambda function to connect to a mysql DB using the node package 'mysql'.
If i run the function from command line on my pc using the command 'sls function run function1' and make different queries everything is fine.
But when I call the function from a web browser using the link, I have to refresh the page 2 times to get the right result because at the first refresh the server respond with the old result.
I have noticed that from the command line I always have different thredID while from webbrowser is always the same.
Also I don't close the connection in the lambda function code because everything is fine if i run the function from command line but from browser I can only make 2 queries and then I get a message that say that I cannot use a closed connection.
So it seems like Lambda store the old query result when I call it from web browser.
Obviously I'm making same stupid mistake but I don't know how to solve it.
Does anyone have an idea?
Thanks :)
'use strict';
//npm packages
var mysql=require('mysql');
var deasync = require('deasync');
//variables
var goNext=false; //use to synchronize deasync
var error=false; //it becomes TRUE if an error occured during the connection to the DB
var dataColumnTable; //the data thet you extract from the query to the DB
var errorMessage;
//----------------------------------------------------------------------------------------------------------------
//always same credentials
var connection = mysql.createConnection({
host : 'hostAddress',
user : 'Puser',
password : 'password',
port : '3306',
database : 'database1',
});
//----------------------------------------------------------------------------------------------------------------
module.exports.handler = function(event, context) {
var Email=event.email;
connection.query('SELECT City, Address FROM Person WHERE E_Mail=?', Email, function(err, rows) {
if(err){
console.log("Cannot connect to DB");
console.log(err);
error=true;
errorMessage=err;
}
else{
console.log("data from column acquired!");
dataColumnTable=rows;
}
//connection.end(function(err) {
// connection.destroy();
//});
//console.log("Connection closed!");
goNext=true;
});
require('deasync').loopWhile(function(){return goNext!=true;});
//----------------------------------------------------------------------------------------------------------------
if(error==true)
return callback('Error '+ errorMessage);
else
return callback(null,dataColumnTable); //return a JsonFile
//fine headler
};
Disclaimer: I'm not very familiar with AWS and/or AWS Lambda.
http://docs.aws.amazon.com/lambda/latest/dg/programming-model-v2.html states (emphasis mine):
Your Lambda function code must be written in a stateless style, and have no affinity with the underlying compute infrastructure. Your code should expect local file system access, child processes, and similar artifacts to be limited to the lifetime of the request. Persistent state should be stored in Amazon S3, Amazon DynamoDB, or another cloud storage service. Requiring functions to be stateless enables AWS Lambda to launch as many copies of a function as needed to scale to the incoming rate of events and requests. These functions may not always run on the same compute instance from request to request, and a given instance of your Lambda function may be used more than once by AWS Lambda.
Opening a connection and storing it in a variable outside your handler function is state. The connection will likely be closed between requests or even before your first request. Your lambda function may be reused (hence identical thread ids).
My assumption would be (and an attempt to solve this problem), that you need to create the connection on every request (i.e., inside your handler) and may not expect any value be as initialized or as on last request. (except for constants probably).
I'm testing a web page where the user can send a message to another via a textinput. A POST request is then send on the server and the message is dumped on the disk in the var/mail/new folder.
After automatising the sending of the message in the page with Protractor I'm calling browser.waitForAngular() and browser.driver.sleep(4000) to leave time for the backend to write the mail on the disk.
After these calls the check of the email's presence fails. When looking in the Unix shell, I can confirm that the email was sent and also the next test marked with in Jasmine with it confirms the presence of the email.
Why is browser.driver.sleep(4000) not effective to wait for the backend to proceed? How can I correct the following code?
it("is possible to send a message", function() {
shared.loginContributor();
var mailsBeforeMessaging =
fs.readdirSync(browser.params.mail.queue_path + "/new");
console.log('mailsBeforeMessaging');
console.log(mailsBeforeMessaging.length);
console.log(fs.lstatSync(browser.params.mail.queue_path + "/new"));
var usersListing = new UserPages.UsersListing().get();
var annotatorPage = usersListing.getUserPage("annotator");
annotatorPage.sendMessage("title5", "content64");
exec("/tmp/check.sh");
// we expect the message widget to disappear
var button = element(by.css(".user-profile-info-button"));
console.log('waiting');
browser.wait(EC.elementToBeClickable(button), 5000);
console.log('waiting is finished');
expect(EC.elementToBeClickable(button)).toBeTruthy();
// wait for mail to be dumped on the disk?
browser.waitForAngular();
browser.driver.sleep(4000);
exec("/tmp/check.sh");
var mailsAfterMessaging =
fs.readdirSync(browser.params.mail.queue_path + "/new");
console.log('mailsAfterMessaging');
// ERROR: here the number of emails is NOT incremented
console.log(mailsAfterMessaging.length);
console.log(fs.lstatSync(browser.params.mail.queue_path + "/new"));
});
it("xyz", function() {
console.log(fs.lstatSync(browser.params.mail.queue_path + "/new"));
// here the number of emails is incremented
var mailsAfterMessaging =
fs.readdirSync(browser.params.mail.queue_path + "/new");
console.log('mailsAfterMessaging');
console.log(mailsAfterMessaging.length);
});
Most of the Protractor functions do not do anything. They queue something up to be done later, and return promise to do it. After an it block schedules a bunch of things to do, they actually start happening (via the promises they registered in the ControlFlow).
Your checks, however, are all executing immediately. So, they are happening before any of the protractor calls accomplish anything.
Use then to make the waiting and dependencies explicit in your test. Like this:
annotatorPage.sendMessage("title5", "content64").then(function() {
exec("/tmp/check.sh");
});
or:
browser.wait(EC.elementToBeClickable(button), 5000).then(function() {
console.log('wait-for-clickable has completed'); // B
});
console.log('wait-for-clickable has been scheduled'); // A
See the Protractor Control Flow documentation and the Webdriver JS API doc.
Its not you. This is a crazy API to learn because it does not act at all like anyone familiar with normal synchronous programming would expect.
Is it possible to check/log how much data has been transferred during each run of PhantomJs/CasperJS?
Each instance of Phantom/Casper has a instance_id assigned to it (by the PHP function that spun up the instance). After the run has finished, the amount of data transferred and the instance_id will have to make its way to be inserted into a MySQL database, possibly via the PHP function that spawned the instance. This way the bandwidth utilization of individual phantomjs runs can be logged.
There can be many phantom/casper instances running, each lasting a minute or two.
The easiest and most accurate approach when trying to capture data is to get the collector and emitter as close as possible. In this case it would be ideal if phantomjs could capture that data that you need and send it back to your PHP function to associate it to the instance_id and do the database interaction. Turns out it can (at least partially).
Here is one approach:
var page = require('webpage').create();
var bytesReceived = 0;
page.onResourceReceived = function (res) {
if (res.bodySize) {
bytesReceived += res.bodySize;
}
};
page.open("http://www.google.com", function (status) {
console.log(bytesReceived);
phantom.exit();
});
This captures the size of all resources retrieved, adds them up, and spits out the result to standard output where your PHP code is able to work with it. This does not include the size of headers or any POST activity. Depending upon your application, this might be enough. If not, then hopefully this gives you a good jumping off point.
In my app I am polling the webserver for messages every second and displaying them in the frontend.
I use setInterval to achieve this. However as long as the user stays on that page the client keeps polling the server with requests even if there is no data. The server does give an indication when no more messages are being generated by setting a variable.
I thought of using this variable to clearInterval and stop the timer but that didn't work. What else can I use in this situation?
I am using jquery and django. Here is my code:
jquery:
var refresh = setInterval(
function ()
{
var toLoad = '/myMonitor'+' #content';
$('#content').load(toLoad).show();
}, 1000); // refresh every 1000 milliseconds
});
html:
div id=content is here
I can access the django variable for completion in html with each refresh. How can I set clearInterval if at all ?
Note: stack overflow does not let me put is > < so html is incomplete
Thanks
Updated 03/16/2010
I must be doing something wrong. But cannot figure it out. Here is my script with clearTimer and it does not work.
var timer = null;
$(function(){
if ("{{status}}" == "False")
{
clearInterval(timer);
}
else
{
timer = setInterval(
function(){
var toLoad = '/myMonitor'+' #content';
$('#content').load(toLoad).show();}
,1000); // refresh every 1000 milliseconds
}
});
status is a boolean set in "views.py" (Django).
Thanks a bunch.
A couple people have already answered with specific resources to your problem, so I thought I would provide a bit of background.
In short, you want the server to push data to the browser to avoid extensive client-side polling. There isn't a good cross-browser way to support server push, so a common solution that requires much less polling is to use the Comet (another cleaning product, like AJAX) long-poll technique.
With Comet, the browser makes a request, and the server keeps the connection open without responding until new data is available. When the server does has new data, it sends it over the open connection and the browser receives it right away. If the connection times out, the browser opens a new one. This lets the server send data to the client as soon as it becomes available. As others have indicated, this approach requires special configuration of your web server. You need a script on the server that checks for data at an interval and responds to the client if it exists.
Something to keep in mind with this approach is that most web servers are built to get a request from a client and respond as quickly as possible; they're not intended to be kept alive for a long period of time. With Comet you'll have far more open connections than normal, probably consuming more resources than you expect.
Your clearInterval check is only checking when the document ready event is fired.
If the code you gave is exactly what's in the browser, then you're comparing the string "{{status}}" to the string "False". I'd rather watch paint dry than wait for that to evaluate as true.
What if your requests taking longer than 1 second to complete? : You'll flood your server with requests.
function update () {
$('#content').show().load('/myMonitor'+' #content', function (response, status) {
if (!/* whatever you're trying to check*/) {
setTimeout(update, 1000);
};
});
};
$(document).ready(function () {
update();
});
Is closer than where you were, but you still need to work out how you're going to decide when you want to stop polling.