I am working on a .Net application which has a blank .aspx page which links to a code-behind (.aspx.cs) sheet. It is called by JavaScript functions from many places which call the page with query-string parameters.
Dependent on the parameters - the page runs 2 families of stored procedures. One family creates records - the other updates records. Any SQL errors are then e-mailed over to me.
There is an error that I suspect is due to the code being executed twice in rapid succession - I receive a unique primary key constraint error. It seems that the page is being called a second time before the INSERT has taken place. This unfortunately is causing data-loss.
My question is...
How do I "Lock" or "Queue" aspx.cs page executions/requests to ensure that one "instance" or "thread" of the page completes before another may be fired?
Separate the invoking of the process from the execution of the process.
Basically you would add a table to the database as your "queue", each record of which has the information necessary to perform the process in question. (The data posted from the page, basically, as well as any other meta-data like the user who queued it and the time it was queued, etc.) The code on the page would do nothing more than simply insert a record in that table.
Separate from the ASP.NET application you would have a background process (a Windows Service might be best, a scheduled Console Application may work just as well) which periodically checks that table for new records. When it finds records, it executes the process in question on each one successively.
This allows the web application to still be multi-threaded (which you definitely want) while processing the records in a single thread.
Related
I'm writing a web-based system using flask with react and redux that needs to have web-based clients write conditional statements that can be saved to a configuration file, but also executed in real time without restarting the server or other services.
Obviously this can be done using eval(), but obviously we won't be using that.
Any safe ways to run user conditions that call on live variables to perform calculations?
As an example they might want to perform a standard conditional calculation like:
if(a===1 && (b===2 || c===2)){
//do something
}
Where a, b, and c are values that are provided from the server to the client and change dynamically.
UPDATE based on question:
The server provides real-time updates on alarms monitored by the server. When an alarm changes state - say from no-alarm to in-alarm - it sends the new data to the client.
The client side renders this information as a list of alarms. The list can be filtered easily enough, but one issue is that you can have an alarm flood event where ~1000 alarms all come in simultaneously. You also have a few standard/common events where a particular series of alarms all change to a particular state at the same time and that indicates a particular issue/fault and hence a particular fix.
Each user is unique, so it can;t be a one-size-fits all approach and it would be useful if each user can set some basic rules that determine what message to display based on the value of any combination of alarms and their alarm state. They would use a browser form to select these condition states which they can submit to the server. This will insert a line into thier personal configuration file held on the server so that each time they log in they automatically have access to these calculation.
if an alarm changes state, it is sent to the client, which then automatically performs the calculation in the background to determine if a message needs to be displayed.
Appears like the best approach is to allow the strings, but to ensure that all strings are parsed through a super well constructed function to remove anything malicious.
I'll probably run this function many times - first on submission, and also just before executing.
If these functions are developed using an interactive GUI of dropdown buttons and auto-fill input fields that constructs the code for the user (i.e. the user doesn't actually write the code), it should be safe enough.
I would like to know if it is possible to detect that a thread is already running a Cloud Functions, and if possible to also detect if it is running on a particular ID's data. I think I could have a variable stored in firebase memory of the ID in Firebase Database that the function is being run on from the Database, and to remove the variable when the function is done running,but the concern is of two writes to the database happening subsequently and very rapidly, causing the initial thread to not be able to write to memory fast enough before the second thread checks if the variable is there, especially on a cold start from the firebase thread - which in my understanding is a variable amount of time in which either thread could potentially spin up first.
My use case is this:
Let's say a write to the realtime database happens from the client side that causes a trigger for Cloud Functions to run a handler. This handlers job is to loop through and do work with the snapshot of records that was just written to by the client, and using a loop will parse each record in the snapshot, and when it is done, delete them. The handler works great until another record is written to the same group of records in the database before the handler's job is done, which causes a second handler thread to spin up, and start moving through the records in the same group of records, which would cause records to be iterated over twice, and possibly the data to be handled twice.
I have other solutions for my particular case which I can use instead, but it involves just allowing each record to trigger a separate thread like normal.
Thanks in advance!
There is no way to track running instances "in-memory" for Cloud Functions, as each function invocation may be running in entirely different virtual infra. Instead, what you'd most likely want to do here is have some kind of lock persisted in e.g. the Firebase Realtime Database using a transaction. So you'd do something like:
When the function invocation starts, generate a random "worker ID".
Run a transaction to check a DB path derived from the file you're processing. If it's empty, or populated with a timestamp that is older than a function timeout, write your worker ID and the current timestamp to the location. If it's not empty or the timestamp is fresh, exit your function immediately because there's already an active worker.
Do your file processing.
Run another transaction that deletes the lock from the DB if the worker ID in the DB still matches your worker ID.
This will prevent two functions from processing the same file at the same time. It will mean, however, that any functions that execute while a path is locked will be discarded (which may or may not be what you want).
Let's say I have a couchDB database called "products" and a frontend with a form.
Now if a user opens a document from this database in the form I want to prevent other user from editing this specific document.
Usually pretty simple:
-> read document from couchDB
-> set a variable to true like: { edit : true }
-> save (merge) document to couchDB
-> if someone else tries to open the document he will receive an error, becaus of edit:true.
BUT, what if two user open the document at the exact same time?
The function would be called twice and when the second one opens the document he would falsely receive an edit:false because the first didn't had enough time to save his edit:true. So how to prevent this behaviour?
First solution would be:
Build an array as a cue for database requests and dont allow parallel requests, so all requests would be worked off one after another. But in my opinion this is a bad solution because the system would be incredible slow at some point.
Second solution:
Store the documentIDs of the currently edited documents in an local array in the script. This would work because this is no asynchronous process and the second user would receive his error immediately.
So far so good, BUT, what if some day there are too many user and this system should run in a cluster (the node client server, not the database) - now the second solution would not work anymore because every cluster slave would have its own array of documentIDs. Sharing there would end in another asynchronous task and result in the same problem above.
Now i'm out of ideas, how do big clustered systems usually handle problems like that?
CouchDB uses MVCC to maintain consistency in your database. When a document is being updated, you must supply both the ID (_id) and revision number (_rev) otherwise your change will be rejected.
This means that if 2 clients read the document at revision 1 and both attempt to write a change using that same revision number, only the first will be accepted by the database. The 2nd client will receive an error, and it should fetch the latest revision of the document in order to proceed.
In a single-node environment, this model prevents conflicts outright. However, in cases where replication is occurring, it is still possible to get conflicts, even when using MVCC. This is because conflicting revisions can technically be written to different nodes before they have been replicated to one another. In this case, CouchDB will record the conflict and your application is responsible to resolve them.
CouchDB has stellar documentation, in particular they have an article all about conflicts and replication that I highly recommend for this subject.
Does JSVM run just in one thread?
I am wondering how the JavaScript function executing inside the VM.
The source code below is interesting:
// include jQuery as $
function test() {
$.ajax({url:"xxx.com"})
.success(function() {alert("success 1");})
.fail(function() {alert("fail 1");});
$.ajax({url:"yyy.com"})
.success(function() {alert("success 2");})
.fail(function() {alert("fail 2");});
while(true);
}
It will make die loop at the "while" line and never pop up any alert dialog to show neither "success" nor "fail".
We know inside the $.ajax, the VM creates XMLHttpRequest and sends a HTTP request.
After sending out two requests, it meets the "while" line.
Thus I image that the JSVM:
1) can handle only function call at one time. (function is atomic)
2) follow the rule: first comes, first served.
Does my idea right?
Does anyone can explain the internal implementation of JSVM?
More specific,
If using AngularJS to develop a front end app, we would like to do something and then immediately record a log to remote server in form submit event like ng-submit.
function ngSubmitTest() {
doA();
recordA(ajax, remoteServer); // must after doA()
}
If recordA uses AJAX, we should ensure recordA is complete before ng-submit redirect the page meanwhile kill the old page and also the VM (if the old page is killed, the recordA may not complete). One solution is doing AJAX with async=false. And I wonder if there is any other solutions?
Thanks.
The implementation of JS depends on the context you're runing it.
Each browser has it's own implementantion, and they can do whatever they want as long as they follow the language specification.
It shouldn't bother you if it runs on one or multiple threads, but you can be sure JavaScript is not a "threaded" language, it works with an event loop flow, in which an event is fired, and consecutive functions are fired after that, until there is nothing more to call. This is the reason why it's pretty hard to block the UI in JavaScript if you're writing "good" code.
A good example on how this works, and the diferences betwen event loops and classic threading, is node.js, i'll give you a example:
Supose you're listening for a request on a server, and 2 seconds after the request arrives you'll send a message. Now let's supose you duplicate that listener, and both listeners do the same thing. If you request the server, you'll get the two messages at the same time, 2 seconds after the request is made, instead of one message on 2 seconds, and the other one on 4 seconds. That means both listeners are runing at the same time, instead of following a linear execution as most systems do.
Node runs Chrome's V8 if you're wondering, it's a very professional JS interpreter and it was a breakthorugh when it came out.
I've got a question about data flow that is summarized best by the image below:
I've got the data path from the UI (WaveMaker) down to the hardware working perfectly. The question I have is whether I'm missing something in the connection from the Java Service to Wavemaker.
I'm trying to provide information back to Wavemaker from the HW. The specifics of shared memory and semaphore signaling are worked out already. Where I'm running into a problem is how to get the data from the Java Service back to WaveMaker, when it hasn't specifically requested it. My plan was to generate events when the Java Service returned, but another engineer here insists that it won't work, since there's no direct call from Wavemaker and we don't want to poll.
What I proposed was to call the function after the page loaded, allow the blocking to occur at the .so level, as shown below, and then handle the return string when the call returned. We would then call the function again. That has the serious flaw of blocking out interaction with the user interface.
Another option put forth would be to use a hidden control, somehow pass it into Java, and invoke an event on it from Java, which could then be made to execute a script to update the UI with the HW response. That keeps the option of using threads alive, and possibly resolves the issue. Is there some more elementary way of getting information from Java->JavaScript->UI without it having been asked for?