I've been trying to figure out how to fetch data from firebase and store it in a variable. The code i used works sometimes, but other times it just doesn't run. When I close and reopen it, it often doesn't run the code until i comment out different parts and run it. The parts I need to comment out seems to change every time, so I don't know what the issue is.
// Function that runs when the sign up submit button is pressed
function signup() {
console.log("signup ran");
firebase.database().ref().once("value", gotData);
}
// Snapshot function
function gotData(data) {
console.log("gotData ran");
dv = data.val()
var tempcodes = dv.verifCodes;
var codes = JSON.stringify(tempcodes);
var codelist = codes.split(",")
console.log(codelist);
var vcode = document.getElementById("verifcode").value;
if (codelist.indexOf(vcode) > -1){
console.log("valid code");
}
}
There are no error messages, and "gotData ran" is never printed to the console, while "signup ran" does.
I'm really new to firebase and still don't really understand how snapshots work in the first place, and the tutorials don't really explain it that well. Any help/advice is welcome!
Once again, though, I'd like to mention that this code works flawlessly when it's actually executing, and the only problem is that it isn't doing that. Thanks!
EDIT: So, something strange seems to be happening. I have a separate line of code adding data to the firebase that wasn't relevant, but that code still runs even when console.log doesn't. It looks like the code will affect the database, but won't run anything in the program? I really don't know what the issue is and I'm still kind of lost.
Related
I am creating some code that downloads a file from a website and unzips it automatically. I have made it so that it will check that the latest version of the file has been downloaded every 4 hours, and if it already has been downloaded, then it won't download it again. Now I want to make it so that if the download fails for some reason, rather than waiting another 4 hours to download the file again, it will attempt to download it again instantly, and will keep trying to do that until it downloads successfully.
I have used the "try" and "catch" statements to make it so that if it will try and download the file, and if the file download fails, it will catch the error, and then in the catch statement it will do the same thing but do it every 5 seconds instead of every 4 hours.
try{
interval()
//this is referencing a function that downloads the file every 4 hours
setTimeout(stopInterval, 5000);
//this is to prevent the function from stacking every time the try statement is called
}
}
catch(e){
checkInterval()
//this is referencing a function that does the same as what's in the try statement but does it every 5 seconds instead
}
}
what it successfully does is that if it detects an error within the code itself (e.g. the function it's trying to call is misspelt), it will do what I am asking it to do which is repeat the function every 5 seconds. But what I want it to do is exactly that, except that it will apply to other errors like the internet disconnecting. I have tried using the "Throw New Error" construct but don't really know how to use it properly, or how to specifically apply it to errors related to internet or other problems. If there is a way to do this with the try catch statement, that would be awesome, but if I have to do something else then any help would be appreciated. I am new to this site so sorry if this isn't explained very well, if there is anything you need me to clarify feel free to let me know. Thanks in advance!
I found the solution. For everyone else who wants to know what the solution is, this is what I did.
const dns = require('dns');
const navigator = global.navigator = {onLine: true};
((navigator) => {
const dns = require('dns');
setInterval(() => {
dns.resolve('www.google.com',
error => navigator.onLine = !error);
}, 5000);
})(navigator);
What this does is set "navigator.onLine" manually since it doesn't work by default in NodeJS. And then I cause it to create an error by doing:
var isOnline = navigator.onLine;
if (!isOnline) {
throw new Error('Connection Unsuccessful');
}
By creating the error, it allows the catch statement to detect it as an error and will perform the code that you want it to perform once the internet disconnects.
This is a fairly weird thing, and it's hard to reproduce. Not the best state of a bug report, I apologize.
I'm using .transaction() to write a value to a location in Firebase. Here's some pseudo-code:
var ref = firebase.child('/path/to/location');
var storeSafely = function(val) {
ref.transaction(
function updateFunc(currentData) {
console.log('Attempting update: ' + JSON.stringify(val));
if (currentData) return;
return val;
},
function onTransactionCompleteFunc(err, isCommitted, snap) {
if (err) {
console.log('Error in onTransactionCompleteFunc: ' + JSON.stringify(err));
return;
}
if (! isCommitted) {
console.log('Not committed');
return;
}
ref.onDisconnect().remove();
doSomeStuff();
});
};
var doSomeStuff = function() {
// Things get done, time passes.
console.log('Cleaning up');
ref.onDisconnect().cancel();
ref.set(
null,
function onSetCompleteFunc(err) {
if (err) {
console.log('Error in onSetCompleteFunc: ' + JSON.stringify(err));
}
});
};
storeSafely(1);
// later...
storeSafely(2);
// even later...
storeSafely(3);
I'm effectively using Firebase transactions as a sort of mutex lock:
Store a value at a location via transaction.
Set the onDisconnect for the location to remove the value in case my app dies while working.
Do some stuff.
Remove the onDisconnect for the location, because I'm done with the stuff.
Remove the value at the location.
I do this every few minutes, and it all works great. Things get written and removed perfectly, and the logs show me creating the lock, doing stuff, and then releasing the lock.
The weird part is what happens hours later. Occasionally Firebase has maintenance, and my app gets a bunch of permission denied errors. At the same time this happens, I suddenly start getting a bunch of this output in the logs:
Attempting update 1
Attempting update 2
Attempting update 3
...in other words, it looks like the transactions never fully completed, and they're trying to retry now that the location can't be read any more. It's almost like there's a closure in the transaction() code that never completed, and it's getting re-executed now for some reason.
Am I missing something really important here about how to end a transaction?
(Note: I originally posted this to the Firebase Google Group, but was eventually reminded that code questions are supposed to go to Stack Overflow. I apologize for the cross-posting.)
Just a guess, but I wonder if your updateFunc() function is being called with null when your app gets the permission-denied errors from Firebase. (If so, I could believe that's part of their "Offline Writes" support.)
In any case, you should handle null as a possible state. Saving Transactional Data says:
transaction() will be called multiple times and must be able to handle
null data. Even if there is existing data in your database it may not
be locally cached when the transaction function is run.
I don't know the intricacies of Firebase's transaction mechansim, but I would try changing your .set(null) to set the value to 0 instead, change your .remove() to also set the value with .set(0), and change your line in updateFunc() to:
if (currentData === null || currentData) return;
Unfortunately, that assumes that '/path/to/location' is initially set to 0 at some point. If that's a problem, maybe you can muck around with null versus undefined. For example, it would be nice if Firebase used one of those for non-existent data and another when it's offline.
I have built my own javascript console in a website! Pressing the GO - Button runs the following function, after comments are removed from the code:
var interv;
function go(code) {
code += "clearInterval(interv);";
interv = setInterval(code,1);
}
My problem appears when calling very time-consuming functions or when there's a mistake in the code: it doesn't stop, sometimes even the browser crashes.
What have I done wrong? Or is there a better way to get the same result?
Here's a JSFiddle (it works best in Firefox)
PS: Maybe you like the spirograph function in this jsFiddle. It's similar to a function in this website, but better.
I'm relatively new to CasperJS, have wrote simple scraping scripts, and now I'm in a kind of more difficult task: I want to scrape some sort of data from a list of urls, but some pages some times "fail", I've a captcha solving service because a few of this pages have captcha by default, but phantomjs is rather inconsistent in rendering some captchas, sometimes they load, sometimes they don't.
The solution I thought was to rerun the script with the pages that failed to load the captcha in order to get the amount of data I need. But I don't seem to get it running, I thought of creating a function with the whole thing and then inside the casper.run() method invoke it and check if the amount of data scraped fulfills the minimum I need if not rerun, But I don't really know how to accomplish it, as for what I've seen casperjs adds the steps to the stack before calling the function (correct me if I'm wrong). Also I'm thinking of something using the run.complete event but not so sure how to do it. My script is something like this:
// This variable stores the amount of data collected
pCount = 0;
urls = ["http://page1.com","http://page2.com"];
// Create casperjs instance...
casper.start();
casper.eachThen(urls, function(response) {
if (pCount < casper.cli.options.number) {
casper.thenOpen(response.data, function(response) {
// Here is where the magic goes on
})
}
})
casper.run();
Is there anyway I can wrap the casper.eachThen() block in a function and do something like this?
casper.start();
function sample () {
casper.eachThen(urls, function(response) {
if (pCount < casper.cli.options.number) {
casper.thenOpen(response.data, function(response) {
// Here is where the magic goes on
})
}
})
}
casper.run(sample);
Also, I tried using slimerjs as engine to avoid the "inconsistencies", but I couldn't manage to get working the __utils__.sendAjax() method inside a casper.evaluate() I have, so it's a deal-breaker. Or is there a way to do a GET request asynchronously in a separate instance? if so, I would appreciate your advise
Update: I never managed to solve it with casperjs, I nonetheless found a workaround for my particular use case, check my answer for more info
Maybe with the back function, so something like that :
casper.start()
.thenOpen('your url');
.then(function(){
var count = 0;
if (this.exists("selector contening the captcha")){
//continue the script
}
else if (count==3){
this.echo("in 3 attempts, it failed each time");
this.exit();
}
else{
count++;
casper.back();//back to the previous step, so will re-open the url
}
.run();
I never found a way to do this from casper, this is how I solved it:
There's a program A, that manages user input (in my case written in C#). This program A is the one that executes the casperjs script, and read it's output. If I need to rerun the script, I just output a message with some specifications so that I catch it in the program A.
It may not be the best way, but it worked for me. Hope it helps
The strangest thing just happened to me. I have some javascript that appears to be being executed in the wrong order. This is freaking me out! Observe:
the code
handleKeyDown: function (e) {
console.log("handleKeyDown");
var key = e.which;
var text = this.ui.$input.val();
if (_.isFunction(this[key])) {
// call the appropriate handler method
this[key](text, e);
console.log("before announceEdits");
this.announceEdits();
}
if (key === ENTER || key === ESC) {
console.log("fired field:key:down");
this.trigger("field:key:down", { editable: this, restore: (key === ESC) });
}
},
announceEdits: function () {
console.log("announceEdits");
var edits = this.getEdits();
console.log("edits: %o", edits.data);
console.log("fired field:edits");
this.trigger("field:edits", edits);
},
/* gather all the existing taggies */
getEdits: function () {
var data = this.$taggies().map(function (index, taggy) {
return $(taggy).data("value");
}).toArray();
var edits = {
attribute: "tags",
data: data
};
return edits;
},
When I run this code, the functions appear to be being executed out of order. This is the output in firefox's console of the above code:
the output
Notice that we get before announceEdits long before we get announceEdits, which is the first line in the annouceEdits function... my understanding of the world leads me to believe this is wrong.
what I've done
Now, I have considered the following:
The console statements could be being buffered or some such, causing them to appear out of order.
This could have something to do with the way MarionetteJS handles events.
Believing that this might be evented weirdness, I tried removing the calls to trigger (just by commenting out the lines that trigger the events). After removing the triggers, the log statements still appear out of order. So it doesn't seem to be cause by MarionetteJS's implementation of events (which is to say, BackboneJS's implementation of events ^o^//).
I'm also lead to believe that this isn't a log statement buffer issue, because the events are themselves handled out of order (i.e. the data I expect to have after the handling of the first event is not in order by the time the second event is handled). This causes the code to "not work" in the way I would like (however, see below).
In my explorations, I've tried to narrow things down a bit. I modified the code thusly, in order to simplify the code:
if (_.isFunction(this[key])) {
// call the appropriate handler method
this[key](text, e);
console.log("before announceEdits");
console.log("announceEdits");
var edits = this.getEdits();
console.log("edits: %o", edits.data);
console.log("fired field:edits");
this.trigger("field:edits", edits);
}
This way I am not descending into a subroutine. Running this code, the console statements appear in the right order. What's more interesting is that, in this case, the events are also fired and handled in the order I expect! This code works, but the one with the subroutine doesn't.
I tried to create a fiddle of this code, but jsfiddle doesn't appear to like Backbone (I tried including the library as an external library, but that didn't seem to work). I did create this fiddle, just to reassure myself that somewhere in the world there is still a rock of normality.
update: I changed the fiddle so that the main function is itself a handler. Everything still works fine in the fiddle.
how I thought the world worked
Functions create stack-frames that execute to completion. If a function calls another function, a new stack-frame is pushed to the stack. If a function triggers an event, it creates a message, which is pushed onto a message queue. Whenever (and I'm unclear on this) the stack is empty, the next message in the queue is popped off, and all handlers for this message are then invoked.
In this model, without question, the events should happen in the order that I expect (the order from the second listing). We should first descend into announceEdits, push the field:edits message to the queue, and then pop back out and push the field:key:down message. Then the handlers for field:edits should run, and finally those for field:key:down.
the question
Is there anything that could be causing these functions to be being executed out of order? Or more likely: is there anything that could be causing these functions to appear to be being executed out of order?
If this is a problem with my understanding: where did I go wrong?
If this ends up being something that was caused by a typo, please bear in mind that we are all programmers, and that we have all torn out hair and bellowed at shadows at times.
lesson
Sleep on it. In the heat of the moment, every bug is a mysterious force from another world. Step back from the problem, start from the beginning. Everything will become clear.
I came in this morning, put a debugger at the beginning of handleKeyDown and immediately saw what was wrong. It isn't in the code I gave above, naturally, and it isn't a problem with how javascript works (clearly!). I had wrapped announceEdits in a debounce earlier, to relieve a head-ache I had been having.
Good news is, my understanding of how event handling works does not appear to be in question.