We Keep Coding

What causes these JS scripts to prepend undefined to values in first tsv output column?

I've created a number of JS scripts similar to below which generate a .tsv download of the webscraped data (this particular example assumes you're on the URL of a repo's Contributors page on Gitlab). Everything outputs fine when I open the .tsv in Microsoft Excel, except that the string 'undefined' appears prepended to every value after the header row in the first column only
How do I edit the script to omit undefined from appearing? Even if it's a simple fix, it will allow me to clean up a bunch of scripts' similar output scraping other websites.
javascript:(function(){
var arr = new Array, i, commitsemail, commitsnum, email, fullname, matchnum;
var regexnum = /.+?(?=commits)/g; var regexemail = /(?<=\().*(?=\))/g;
var glab = document.querySelectorAll('div.col-lg-6.col-12.gl-my-5');
var strings='Full name'+'\t'+'Email'+'\t'+'# of commits'+'\r\n';
var endurl = document.URL.split(/[/]+/).pop(); if (endurl != 'master') {
alert('You are not on the contributors page of a Gitlab repo. Press Esc key, go to URL ending with /master and re-run this bookmarklet'); } else {
for (i = 0; i<glab.length; i++) {
fullname = glab[i].getElementsByTagName('h4')[0].textContent;
commitsemail = glab[i].getElementsByTagName('p')[0].textContent;
commitsnum = [...commitsemail.match(regexnum)];
email = [...commitsemail.match(regexemail)];
arr[i] += fullname + '\t' + email + '\t' + commitsnum;
strings += arr[i]+'\r\n'; }
var pom = document.createElement('a');
var csvContent = strings; var blob = new Blob([csvContent],{type: 'text/tsv;charset=utf-8;'});
var url = URL.createObjectURL(blob); pom.href = url; pom.setAttribute('download','gitlab-contributors.tsv'); pom.click(); }
})();

It's because of the += on the line with arr[i] += fullname + '\t' + email + '\t' + commitsnum;. Change that to an = instead.
Before the assignment, arr[i] is undefined. Maybe you mixed up the syntax for assigning an array entry by index, with appending to an array (arr.push(...)), thinking += would push, but it doesn't. It appends the new value to the current value. And since that line is the first time arr[i] is assigned anything, the current value is undefined.

Removing parameter values of a url in the next page using javascript only

I need to remove the values from the url after the ? in the next page the moment i click from my first page. I tried a lot of coding but could not get to a rite path. Need help.
The strings ex- Name, JobTitle and Date are dynamically generated values for ref.
Below are the links associated with the code:
Required url
file:///C:/Users/varun.singh/Desktop/www%20updated%2027.8.2015%20Old/www/Candidates/newOne.html?
Resultant url:
file:///C:/Users/varun.singh/Desktop/www%20updated%2027.8.2015%20Old/www/Candidates/newOne.html?Name=Name%201&JobTitle=Title%201&Date=Entered%20Date%201
listItem.onclick = function(){
var elementData=listData[this.id];
var stringParameter= "Name=" + elementData.name +"&JobTitle="+elementData.job_title+"&Date="+ elementData.entered_date;
//window.location.href = window.location.href.replace("ListCandidateNew", "newOne") + "?" + stringParameter;
window.location.href="file:///C:/Users/varun.singh/Desktop/www%20updated%2027.8.2015%20Old/www/Candidates/newOne.html?"
+ stringParameter;
}

This should work:
var url = file:///C:/Users/varun.singh/Desktop/www%20updated%2027.8.2015%20Old/www/Candidates/newOne.html?Name=Name%201&JobTitle=Title%201&Date=Entered%20Date%201
var index = url.lastIndexOf("?");
url = url.slice(0, index+1); // index+1 so that "?" is included

Thanks everond for trying and attempting to answer my problem. Well, i have found the solution using window.sessionStorage as i wanted by keeping the string parameter alive to pass the values. Here is the full code:
I have two pages for passing the value from one to another: ListCandidateNew.html and newOne.html
ListCandidateNew.html
listItem.onclick = function()
{
var elementData=listData[this.id];
var stringParameter= "Name=" + elementData.name +"&JobTitle="+elementData.job_title+"&Date="+ elementData.entered_date;
window.sessionStorage['Name'] = elementData.name;
window.sessionStorage['JobTitle'] = elementData.job_title;
window.sessionStorage['Date'] = elementData.entered_date;
**newOne.html**
function LoadCandidateDetail()
{
document.getElementById('Name').innerHTML = window.sessionStorage['Name'];
document.getElementById('JobTitle').innerHTML = window.sessionStorage["JobTitle"];
document.getElementById('Date').innerHTML = window.sessionStorage["Date"];
}

Extract string html with regex on interval

There is a public website with this in the source:
</div><script type="text/rocketscript">
function calculateIndexIncome() {
var khs = $('#t9').val();
var btcusd = $('#t9_1').val();
var btckhs = $('#t9_2').val();
var dayprofitperkhs = 0.00000018188885404454654
var arr = btcusd.split(' ');
btcusd = arr[0];
var totalinvestmentusd = ((khs * btckhs) * btcusd).toFixed(2);
var totalinvestmentbtc = (khs * btckhs).toFixed(8);
var dailyincomebtc = (khs * dayprofitperkhs).toFixed(8);
var dailyincomeusd = ((khs * dayprofitperkhs) * btcusd).toFixed(2);
var monthlyincomebtc = (dailyincomebtc * 31).toFixed(8);
var monthlyincomeusd = (dailyincomeusd * 31).toFixed(2);
var breakevendays = (totalinvestmentusd / dailyincomeusd).toFixed(0);
var monthlypercentage = ((100 / breakevendays) * 30).toFixed(2);
$('#tl').html('Total KHS: ' + khs + '<br/>Total Investment: ' + totalinvestmentbtc + ' BTC ($' + totalinvestmentusd + ' USD)<br/><br/>Daily Income: ' + dailyincomebtc + ' BTC ($' + dailyincomeusd + ' USD)<br/>Monthly Income: ' + monthlyincomebtc + ' BTC ($' + monthlyincomeusd + ' USD)<br/><br/>Break Even In: ' + breakevendays + ' Days.<br/><br/>Monthly Rate: ' + monthlypercentage + '%');
}
I need to be able to extract two values: btckhs and dayprofitperkhs.
if I look at page source, dayprofitperkhs is different everytime I refresh.
Edit:
Jimmy Chandra came up with this bookmarklet:
javascript:
setInterval(logging,60000);
w1 = window.open("https://scrypt.cc/index.php");
function logging(){
console.log (w1.$('#t9_2').val());
var re=/var\s*dayprofitperkhs\s*=\s*([0-9\.]+)\s*/gi;
var matches=re.exec(document.body.innerHTML);
console.log(RegExp.$1);
w1.location.href = 'https://scrypt.cc/index.php';
}
This works ALMOST perfectly. it gets the dayprofitperkhs, but only on the first interval.
After that, the value is no longer updated, although t9_2 IS updated...
Anyone?

I don't know where that site is, so I am just running this against this SO question, but the following bookmarklet is getting me what I want...
As I mentioned in the comment, I use Regular Expression against the document body inner html and I am looking for dayprofitperkhs and capturing the numbers and decimal separator on the right side of the equal sign. Also trying to compensate for any extra spaces in between (\s*). RegExp.$1 gave me the number that I am looking for.
javascript:(function(){var re=/var\s*dayprofitperkhs\s*=\s*([0-9\.]+)\s*/gi;var matches=re.exec(document.body.innerHTML);console.log(RegExp.$1);}());
So your final bookmarklet should be something like:
javascript:
setInterval(logging,60000);
w1 = window.open("siteurl.com");
function logging(){
console.log (w1.$('#t9_2').val());
var re=/var\s*dayprofitperkhs\s*=\s*([0-9\.]+)\s*/gi;
var matches=re.exec(w1.document.body.innerHTML);
console.log(RegExp.$1);
w1.location.href = 'siteurl.com';
}

The variables in question are local variables within the calculateIndexIncome() function, so no, you can't access them from outside that function.
The reason the first one "works" is because you're not referring to the variable, but rather the value: $('#t9_2').val(). This is a jquery selector which finds the element with the ID t9_2 and grabs its value.

You cannot visit it because its a local variable, it only exists in calculateIndexIncome() function.
By the way, you needn't open a new window to visit the variables. You can use chrome dev tools to directly modify the javascript to print the values, or set a breakpoint to debug the code.
Here is a tutorial for chrome dev tools: https://www.codeschool.com/courses/discover-devtools

Unexpected Number error

I have been writing a function to allow users to upload images from their local file system to a website using JavaScript. I have successfully been able to upload images to the browser.
I have also written a function to allow the user to delete these images.
var count = 0;
function getPhoto(){
var file = document.getElementById('ad_photo');
var list = document.getElementById('ad_photo_upload');
var fReader = new FileReader();
var photo_list = [];
var counter;
fReader.readAsDataURL(file.files[0]);
fReader.onloadend = function(event){
counter = count.toString();
list.innerHTML += "<li id = 'pic " + counter + "'><img src='" + event.target.result + "'></img><a class = 'close' onclick = 'rem_pic(pic " + counter + ")'>X</a></li>";
photo_list[count] = event.target.result;
count++;
}
}
function rem_pic(theID){
var element = document.getElementById(theID);
element.outerHTML = "";
delete element;
}
My issue is whenever I call the "rem_pic(theID)" function I get a Chrome Browser error message that says "Uncaught SyntaxError: Unexpected number". Does anyone have any clue to why this might be? And how I could possibly improve the functions I have written so they work correctly?
Thanks

This happens because you pass a number to your function:
'rem_pic(pic " + counter + ")'
will render to
'rem_pic(pic 1)'
^ or any other number according to your counter value
And this is wrong as javascript params can't contain spaces.
So you probably need to pass a string:
rem_pic(\"pic " + counter + "\")
Looking at your code seems like you use it's as HTML id attribute. id attribute can't contain space chars too so your code should be like
rem_pic(\"pic" + counter + "\")
if your id in layout has format id="pic1", id="pic2", etc.

Updating a ScriptProperty to avoid retrieving duplicate Twitter statuses

I was interested in writing a twitter bot to help out some friends at a local ski resort. I found this tutorial from Amit Agarwal which gave me enough to get started (it did take me more than 5 minutes since I did a lot of modifying). I host the script on google docs.
FIRST I think this is javascript (my understanding is that google apps script uses javascript...) and when I have had problems with the code so far, google searches for javascript-such-and-such have been helpful, but if this is not actually javascript, please let me know so I can update the tag accordingly!
I have no prior experience with javascript, so I am pretty happy that it's actually working. But I want to see if I'm doing this right.
The start function initiates the trigger, which kicks off the fetchTweets() function every interval (30 minutes). In order to avoid duplicates (the first errors I encountered) & potentially being flagged as spam, I needed a way to ensure that I was not posting the same tweets over and over again. Within the start() function, the initial since_id value is assigned:
ScriptProperties.setProperty("SINCE_TWITTER_ID", "404251049889759234");
Within the fetchTweet() function, I think I am updating this property with the statement:
ScriptProperties.setProperty("SINCE_TWITTER_ID", lastID + '\n');
Is this a good way to do this? Or is there a better/more reliable way? And if so, how can I be sure it's updating the property? (I can check the log file and it seems to be doing it, so I probably just need to create a permanent text file for the logger).
Any help is greatly appreciated!!
/** A S I M P L E T W I T T E R B O T **/
/** ======================================= **/
/** Written by Amit Agarwal #labnol on 03/08/2013 **/
/** Modified by David Zemens #agnarchy on 11/21/2013 **/
/** Tutorial link: http://www.labnol.org/?p=27902 **/
/** Live demo at http://twitter.com/DearAssistant **/
/** Last updated on 09/07/2013 - Twitter API Fix **/
function start() {
Logger.log("start!" + '\n')
// REPLACE THESE DUMMY VALUES
// https://script.google.com/macros/d/18DGYaa-jbaAK9rEv0HZ2cMcWjFGgkvVcvr6TfksMNbbu2Brk3gZeZ46R/edit
var TWITTER_CONSUMER_KEY = "___REDACTED___";
var TWITTER_CONSUMER_SECRET = "___REDACTED___";
var TWITTER_HANDLE = "___REDACTED___";
var SEARCH_QUERY = "___REDACTED___" + TWITTER_HANDLE;
// Store variables
ScriptProperties.setProperty("TWITTER_CONSUMER_KEY", TWITTER_CONSUMER_KEY);
ScriptProperties.setProperty("TWITTER_CONSUMER_SECRET", TWITTER_CONSUMER_SECRET);
ScriptProperties.setProperty("TWITTER_HANDLE", TWITTER_HANDLE);
ScriptProperties.setProperty("SEARCH_QUERY", SEARCH_QUERY);
ScriptProperties.setProperty("SINCE_TWITTER_ID", "404251049889759234");
// Delete exiting triggers, if any
var triggers = ScriptApp.getScriptTriggers();
for(var i=0; i < triggers.length; i++) {
ScriptApp.deleteTrigger(triggers[i]);
}
// Setup trigger to read Tweets every 2 hours
ScriptApp.newTrigger("fetchTweets")
.timeBased()
.everyMinutes(30)
//.everyHours(2)
.create();
}
function oAuth() {
//Authentication
var oauthConfig = UrlFetchApp.addOAuthService("twitter");
oauthConfig.setAccessTokenUrl("https://api.twitter.com/oauth/access_token");
oauthConfig.setRequestTokenUrl("https://api.twitter.com/oauth/request_token");
oauthConfig.setAuthorizationUrl("https://api.twitter.com/oauth/authorize");
oauthConfig.setConsumerKey(ScriptProperties.getProperty("TWITTER_CONSUMER_KEY"));
oauthConfig.setConsumerSecret(ScriptProperties.getProperty("TWITTER_CONSUMER_SECRET"));
}
function fetchTweets() {
oAuth();
// I put this line in to monitor whether the property is getting "stored" so as to avoid
// reading in duplicate tweets.
Logger.log("Getting tweets since " + ScriptProperties.getProperty("SINCE_TWITTER_ID"))
var twitter_handle = ScriptProperties.getProperty("TWITTER_HANDLE");
var search_query = ScriptProperties.getProperty("SEARCH_QUERY")
Logger.log("searching tweets to " + search_query + '\n');
// form the base URL
// restrict to a certain radius ---:
//var search = "https://api.twitter.com/1.1/search/tweets.json?count=5&geocode=42.827934,-83.564306,75mi&include_entities=false&result_type=recent&q=";
// unrestricted radius:
var search = "https://api.twitter.com/1.1/search/tweets.json?count=5&include_entities=false&result_type=recent&q=";
search = search + encodeString(search_query) + "&since_id=" + ScriptProperties.getProperty("SINCE_TWITTER_ID");
var options =
{
"method": "get",
"oAuthServiceName":"twitter",
"oAuthUseToken":"always"
};
try {
var result = UrlFetchApp.fetch(search, options);
var lastID = ScriptProperties.getProperty("SINCE_TWITTER_ID");
if (result.getResponseCode() === 200) {
var data = Utilities.jsonParse(result.getContentText());
if (data) {
var tweets = data.statuses;
//Logger.log(data.statuses);
for (var i=tweets.length-1; i>=0; i--) {
// Make sure this is a NEW tweet
if (tweets[i].id > ScriptProperties.getProperty("SINCE_TWITTER_ID")) {
lastID = (tweets[i].id_str);
var answer = tweets[i].text.replace(new RegExp("\#" + twitter_handle, "ig"), "").replace(twitter_handle, "");
// I find this TRY block may be necessary since a failure to send one of the tweets
// may abort the rest of the loop.
try {
Logger.log("found >> " + tweets[i].text)
Logger.log("converted >> " + answer + '\n');
sendTweet(tweets[i].user.screen_name, tweets[i].id_str, answer.substring(0,140));
// Update the script property to avoid duplicates.
ScriptProperties.setProperty("SINCE_TWITTER_ID", lastID);
Logger.log("sent to #" + tweets[i].user.screen_name + '\n');
} catch (e) {
Logger.log(e.toString() + '\n');
}
}
}
}
}
} catch (e) {
Logger.log(e.toString() + '\n');
}
Logger.log("Last used tweet.id: " + lastID + + "\n")
}
function sendTweet(user, reply_id, tweet) {
var options =
{
"method": "POST",
"oAuthServiceName":"twitter",
"oAuthUseToken":"always"
};
var status = "https://api.twitter.com/1.1/statuses/update.json";
status = status + "?status=" + encodeString("RT #" + user + " " + tweet + " - Thanks\!");
status = status + "&in_reply_to_status_id=" + reply_id;
try {
var result = UrlFetchApp.fetch(status, options);
Logger.log("JSON result = " + result.getContentText() + '\n');
}
catch (e) {
Logger.log(e.toString() + '\n');
}
}
// Thank you +Martin Hawksey - you are awesome
function encodeString (q) {
// Update: 09/06/2013
// Google Apps Script is having issues storing oAuth tokens with the Twitter API 1.1 due to some encoding issues.
// Henc this workaround to remove all the problematic characters from the status message.
var str = q.replace(/\(/g,'{').replace(/\)/g,'}').replace(/\[/g,'{').replace(/\]/g,'}').replace(/\!/g, '|').replace(/\*/g, 'x').replace(/\'/g, '');
return encodeURIComponent(str);
// var str = encodeURIComponent(q);
// str = str.replace(/!/g,'%21');
// str = str.replace(/\*/g,'%2A');
// str = str.replace(/\(/g,'%28');
// str = str.replace(/\)/g,'%29');
// str = str.replace(/'/g,'%27');
// return str;
}

When you use ScriptProperties.setProperty("KEY", "VALUE");, internally Script Properties will overwrite a duplicate key (i.e., if an old Property has the same key, your new one will replace it). So in your case, since you are using the same identifier for the key (SINCE_TWITTER_ID), it will replace any previous Script Property that is that key.
Furthermore, you can view Script Properties via File -> Project properties -> Project properties (tab). Imo Google didn't name that very well. User properties as specific to Google users. Script properties as specific to the Script Project you are working under.
Also, it probably isn't a good idea to include \n in your value when you set the property. That will lead to all sorts of bugs down the road, because you'll have to compare with something like the following:
var valToCompare = "My value\n";
instead of:
var valToCompare = "My value";
because the value in SINCE_TWITTER_ID will actually be "some value\n" after you call your fetchTweet() function.
Of course, one seems more logical I think, unless you really need the line breaks (in which case you should be using them somewhere else, for this application).

Its ok like that thou I dont know why you are adding \n at fhe end. Might confuse other code. You can see script properties in the script's file menu+ properties