So I am goofing off and wrote something that first queries other websites and loads all of the pertinent stock exchange symbols and the tries to iterate through the symbols and load Yahoo's server for the latest stock price. For some reason no matter what I try to search by using HtmlAgilityPack, I can't seem to get any pertinent info back. I don't think there's an issue with some javascript running after the page is loaded (but I could be wrong).
The following is a generalized routine that can be tinkered with to try and get the percentage change of the stock symbol back from Yahoo:
string symbol = "stock symbol"
string link = #"http://finance.yahoo.com/q?uhb=uh3_finance_vert&s=" + symbol;
string data = String.Empty;
try
{
// keep in this scope so wedget and doc deconstruct themselves
var webget = new HtmlWeb();
var doc = webget.Load(link);
string percGrn = doc.FindInnerHtml("//span[#id='yfs_pp0_" + symbol + "']//b[#class='yfi-price-change-up']");
string percRed = doc.FindInnerHtml("//span[#id='yfs_pp0_" + symbol + "']//b[#class='yfi-price-change-down']");
// create somthing we'll nuderstand later
if ((String.IsNullOrEmpty(percGrn) && String.IsNullOrEmpty(percRed)) ||
(!String.IsNullOrEmpty(percGrn) && !String.IsNullOrEmpty(percRed)))
throw new Exception();
// adding string to empty gives string
string perc = percGrn + percRed;
bool isNegative = String.IsNullOrEmpty(percGrn);
double percDouble;
if (double.TryParse(Regex.Match(perc, #"\d+([.])?(\d+)?").Value, out percDouble))
data = (isNegative ? 0 - percDouble : percDouble).ToString();
}
catch (Exception ex) { }
finally
{
// now we need to check what we have and load into the datgridView
if (!newData_d.ContainsKey(symbol)) newData_d.Add(symbol, data);
else MessageBox.Show("ERROR: Duplicate stock Symbols for: " + symbol);
}
And here is the extended method for FindInnerHtml:
// this is for the html agility class
public static string FindInnerHtml( this HtmlAgilityPack.HtmlDocument _doc, string _options)
{
var node = _doc.DocumentNode.SelectSingleNode(_options);
return (node != null ? node.InnerText.ToString() : String.Empty);
}
Any help with getting something back would be appreciated, thanks!
///////////////////////////////////////
EDIT:
///////////////////////////////////////
I highlighted the span id and then check out line 239 for where I saw 'yfi-price-change-up' reference:
The following XPath successfully find the target <span> which contains percentage of increase (or decrease) :
var doc = new HtmlWeb().Load("http://finance.yahoo.com/q?uhb=uh3_finance_vert&s=msft");
var xpath = "//span[contains(#class,'time_rtq_content')]/span[2]";
var span = doc.DocumentNode.SelectSingleNode(xpath);
Console.WriteLine(span.InnerText);
output :
(0.60%)
Related
I can't figure out why the following code isn't working properly in case of special characters:
[...]
// get strings (names) from spreadsheet
var persons = SpreadsheetApp.getActiveSheet().getRange(2, 1, 31, 4).getValues();
// for each row
for (var row in persons) {
// build the filename
var myFile = persons[row][1] + "_" + persons[row][0] + "_20180124.txt";
[...]
// handle from Google Drive only file
var driveFiles = DriveApp.getFilesByName(myFile);
while (driveFiles.hasNext()) {
var file = driveFiles.next();
if(driveFiles.getName() == myFile) {
/* write to Log
Saša_KLANJŠČEK_20180124.txt not written
Peter_MARINČIČ_20180124.txt not written
Peter_KLANJŠČEK_20180124.txt not written
Niko_ČERNIC_20180124.txt not written
Tjaša_KOGOJ_20180124.txt written
*/
Logger.log(myFile + "\n");
[...]
Strings with uppercase unicode characters causes the conditional statement to fail. I tried with toString("UTF-8") method, but it still doesn't work. Is it an encoding problem?
Change:
if(driveFiles.getName() == myFile) {
into
if(file.getName() == myFile) {
I'm working on my final project of the Winter 2017 quarter to demonstrate how to use Regular Expressions in both C# and JavaScript code behind pages. I've got the C# version of my demonstration program done, but the JavaScript version is making me pull what little hair I have left on my head out (no small achievement since I got a fresh buzz cut this morning!). The problem involves not getting any output after applying a Regular Expression in a While loop to get each instance of the expression and printing it out.
On my HTML page I have an input textarea, seven radio buttons, an output textarea, and two buttons underneath (one button is to move the output text to the input area to perform multiple iterations of applying expressions, and the other button to clear all textareas for starting from scratch). Each radio button links to a function that applies a regular expression to the text in the input area. Five of my seven functions work; the sixth is the one I can't figure out, and the seventh is essentially the same but with a slightly different RegEx pattern, so if I fix the sixth function, the seventh function will be a snap.
(I tried to insert/upload a JPG of the front end, but the photo upload doesn't seem to be working. Hopefully you get the drift of what I've set up.)
Here are my problem children from my JS code behind:
// RegEx_Demo_JS.js - code behind for RegEx_Demo_JS
var inputString; // Global variable for the input from the input text box.
var pattern; // Global variable for the regular expression.
var result; // Global variable for the result of applying the regular expression to the user input.
// Initializes a new instance of the StringBuilder class
// and appends the given value if supplied
function StringBuilder()
{
var strings = [];
this.append = function (string)
{
string = verify(string);
if (string.length > 0) strings[strings.length] = string;
}
this.appendLine = function (string)
{
string = verify(string);
if (this.isEmpty())
{
if (string.length > 0) strings[strings.length] = string;
else return;
}
else strings[strings.length] = string.length > 0 ? "\r\n" + string : "\r\n";
}
this.clear = function () { strings = []; };
this.isEmpty = function () { return strings.length == 0; };
this.toString = function () { return strings.join(""); };
var verify = function (string)
{
if (!defined(string)) return "";
if (getType(string) != getType(new String())) return String(string);
return string;
}
var defined = function (el)
{
// Changed per Ryan O'Hara's comment:
return el != null && typeof(el) != "undefined";
}
var getType = function (instance)
{
if (!defined(instance.constructor)) throw Error("Unexpected object type");
var type = String(instance.constructor).match(/function\s+(\w+)/);
return defined(type) ? type[1] : "undefined";
}
}
Within the code of the second radio button (which will be the seventh and last function to complete), I tested the ScriptBuilder with data in a local variable, and it ran successfully and produced output into the output textarea. But I get no output from this next function that invokes a While loop:
function RegEx_Match_TheOnly_AllInstances()
{
inputString = document.getElementById("txtUserInput").value;
pattern = /(\s+the\s+)/ig; // Using an Flag (/i) to select either lowercase or uppercase version. Finds first occurrence either as a standalone word or inside a word.
//result = pattern.exec(inputString); // Finds the first index location
var arrResult; // Array for the results of the search.
var sb = getStringBuilder(); // Variable to hold iterations of the result and the text
while ((arrResult = pattern.exec(inputString)) !==null)
{
sb.appendLine = "Match: " + arrResult[0] ;
}
document.getElementById("txtRegExOutput").value = sb.toString();
/* Original code from C# version:
// string pattern = #"\s+(?i)the\s+"; // Same as above, but using Option construct for case insensitive search.
string pattern = #"(^|\s+)(?i)the(\W|\s+)";
MatchCollection matches = Regex.Matches(userTextInput, pattern);
StringBuilder outputString = new StringBuilder();
foreach (Match match in matches)
{
string outputRegExs = "Match: " + "\"" + match.Value + "\"" + " at index [" + match.Index + ","
+ (match.Index + match.Length) + "]" + "\n";
outputString.Append(outputRegExs);
}
txtRegExOutput.Text = outputString.ToString();
*/
} // End RegEx_Match_The_AllInstances
I left the commented code in to show what I had used in the C# code behind version to illustrate what I'm trying to accomplish.
The test input/string I used for this function is:
Don’t go there. If you want to be the Man, you have to beat The Man.
That should return two hits. Ideally, I want it to show the word that it found and the index where it found the word, but at this point I'd be happy to just get some output showing every instance it found, and then build on that with the index and possibly the lastIndex.
So, is my problem in my While loop, the way I'm applying the StringBuilder, or a combination of the two? I know the StringBuilder code works, at least when not being used in a loop and using some test data from the site I found that code. And the code for simply finding the first instance of "the" as a standalone or inside another word does work and returns output, but that doesn't use a loop.
I've looked through Stack Overflow and several other JavaScript websites for inspiration, but nothing I've tried so far has worked. I appreciate any help anyone can provide! (If you need me to post any other code, please advise and I'll be happy to oblige.)
I'm having trouble with string manipulation. I have some code written in java, that will use an xls translator to generate some html for me - in string form.
I use spring framework to communicate this string back to my web code, but when the string arrives in the javascript, it fails with an "invalid or unexpected token error. Furthermore, when the string is written to the console, it seems that the string now containts newline characters for each new tag.
For my javascript I really need the html to be all one line.
here are some code bits:
try {
SimpleResultSet rs = dbClient.executeQuery("select MediaContent from call where id = " + callID);
if (rs.next()) {
media = rs.getString("MediaContent");
mimeType = rs.getString("MediaTypeID");
if(media.startsWith("<?xml")) {
trace.info("XSLT: " + xltString);
trace.info("Database XML: " + media);
media = Transform(media, xltString, response);
//trace.info("result HTML: " + media);
if (!media.isEmpty()) {
media = media.replaceAll("\n\r", "")
.replaceAll("\n", "")
.replaceAll(System.lineSeparator(), "");
}
}
//media = media.replaceAll("\"","\\\\\"");
}
} catch (DBException e) {
trace.warning("Failed to get call content media for call id = " + callID, e);
return media;
}
trace.info("cleaned HTML: " + media);
return media == null ? "" : media;
}
At this point the trace printing out the cleaned HTML, shows the string all on one line, without any newline characters. The string is then propagated to the ModelAndView like this:
return new ModelAndView("media", "media", mediaStr);
and on the javascript side:
<script>
var contentString = "${media}";
document.getElementById("mediaContentIFrame").srcdoc = contentString;
it is the contentString variable on the javascript side that fails with the invalid or unexpected token error.
The contentString is used to initialise the srcdoc property of an IFrame.
You have already added two checks \r\n and \n.. Also add \r and i hope it will start working :)
Is it possible to convert mongo objectId into string.
The above pictures shows data i received and shown in console.I need id value in string form .but ObjectId is returning as object
In Database id is look like this- 565d3bf4cefddf1748d1fc5e -objectId and i need id exactly like this –
According to the Mongo documentation:
a 4-byte value representing the seconds since the Unix epoch,
a 3-byte machine identifier,
a 2-byte process id, and
a 3-byte counter, starting with a random value.
You can check it out here: https://docs.mongodb.org/manual/reference/object-id/
So in javascript you could do something like this.
var mId = {
Timestamp:1448950573,
Machine:13565407,
Pid:1756,
Increment:8888962
};
function getId(mongoId) {
var result =
pad0(mongoId.Timestamp.toString(16), 8) +
pad0(mongoId.Machine.toString(16), 6) +
pad0(mongoId.Pid.toString(16), 4) +
pad0(mongoId.Increment.toString(16), 6);
return result;
}
function pad0(str, len) {
var zeros = "00000000000000000000000000";
if (str.length < len) {
return zeros.substr(0, len-str.length) + str;
}
return str;
}
console.log(getId(mId))
It produces "565d3b2dcefddf06dc87a282" which was not exactly the id you had, but that might just be a tweak or i was working with different data :D.
EDIT
Added a padding function so that zeros are not truncated.
Hope that helps
EDIT:
I assume you are using c# to connect to and serve documents from the mongo DB. In that case, there is a driver that also supports toString().
Here is an example using the mongo csharp driver:
using MongoDB.Bson;
using MongoDB.Bson.IO;
using MongoDB.Bson.Serialization;
using MongoDB.Driver;
// ...
string outputFileName; // initialize to the output file
IMongoCollection<BsonDocument> collection; // initialize to the collection to read from
using (var streamWriter = new StreamWriter(outputFileName))
{
await collection.Find(new BsonDocument())
.ForEachAsync(async (document) =>
{
using (var stringWriter = new StringWriter())
using (var jsonWriter = new JsonWriter(stringWriter))
{
var context = BsonSerializationContext.CreateRoot(jsonWriter);
collection.DocumentSerializer.Serialize(context, document);
var line = stringWriter.ToString();
await streamWriter.WriteLineAsync(line);
}
});
}
ORIGINAL:
These are Mongo ObjectId's and if you haven't already deserialised the document they should support a toString method that will return a hexadecimal string.
but if you want this applied to the whole document, using JSON.stringify(MogoDocument) should deserialize this for you into a plain object.
I was interested in writing a twitter bot to help out some friends at a local ski resort. I found this tutorial from Amit Agarwal which gave me enough to get started (it did take me more than 5 minutes since I did a lot of modifying). I host the script on google docs.
FIRST I think this is javascript (my understanding is that google apps script uses javascript...) and when I have had problems with the code so far, google searches for javascript-such-and-such have been helpful, but if this is not actually javascript, please let me know so I can update the tag accordingly!
I have no prior experience with javascript, so I am pretty happy that it's actually working. But I want to see if I'm doing this right.
The start function initiates the trigger, which kicks off the fetchTweets() function every interval (30 minutes). In order to avoid duplicates (the first errors I encountered) & potentially being flagged as spam, I needed a way to ensure that I was not posting the same tweets over and over again. Within the start() function, the initial since_id value is assigned:
ScriptProperties.setProperty("SINCE_TWITTER_ID", "404251049889759234");
Within the fetchTweet() function, I think I am updating this property with the statement:
ScriptProperties.setProperty("SINCE_TWITTER_ID", lastID + '\n');
Is this a good way to do this? Or is there a better/more reliable way? And if so, how can I be sure it's updating the property? (I can check the log file and it seems to be doing it, so I probably just need to create a permanent text file for the logger).
Any help is greatly appreciated!!
/** A S I M P L E T W I T T E R B O T **/
/** ======================================= **/
/** Written by Amit Agarwal #labnol on 03/08/2013 **/
/** Modified by David Zemens #agnarchy on 11/21/2013 **/
/** Tutorial link: http://www.labnol.org/?p=27902 **/
/** Live demo at http://twitter.com/DearAssistant **/
/** Last updated on 09/07/2013 - Twitter API Fix **/
function start() {
Logger.log("start!" + '\n')
// REPLACE THESE DUMMY VALUES
// https://script.google.com/macros/d/18DGYaa-jbaAK9rEv0HZ2cMcWjFGgkvVcvr6TfksMNbbu2Brk3gZeZ46R/edit
var TWITTER_CONSUMER_KEY = "___REDACTED___";
var TWITTER_CONSUMER_SECRET = "___REDACTED___";
var TWITTER_HANDLE = "___REDACTED___";
var SEARCH_QUERY = "___REDACTED___" + TWITTER_HANDLE;
// Store variables
ScriptProperties.setProperty("TWITTER_CONSUMER_KEY", TWITTER_CONSUMER_KEY);
ScriptProperties.setProperty("TWITTER_CONSUMER_SECRET", TWITTER_CONSUMER_SECRET);
ScriptProperties.setProperty("TWITTER_HANDLE", TWITTER_HANDLE);
ScriptProperties.setProperty("SEARCH_QUERY", SEARCH_QUERY);
ScriptProperties.setProperty("SINCE_TWITTER_ID", "404251049889759234");
// Delete exiting triggers, if any
var triggers = ScriptApp.getScriptTriggers();
for(var i=0; i < triggers.length; i++) {
ScriptApp.deleteTrigger(triggers[i]);
}
// Setup trigger to read Tweets every 2 hours
ScriptApp.newTrigger("fetchTweets")
.timeBased()
.everyMinutes(30)
//.everyHours(2)
.create();
}
function oAuth() {
//Authentication
var oauthConfig = UrlFetchApp.addOAuthService("twitter");
oauthConfig.setAccessTokenUrl("https://api.twitter.com/oauth/access_token");
oauthConfig.setRequestTokenUrl("https://api.twitter.com/oauth/request_token");
oauthConfig.setAuthorizationUrl("https://api.twitter.com/oauth/authorize");
oauthConfig.setConsumerKey(ScriptProperties.getProperty("TWITTER_CONSUMER_KEY"));
oauthConfig.setConsumerSecret(ScriptProperties.getProperty("TWITTER_CONSUMER_SECRET"));
}
function fetchTweets() {
oAuth();
// I put this line in to monitor whether the property is getting "stored" so as to avoid
// reading in duplicate tweets.
Logger.log("Getting tweets since " + ScriptProperties.getProperty("SINCE_TWITTER_ID"))
var twitter_handle = ScriptProperties.getProperty("TWITTER_HANDLE");
var search_query = ScriptProperties.getProperty("SEARCH_QUERY")
Logger.log("searching tweets to " + search_query + '\n');
// form the base URL
// restrict to a certain radius ---:
//var search = "https://api.twitter.com/1.1/search/tweets.json?count=5&geocode=42.827934,-83.564306,75mi&include_entities=false&result_type=recent&q=";
// unrestricted radius:
var search = "https://api.twitter.com/1.1/search/tweets.json?count=5&include_entities=false&result_type=recent&q=";
search = search + encodeString(search_query) + "&since_id=" + ScriptProperties.getProperty("SINCE_TWITTER_ID");
var options =
{
"method": "get",
"oAuthServiceName":"twitter",
"oAuthUseToken":"always"
};
try {
var result = UrlFetchApp.fetch(search, options);
var lastID = ScriptProperties.getProperty("SINCE_TWITTER_ID");
if (result.getResponseCode() === 200) {
var data = Utilities.jsonParse(result.getContentText());
if (data) {
var tweets = data.statuses;
//Logger.log(data.statuses);
for (var i=tweets.length-1; i>=0; i--) {
// Make sure this is a NEW tweet
if (tweets[i].id > ScriptProperties.getProperty("SINCE_TWITTER_ID")) {
lastID = (tweets[i].id_str);
var answer = tweets[i].text.replace(new RegExp("\#" + twitter_handle, "ig"), "").replace(twitter_handle, "");
// I find this TRY block may be necessary since a failure to send one of the tweets
// may abort the rest of the loop.
try {
Logger.log("found >> " + tweets[i].text)
Logger.log("converted >> " + answer + '\n');
sendTweet(tweets[i].user.screen_name, tweets[i].id_str, answer.substring(0,140));
// Update the script property to avoid duplicates.
ScriptProperties.setProperty("SINCE_TWITTER_ID", lastID);
Logger.log("sent to #" + tweets[i].user.screen_name + '\n');
} catch (e) {
Logger.log(e.toString() + '\n');
}
}
}
}
}
} catch (e) {
Logger.log(e.toString() + '\n');
}
Logger.log("Last used tweet.id: " + lastID + + "\n")
}
function sendTweet(user, reply_id, tweet) {
var options =
{
"method": "POST",
"oAuthServiceName":"twitter",
"oAuthUseToken":"always"
};
var status = "https://api.twitter.com/1.1/statuses/update.json";
status = status + "?status=" + encodeString("RT #" + user + " " + tweet + " - Thanks\!");
status = status + "&in_reply_to_status_id=" + reply_id;
try {
var result = UrlFetchApp.fetch(status, options);
Logger.log("JSON result = " + result.getContentText() + '\n');
}
catch (e) {
Logger.log(e.toString() + '\n');
}
}
// Thank you +Martin Hawksey - you are awesome
function encodeString (q) {
// Update: 09/06/2013
// Google Apps Script is having issues storing oAuth tokens with the Twitter API 1.1 due to some encoding issues.
// Henc this workaround to remove all the problematic characters from the status message.
var str = q.replace(/\(/g,'{').replace(/\)/g,'}').replace(/\[/g,'{').replace(/\]/g,'}').replace(/\!/g, '|').replace(/\*/g, 'x').replace(/\'/g, '');
return encodeURIComponent(str);
// var str = encodeURIComponent(q);
// str = str.replace(/!/g,'%21');
// str = str.replace(/\*/g,'%2A');
// str = str.replace(/\(/g,'%28');
// str = str.replace(/\)/g,'%29');
// str = str.replace(/'/g,'%27');
// return str;
}
When you use ScriptProperties.setProperty("KEY", "VALUE");, internally Script Properties will overwrite a duplicate key (i.e., if an old Property has the same key, your new one will replace it). So in your case, since you are using the same identifier for the key (SINCE_TWITTER_ID), it will replace any previous Script Property that is that key.
Furthermore, you can view Script Properties via File -> Project properties -> Project properties (tab). Imo Google didn't name that very well. User properties as specific to Google users. Script properties as specific to the Script Project you are working under.
Also, it probably isn't a good idea to include \n in your value when you set the property. That will lead to all sorts of bugs down the road, because you'll have to compare with something like the following:
var valToCompare = "My value\n";
instead of:
var valToCompare = "My value";
because the value in SINCE_TWITTER_ID will actually be "some value\n" after you call your fetchTweet() function.
Of course, one seems more logical I think, unless you really need the line breaks (in which case you should be using them somewhere else, for this application).
Its ok like that thou I dont know why you are adding \n at fhe end. Might confuse other code. You can see script properties in the script's file menu+ properties