Read csv directly from url in html using js - javascript

I am working on html & js in which i display yahoo finance stock in table format. The data get in csv. I want js directly read data from url
The url is http://ichart.finance.yahoo.com/table.csv?s=RIL.BO
The code i try which i get from stackoverflow is working in localhost url.
var txtFile = new XMLHttpRequest();
txtFile.open("GET", "http://ichart.finance.yahoo.com/table.csv?s=RIL.BO", true);
txtFile.onreadystatechange = function() {
if (txtFile.readyState === 4) { // Makes sure the document is ready to parse.
if (txtFile.status === 200) { // Makes sure it's found the file.
allText = txtFile.responseText;
lines = txtFile.responseText.split("\n"); // Will separate each line into an array
alert(allText);
}
}
}
Thanks

In order to get around the Cross Domain request restrictions put in place by the Same Origin Policy, you need an endpoint that allows you to do a JSONP request or that has enabled CORS. Unfortunately, the Yahoo! Finance endpoint has neither.
So, as James mentioned, you ned a middle man.
Usually, my recommendation for this is to use YQL, which allows you to quickly and easily build a server that sits between you and the finance site. In fact, they already have a Yahoo! Finance endpoint for exactly the data you're trying to get: link
However, as that can be unreliable, I also have a website scraper that I've used in various projects. It's hosted on Heroku and allows you to fetch almost any content from any site. I don't recommend using it for high volume projects, but for occaisional data fetches it's great. In your case, you would use it like this:
http://websitescraper.herokuapp.com/?url=http://ichart.finance.yahoo.com/table.csv?s=RIL.BO&callback=jsCallback
Edit: ichart.finance.yahoo.com has been deprecated, so this fails. Keeping it here for reference
Now that you have that out of the way, I recommend using jQuery and the csv-to-array plugin:
jQuery.getJSON('http://websitescraper.herokuapp.com/?url=http://ichart.finance.yahoo.com/table.csv?s=RIL.BO&callback=?', function (csvdata) {
console.log(csvdata.csvToArray());
});
Also, if you want to launch your own middle man, you can use the website-scraper that I've built. The source code is on GitHub and it's released under the MIT license.

You are trying to do a cross domain request so its being blocked.
You will need to write a server side script to fetch the data for you.

Related

How to use GTMetrix Api using Javascript?

I am new to API usage. I have properly managed to utilize Google Page Insights V.5 API through javascript code, but I cannot for the life of me succeed in doing so for GTMetrix. It seems the only information relating to GTMetrix API & Javascript is a link to the RapidApi website. I simply wish to achieve the same simple retrieval of data from GTMetrix as I have from Google. Is this possible?
Am I simply structuring my request incorrectly when I set it as:
https://gtmetrix.com/api/0.1/?login-user=myemail#email.com&login-pass=MyRanDomApIKeY&location=2&url=https://sitetotest.com
Because when I set my Google Page Insights Request URL as the following it works.
https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=https://websitetotest.com&category=performance&strategy=desktop&key=MyRanDomApIKeY
The below code works for Google Page Insights and I am even able to retrieve JSON data in a browser window with a URL such as:
<div id="firstmetric"></div>
<br>
<div id="domSize"></div>
<button>Click Me</button>
<script>
$('button').click(function(){
var baseUrl = "https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=";
var fieldUrl = "https://websitetotest.com";
var trailing = "&category=performance&strategy=desktop&key=MyRanDomApIKeY";
$.getJSON(baseUrl + fieldUrl + trailing, function(data){
console.log(data);
var item = data.lighthouseResult.categories.performance.auditRefs[0].weight;
var domSize = data.lighthouseResult.audits['dom-size'].displayValue;
$("#firstmetric").html( item );
$("#domSize").html( domSize );
});
});
I truly need it spelled out for me because anything less is going to lead me to ask follow up questions and put us in a tail spin. :/
As a newbie, JSFiddle has been a life saving resource for testing and trying, breaking, and building in my learning process. If it wouldn't be too much to ask for, a fiddle would help me get my brain around things.
The parameters that you are using: login-user and login-pass refer to HTTP authentication on the page you are analyzing (as in, GTmetrix will pass these parameters on your analysis) not your GTmetrix API credentials.
The authentication used for the GTmetrix API is your e-mail for the username and your API key as the password, as pointed out by the API docs.
Another thing to keep in mind is that GTmetrix will not allow you to do API calls through your web application frontend, since they disallow CORS requests. If you do it through your Web application on a normal website, you would be exposing your GTmetrix API key, which is probably not a good idea.
So, you would then do it through your backend code. For example if done through Node JavaScript:
fetch("https://gtmetrix.com/api/0.1/locations", {
headers: new Headers({
"Authorization": 'Basic ' + btoa("[YOUR E-MAIL]" + ":" +"[YOUR API KEY]"),
}),
}).then(res => res.json())
.then(response => console.log(response));
would print me the array of locations.
Note that whichever backend code you choose, you need to add the basic authorization header request for you API call and encode it properly (that is what the btoa function call does).

Where to store private API key for access by plain Javascript Azure Web Application? No Web.config or C#

I have a simple website that's hosted on an Azure Web App that's literally just one .html file, one .css file, and one .js file. It uses the Bing API to get data and chartjs to graph that data on the page.
The issue is I obviously don't want to store the API key in the code anywhere anyone else can find it, so I'm not sure where to put it. I tried setting up an Azure Key Vault and adding the API Key as a secret, so I could just do a REST GET and retrieve the key, but I'm not sure how to setup permissions to allow the server the Web app is on (and only that server) to access it.
The more I research do and reading other similar questions the more I feel like this isn't the best solution out there (couldn't anyone just intercept the response in plain text?), but I'm yet to find a "best practice" for this problem. Any alternative solutions or advice on how to make this work are welcome.
EDIT: The more I'm thinking about this the more I realize using the Key Vault won't work. There is no server side to this application, so the request for the API key will come from any IP. I'm thinking I'm going to have add a whole server side application for this to work. Just to give you an idea of what I've got going on so far here's some pseudo code.
----------index.html----------
<head>
<script src = "./chart.js"></script>
</head>
<body>
<canvas id="mainChart" width="400" height="400"></canvas>
<script src="./app.js"></script>
</body>
----------app.js----------
var BING_API_KEY = "myprivatebingapikey";
var CLIENT_ID = "bingapikeyclientid";
function getData() {
return new Promise(resolve => {
var xhttp = new XMLHttpRequest();
xhttp.open("GET", "https://api.cognitive.microsoft.com/bing/v7.0/myquery", true);
xhttp.setRequestHeader("Ocp-Apim-Subscription-Key", BING_API_KEY);
xhttp.setRequestHeader("Accept", "application/json");
xhttp.setRequestHeader("X-MSEdge-ClientID", CLIENT_ID);
xhttp.onreadystatechange = function() {
if (xhttp.readyState == 4 && xhttp.status == 200) {
var response = JSON.parse(xhttp.responseText);
if (!response.value || response.value.length <= 0) {
throw Error;
}
resolve(parseData(response.value));
}
}
xhttp.send(null);
});
}
function parseData(data) {
// put the data into global variables
}
function graphData() {
await getData();
var chart = new Chart(document.getElementById('mainChart'),
data: { // all the data from the global variables },
labels: { // labels from global variables },
options: { // predefined options }
}
graphData();
The code isn't perfect, but it's just to give you an idea about how it's set up.
It looks like you have realised you will need some kind of server side code to keep your secrets secret. Even if you store your secret in a Key Vault that won't help, anyone could just open a web debugger (F12) and see the secret in plain text.
I would suggest an Azure Function with something like an HTTP trigger. This will enable you to build a very light weight server side web service. Put all your getData() code into the Azure Function (you can provide the implementation in JavaScript), along with your secrets. Call the Azure Function in your JavaScript client side to get the data you need for the chart.
An introduction to Azure Functions
Azure Functions is a solution for easily running small pieces of code,
or "functions," in the cloud. You can write just the code you need for
the problem at hand, without worrying about a whole application or the
infrastructure to run it. Functions can make development even more
productive, and you can use your development language of choice, such
as C#, F#, Node.js, Java, or PHP. Pay only for the time your code runs
and trust Azure to scale as needed. Azure Functions lets you develop
serverless applications on Microsoft Azure.

Autodesk forge: Get access token using POST/XMLHttpRequest and Javascript?

I am new to the Autodesk API's, and trying to create the Viewer using javascript on my website, which is hosted by a 3rd party (think Weebly, Squarespace, etc.) that supports JavaScript. I can successfully get an access token if I use Postman to make the POST call to https://developer.api.autodesk.com/authentication/v1/authenticate, but that doesn't help me from a general use perspective. When I try to use XMLHttpRequest and make the same POST call from my javascript, I get an error related to CORS ("No 'Access-Control-Allow-Origin' header is present on the requested resource."). I can't find anywhere where it seems possible to use javascript to call out to Autodesk's API's and create an Autodesk viewer on my own website. Is this possible using javascript alone? Any info would be great.
I am working from the step-by-step API tutorial at https://developer.autodesk.com/en/docs/viewer/v2/tutorials/basic-viewer/, which is great, but doesn't seem to indicate how an actual POST call is worked into your application, instead of getting the token via Postman or some other testing tool.
JavaScript:
function getToken() {
var xhttp = new XMLHttpRequest();
var url = "https://developer.api.autodesk.com/authentication/v1/authenticate";
var params = "client_id=MY_CLIENT_ID&client_secret=MY_CLIENT_SECRET&grant_type=client_credentials&scope=data:read";
xhttp.open("POST", url, true);
xhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xhttp.send(params);
}
Thanks.
You cannot, this is not supported. Autodesk Forge only supports OAuth from the server side, which is more secure. Explaining, your approach requires the Client ID & Secret to be exposed on the client, so the API blocks it via CORS header.
If you are using one of the following programming languages: JavaScript (node.js), .Net, Java, Ruby, simply use the existing SDK. It will make things much easier.
See here for more details.

Twitter OAuth Pin based authorization 'oob' in Google Apps Script

I am trying to use Twitter Pin-based authorization in my Google Apps Script to eventually send tweets on behalf of other uses.
I freely admit that I don't relay know what I'm doing but I have read a lot of info on the internet and feel I have tried everything.
My current Google Apps Script JavaScript code:
var method = 'post';
var url = 'https://api.twitter.com/oauth/request_token';
var consumerKey = '[my consumer key]';
var ticks = '1422745454';
var nonce = '6826266';
var options = {
'method': method,
'oauth_callback': 'oob',
'oauth_consumer_key': consumerKey,
'oauth_nonce': nonce,
'oauth_signature': 'cIFeptE5HjHp7xrp%2BZt9xFhHox4%3D',
'oauth_signature_method': 'HMAC-SHA1',
'oauth_timestamp': ticks,
'oauth_version': '1.0'
};
var response = UrlFetchApp.fetch(url, options);
For testing I set the ticks just before each test run to the value here
The nonce is a random number between 111111 and 9999999 which is regenerated before each test run.
The oauth signature I have been generating with some c# code lifted from the linq2twitter project
I suspect the problem is the signature. I have read the twitter documentation on creating a signature and I think the C# code is doing it correctly but I am not sure.
The problem is that whatever I try I always get this error:
"Request failed for https://api.twitter.com/oauth/request_token returned code 401. Truncated server response: Failed to validate oauth signature and token (use muteHttpExceptions option to examine full response)"
I have been trying to find an example of Twitter Pin-based authorization in a Google Apps Script but have so far not found anything.
My attempts to translate examples in C#, PHP, etc. have also failed.
Please help.
Apps Script provides an Oauth API that works with UrlFetchApp, they even use twitter in their examples. Work with those if at all possible, troubleshooting signature generation is a real hassle.
https://developers.google.com/apps-script/reference/url-fetch/o-auth-config
https://developers.google.com/apps-script/articles/twitter_tutorial
If you absolutely must do it from scratch, the best approach is to get requests working with an existing library (like the c# one you mention), then work on getting your apps script to generate the exact same request.
I get the sense that is what you are doing now, so it may just be a matter of base64 encoding your Signature in the outgoing request:
https://developers.google.com/apps-script/reference/utilities/utilities#base64Encode(String)
Ultimately, it's very difficult to do the whole Oauth process manually in Apps Script. When I tried something like this from scratch about a year ago I ultimately gave up and used a Python application deployed to Google App Engine instead. I submit requests from Apps Script to the App Engine application, and the App Engine application handles Oauth and relays my requests on to the external service, before returning requests to my Apps Script. This approach comes with complications of it's own.

Upload to OneDrive Business using JavaScript, outside Sharepoint

Is it possible to upload a file to Office 365 OneDrive Business (SharePoint), using pure JavaScript or jQuery, when running from an external website? (Not a SharePoint site)
I have written some C# code, which authenticates a user, and gets a FormDigestValue. But I don't know where to go from there.
None of the examples I found, seems to work for me.
you can always use the Office365 REST API to access OneDrive Business. This works well with JavaScript, especially using jQuery or AngularJS since they are more easy to use refering to REST.
You will find some examples here:
http://blogs.msdn.com/b/sharepointdev/archive/2013/08/13/access-skydrive-pro-using-the-sharepoint-2013-apis.aspx
Have you checked the new JS library for O365 APIs, including OneDrive Business service?
It should support CORS.
Yes, it is, I am doing it and it works great..
0) Please, have a look here first, it's a great study and helps to understand CORS:
https://msdn.microsoft.com/en-us/office/office365/howto/create-web-apps-using-cors-to-access-files-in-office-365
1) Register your app (www.yourdomain.name) in Azure AD (Active Directory) - you must have Office 365 and Azure AD subscription - make sure you change the manifest in Azure AD and set OAuth Implicit = Yes This allows you to get across
2) make sure you're able to receive oAuth token on your site which is registered in Azure AD - again (your.domain.name)
3) Then you AJAX (http://www.yourdomain.name) should have something like this:
xhr.open("PUT", OneDriveForBusinessEndPointURL, true);
xhr.setRequestHeader("Authorization", "Bearer " + token);
xhr.setRequestHeader("accept", "application/json");
xhr.setRequestHeader("Content-Type", fileInput.files[0].type);
xhr.send(fileInput.files[0]);
var FileName = fileInput.files[0];
//// Your Endpoint should be like this
var OneDriveForBusinessEndPointURl = https://mydomain-mysharepoint.com/_api/v2.0/drives/{drive-id}/items/{folder-id}:/FileName:/content
Once you get JSON with xhr.status = 201 or 200, you've won.

Categories