Enviroment: Visual Studio 2012, MVC4, Razor, Internet Application.
I'm working with eBay API and I want to show the search results (JSON).
I have a view page with code...
<script>
function _cb_findItemsByKeywords(root)
{
var items = root.findItemsByKeywordsResponse[0].searchResult[0].item || [];
var html = [];
html.push('<table width="100%" border="0" cellspacing="0" cellpadding="3"><tbody>');
for (var i = 0; i < items.length; ++i)
{
var item = items[i];
var title = item.title;
var pic = item.galleryURL;
var viewitem = item.viewItemURL;
if (null != title && null != viewitem)
{
html.push('<tr><td>' + '<img src="' + pic + '" border="0">' + '</td>' +
'<td>' + title + '</td></tr>');
}
}
html.push('</tbody></table>');
document.getElementById("results").innerHTML = html.join("");
}
</script>
This line in ".js" file:
var url = "http://ebay.com?..."
How can I execute this url from ".js" file automatically, when I openning this View Page? (This url sending request to Ebay server and receiving data, which will be showed on this View Page.)
I will change a question a little...
If I'm running this code from the View page, everything works fine:
<script src=http://ebay.com?... </script>
How can I receive this part("http://ebay.com?..." as a variable) from ".js" file? Is it possible?
If you just want to send the request, you could add an image to the DOM with that as the src, for instance.
If you want to receive data from the request, you're going to have to do an AJAX call. This is handled quite differently in different browsers, so here's a good idea to use a framework, such as jQuery.
Since the URL is on a different domain than yours, however, you won't be able to access it with a regular AJAX request. You'd have to refer to what is called a JSONP request. This requires that the document you're fetched is formatted in a specific manner to allow this. If it isn't, JavaScript simply won't allow this interaction, due to the Same-Origin Policy.
JSONP requires that the remote document has the following format:
someCallbackFunction(javaScriptObjectWithData);
If it does, you'd be able to include a script file to the DOM with that URL as the src, the content of the document, once fetched, will be immediately executed in your browser. You should by then have specified a callback function with a name matching the callback being made in the document (this is usually something you can specify with through querystrings in the original request).
If none of these options are available for you, because of the format of the remote document, then you're going to have to request the document from server side. If you don't have access to a serverside environment yourself, in order to do this, there is the option of using somebody elses server. Yahoo's custom query language – YQL – can be used for querying the content of remote documents, and YQL is available through JSONP, so you could possibly relay your request through them.
See this post on using YQL with JSONP
Update, now that you've added more data, eBay API is available for JSONP, and I think that's the solution you're looking for.
Resolved...
<script src="/Scripts/ebay.js" type="text/javascript"></script>
<script>
s = document.createElement( 'script' );
s.src = url;
document.body.appendChild( s );
</script>
Related
I've seen some answers to this that refer the askee to other libraries (like phantom.js), but I'm here wondering if it is at all possible to do this in just node.js?
Considering my code below. It requests a webpage using request, then using cheerio it explores the dom to scrape the page for data. It works flawlessly and if everything had gone as planned, I believe it would have outputted a file as i imagined in my head.
The problem is that the page I am requesting in order to scrape, build the table im looking at asynchronously using either ajax or jsonp, i'm not entirely sure how .jsp pages work.
So here I am trying to find a way to "wait" for this data to load before I scrape the data for my new file.
var cheerio = require('cheerio'),
request = require('request'),
fs = require('fs');
// Go to the page in question
request({
method: 'GET',
url: 'http://www1.chineseshipping.com.cn/en/indices/cbcfinew.jsp'
}, function(err, response, body) {
if (err) return console.error(err);
// Tell Cherrio to load the HTML
$ = cheerio.load(body);
// Create an empty object to write to the file later
var toSort = {}
// Itterate over DOM and fill the toSort object
$('#emb table td.list_right').each(function() {
var row = $(this).parent();
toSort[$(this).text()] = {
[$("#lastdate").text()]: $(row).find(".idx1").html(),
[$("#currdate").text()]: $(row).find(".idx2").html()
}
});
//Write/overwrite a new file
var stream = fs.createWriteStream("/tmp/shipping.txt");
var toWrite = "";
stream.once('open', function(fd) {
toWrite += "{\r\n"
for(i in toSort){
toWrite += "\t" + i + ": { \r\n";
for(j in toSort[i]){
toWrite += "\t\t" + j + ":" + toSort[i][j] + ",\r\n";
}
toWrite += "\t" + "}, \r\n";
}
toWrite += "}"
stream.write(toWrite)
stream.end();
});
});
The expected result is a text file with information formatted like a JSON object.
It should look something like different instances of this
"QINHUANGDAO - GUANGZHOU (50,000-60,000DWT)": {
"2016-09-29": 26.7,
"2016-09-30": 26.8,
},
But since the name is the only thing that doesn't load async, (the dates and values are async) I get a messed up object.
I tried Actually just setting a setTimeout in various places in the code. The script will only be touched by developers that can afford to run the script several times if it fails a few times. So while not ideal, even a setTimeout (up to maybe 5 seconds) would be good enough.
It turns out the settimeouts don't work. I suspect that once I request the page, I'm stuck with the snapshot of the page "as is" when I receive it, and I'm in fact not looking at a live thing I can wait for to load its dynamic content.
I've wondered investigating how to intercept the packages as they come, but I don't understand HTTP well enough to know where to start.
The setTimeout will not make any difference even if you increase it to an hour. The problem here is that you are making a request against this url:
http://www1.chineseshipping.com.cn/en/indices/cbcfinew.jsp
and their server returns back the html and in this html there are the js and css imports. This is the end of your case, you just have the html and that's it. Instead the browser knows how to use and to parse the html document, so it is able to understand the javascript scripts and to execute/run them and this is exactly your problem. Your program is not able to understand that has something to do with the HTML contents. You need to find or to write a scraper that is able to run javascript. I just found this similar issue on stackoverflow:
Web-scraping JavaScript page with Python
The guy there suggests https://github.com/niklasb/dryscrape and it seems that this tool is able to run javascript. It is written in python though.
You are trying to scrape the original page that doesn't include the data you need.
When the page is loaded, browser evaluates JS code it includes, and this code knows where and how to get the data.
The first option is to evaluate the same code, like PhantomJS do.
The other (and you seem to be interested in it) is to investigate the page's network activity and to understand what additional requests you should perform to get the data you need.
In your case, these are:
http://index.chineseshipping.com.cn/servlet/cbfiDailyGetContrast?SpecifiedDate=&jc=jsonp1475577615267&_=1475577619626
and
http://index.chineseshipping.com.cn/servlet/allGetCurrentComposites?date=Tue%20Oct%2004%202016%2013:40:20%20GMT+0300%20(MSK)&jc=jsonp1475577615268&_=1475577620325
In both requests:
_ is a decache parameter to prevent caching.
jc is a name of a JS wrapper function which should be invoked with the result (https://en.wikipedia.org/wiki/JSONP)
So, scrapping the table template at http://www1.chineseshipping.com.cn/en/indices/cbcfinew.jsp and performing two additional requests you will be able to combine them into the same data structure you see in the browser.
I don't even know if my project is possible. After looking around for a few hours and reading up on other Stack Overflow questions, my hopes are slowly diminishing, but it will not stop me from asking!
My Project: To create a simple HTML table categorizing our Sales Team phone activity for my superior. Currently I need something to pull data values from a file and use those values inside the table.
My Problem: Can Javascript even do this? I know it reads cookies on the client side computer, but can it read a file in the same directory as the webpage? (If the webpage is on the company server?)
My Progress: I will update as I find more information.
Update: Many of you are curious about how the file is stored. It is a static webpage (table.html) on our fileserver. The text file (data.txt) will be in the same directory.
I've recently completed a project where i had almost the exact conditions as yourself (the only difference is that users exclusively use IE).
I ended up using JQuery's $.ajax() function, and pulled the data from an XML file.
This solution does require the use of either Microsoft Access or Excel. I used as early as the 2003 version, but later releases work just fine.
My data is held in a table on Access (on Excel i used a list). Once you've created your table in Access; it's honestly as simple as hitting 'Export', saving as XML and then playing around with your 'ajax()' function (http://api.jquery.com/jQuery.ajax/) to manipulate the data which you want to be output, and then CSS/HTML for the layout of your page.
I'd recommend Access as there's less hastle in getting it to export XML in the right manner, though Excel does it just fine with a little more tinkering.
Here's the steps with ms-access:
Create table in access & export as XML
The XML generated will look like:
<?xml version="1.0" encoding="UTF-8"?>
<dataroot xmlns:od="urn:schemas-microsoft-com:officedata" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Calls.xsd" generated="2013-08-12T19:35:13">
<Calls>
<CallID>1</CallID>
<Advisor>Jenna</Advisor>
<AHT>125</AHT>
<Wrap>13</Wrap>
<Idle>6</Idle>
</Calls>
<Calls>
<CallID>3</CallID>
<Advisor>Edward</Advisor>
<AHT>90</AHT>
<Wrap>2</Wrap>
<Idle>4</Idle>
</Calls>
<Calls>
<CallID>2</CallID>
<Advisor>Matt</Advisor>
<AHT>246</AHT>
<Wrap>11</Wrap>
<Idle>5</Idle>
</Calls>
Example HTML
<table id="doclib">
<tr><th>Name</th><th>AHT</th><th>Wrap</th><th>Idle</th></tr>
</table>
jQuery:
$(document).ready(function(){
$.ajax({
type: "GET",
url: "Calls.xml",
dataType: "xml",
success: function(xml) {
$(xml).find('Calls').each(function(){
var advisor = $(this).find('Advisor').text(),
aht = $(this).find('AHT').text(),
wrap = $(this).find('Wrap').text(),
idle = $(this).find('Idle').text(),
td = "<td>",
tdc = "</td>";
$('#doclib').append("<tr>" +
td + advisor + tdc + td + aht + tdc + td + wrap + tdc + td + idle + tdc + "</tr>")
});
}
});
});
JavaScript cannot automatically read files due to security reasons.
You have two options:
If you can rely on IE being used, you could use some fancy ActiveX stuff.
Use a backend which either constantly pushs data to the JS client or provides the data on pull requests.
This could work if you had a server like build with Node.js, PHP, ...etc.
JavaScript can read files with the Ajax protocol, but this mean that you need a server.
Otherwise your requests will go through the file:// protocol which doesn't support Ajax.
You can try looking into FileReader:
https://developer.mozilla.org/en-US/docs/Web/API/FileReader
The FileReader object lets web applications asynchronously read the contents of files (or raw data buffers) stored on the user's computer
I've never personally gotten it to work properly, but it's supposed to be able to allow this sort of thing.
Try with XMLHttpRequest or ActiveXObject in IE 5 or IE 6.
Here you can find an explanation:
http://www.w3schools.com/xml/xml_http.asp
Or try this example:
http://www.w3schools.com/ajax/tryit.asp?filename=tryajax_first
It sounds like you just want to get the contents of a static file from your server; is that right? If that's what you need to do, you're in luck. That's very easy.
load('textTable.txt', function(err, text) {
buildTable(text);
});
function load(url, callback) {
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (xhr.readyState < 4) return;
if (xhr.status !== 200) {
return callback('HTTP Status ' + xhr.status);
}
if (xhr.readyState === 4) {
callback(null, xhr.responseText);
}
};
xhr.open('GET', url, true);
xhr.send('');
}
If you go with qwest, it'll look something like this:
qwest.get('textTable.txt').success(function(text) {
buildTable(text);
});
With jQuery:
jQuery.get('textTable.txt', function(text) {
buildTable(text);
});
This is similar to: How to open a file using JavaScript?
Goal: to retrieve/open a file on an image's double click
function getFile(filename){
// setting mime this way is for example only
var mime = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document';
jQuery.ajax({ url : 'get_file.pl',
data : {filename:filename},
success : function(data){
var win = window.open('','title');
win.document.open(mime);
win.document.write(data);
win.document.close();
}
});
}
jQuery('#imgID').dblclick(function(){
getFile('someFile.docx');
});
I'm doing this off the top of my head, but I think the above would work for text files, but not binary. Is there a plugin that does this properly? The ideal would be to open the file in the browser (or application), rather than download, but I doubt that is a dream. If the file must be downloaded with the save/open dialog, that's fine.
Edit:
One piece of information that I forgot to mention is that I'd like this to be a POST request. This is partly why I was looking at AJAX to begin with. I've seen workarounds that have created forms/iframes to do something similar, but I was looking for a better handler of the returned info.
Seems to me there's no reason to do this via AJAX. Just open the new window to get_file.pl?filename=... and let the browser handle it. If the user has a plugin capable of handling the Content-Type sent by get_file.pl, the file will display; otherwise, it should download like any other file.
function getFile(filename) {
window.open('get_file.pl?filename=' + filename,'title');
}
jQuery('#imgID').dblclick(function() {
getFile('someFile.docx');
});
Edit: If you want to POST to your script, you can do it with some <form> hackery:
function getFile(filename) {
var win = 'w' + Math.floor(Math.random() * 10000000000000);
window.open('', win,'width=250,height=100');
var f = $('<form></form>')
.attr({target: win, method:'post', action: 'get_file.pl'})
.appendTo(document.body);
var i = $('<input>')
.attr({type:'hidden',name:'filename',value:filename})
.appendTo(f);
f[0].submit();
f.remove();
}
Of course, this is somewhat silly since it is impossible to hide your data from "prying eyes" with developer tools. If your filename really is sensitive, issue access tokens to the client, and look up the data in your sever script.
I'm trying to write a plugin. I can not use any libraries or frameworks.
At any website (domain) I would like to start a script from my own domain.
For example:
In the code of the website under domain A I put a code starting the script from domain B
<script src="http://domain-b.com/myscript.js" type="text/javascript"></script>
The code of JavaScript (myscript.js)
type = 'GET';
url = 'http://domain-b.com/echojson.php';
data = ‘var1=1&var2=2’;
_http = new XMLHttpRequest();
_http.open(type, url + '?callback=jsonp123' + '&' + data, true);
_http.onreadystatechange = function() {
alert(‘Get data: ’ + _http.responseText);
}
_http.send(null);
Script from http://domain-b.com/echojson.php always gives the answer:
jsonp123({answer:”answer string”});
But in a JavaScript console I see an error (200) and AJAX doesn’t get anything.
Script loaders like LAB, yep/nope or Frame.js were designed to get around the same-origin policy. They load a script file, so the requested file would have to look like:
response = {answer:”answer string”};
If you use your code like you have posted it here, it does not work, because you are using apostrophs for the data variable!
I have seen similar questions around, but none of them seem to have answers that help my case...
Basically, I want to load some HTML in using $.ajax() (Which is on a different domain), and have it parsed into it's own DOM, so I can apply attributes and manipulate HTML in my actual window DOM.
$.ajax({
type: 'GET',
url: 'http://example.com/index.html',
dataType: 'html',
crossDomain: true,
cache: false,
success: function(data)
{
var src = $('body img', data).first().attr("src");
//also tried: var src = $('body', $(data)).first().attr("src");
$('#someDiv img').attr("src", src);
}
});
Where an example HTML file is:
<html>
<body>
<img src="someurl"></img>
</body>
</html>
It works in Firefox, but not IE, no matter what I try, whenever I try to parse and read, it returns null.
Any suggestions?
EDIT:
It appears there was some ambiguity with my question. The issue is the parsing, not the AJAX. The AJAX returns the html string correctly, but jQuery fails to parse it.
EDIT 2:
I found a 'solution', but it isn't nearly as nice as I wanted it to be, it chopping and sorting through the HTML string, and extracting data, rather than applying it to a DOM. Seems to run efficiently, as I can predict the order of data.
Boiled down, it is something like this:
var imgsrcs = new Array(5);
var searchItem = '<img src="';
for (var a=0; a<5; a++) {
var startLoc = data.search(searchItem) + searchItem.length;
for (var i=0; i<data.length; i++) {
if (data.charAt(startLoc + i) == '"')
break;
imgsrcs[a] += data.charAt(startLoc + i);
}
data = data.substring(startLoc + i, data.length);
}
$('.image').each(function(i) {
$(this).attr("src", imgsrcs[i]);
});
Fairly ugly, but I solved my problem, so I thought I may as well post it.
This is a Same Origin Policy problem.
The crossDomain flag in jquery's ajax function doesn't automatically make cross domain requests work in all browsers (not all browsers support CORS). Since you're requesting this from a different domain, a normal request won't actually be able to read the data (or even make the request).
Normally, for json data, you can do JSONP, which is what the crossDomain often flag enables. However, JSON is unique because it can be natively read in javascript. Since HTML cannot be read, you'd need to wrap it in parseable javascript to employ a trick like JSONP.
Rather than do that on your own, though, I'd highly suggest that you look into the easyXDM library in order to do cross domain messages like this. You'd essentially open up a hidden iframe on the other domain, and pass messages back and forth between the parent and the hidden frame. And, since the hidden frame is on the same domain as the html, it will have no problem ajaxing for it.
http://easyxdm.net/wp/