Lower intermediate JS/JQ person here.
I'm trying to escape callback hell by using JS fetch. This is billed as "the replacement for AJAX" and seems to be pretty powerful. I can see how you can get HTML and JSON objects with it... but is it capable of running another JS script from the one you're in? Maybe there's another new function in ES6 to do:
$.getScript( 'xxx.js' );
i.e.
$.ajax({ url : 'xxx.js', dataType : "script", });
...?
later, response to Joseph The Dreamer:
Tried this:
const createdScript = $(document.createElement('script')).attr('src', 'generic.js');
fetch( createdScript )...
... it didn't run the script "generic.js". Did you mean something else?
Fetch API is supposed to provide promise-based API to fetch remote data. Loading random remote script is not AJAX - even if jQuery.ajax is capable of that. It won't be handled by Fetch API.
Script can be appended dynamically and wrapped with a promise:
const scriptPromise = new Promise((resolve, reject) => {
const script = document.createElement('script');
document.body.appendChild(script);
script.onload = resolve;
script.onerror = reject;
script.async = true;
script.src = 'foo.js';
});
scriptPromise.then(() => { ... });
SystemJS is supposed to provide promise-based API for script loading and can be used as well:
System.config({
meta: {
'*': { format: 'global' }
}
});
System.import('foo.js').then(() => { ... });
There are a few things to mention on here.
Yes, it is possible to execute a javascript just loaded from the server. You can fetch the file as text and user eval(...) while this is not recommended because of untrackeable side effects and lack of security!
Another option would be:
1. Load the javascript file
2. Create a script tag with the file contents (or url, since the browser caches the file)
This works, but it may not free you from callback hell perse.
If what you want is load other javascript files dinamically you can use, for example requirejs, you can define modules and load them dinamically. Take a look at http://requirejs.org/
If you really want to get out of the callback hell, what you need to do is
Define functions (you can have them in the same file or load from another file using requirejs in the client, or webpack if you can afford a compilation before deployment)
Use promises or streams if needed (see Rxjs https://github.com/Reactive-Extensions/RxJS)
Remember that promise.then returns a promise
someAsyncThing()
.then(doSomethingAndResolveAnotherAsncThing)
.then(doSomethingAsyncAgain)
Remember that promises can be composed
Promise.all(somePromise, anotherPromise, fetchFromServer)
.then(doSomethingWhenAllOfThoseAreResolved)
yes u can
<script>
fetch('https://evil.com/1.txt').then(function(response) {
if (!response.ok) {
return false;
}
return response.blob();
}) .then(function(myBlob) {
var objectURL = URL.createObjectURL(myBlob);
var sc = document.createElement("script");
sc.setAttribute("src", objectURL);
sc.setAttribute("type", "text/javascript");
document.head.appendChild(sc);
})
</script>
dont listen to the selected "right" answer.
Following fetch() Api works perfectly well for me, as proposed by answer of #cnexans (using .text() and then .eval()). I noticed an increased performance compared to method of adding the <script> tag.
Run code snippet to see the fetch() API loading async (as it is a Promise):
// Loading moment.min.js as sample script
// only use eval() for sites you trust
fetch('https://momentjs.com/downloads/moment.min.js')
.then(response => response.text())
.then(txt => eval(txt))
.then(() => {
document.getElementById('status').innerHTML = 'moment.min.js loaded'
// now you can use the script
document.getElementById('today').innerHTML = moment().format('dddd');
document.getElementById('today').style.color = 'green';
})
#today {
color: orange;
}
<div id='status'>loading 'moment.min.js' ...</div>
<br>
<div id='today'>please wait ...</div>
The Fetch API provides an interface for fetching resources (including across the network). It will seem familiar to anyone who has used XMLHttpRequest, but the new API provides a more powerful and flexible feature set. https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
That's what it's supposed to do, but unfortunately it doesn't evaluate the script.
That's why I released this tiny Fetch data loader on Github.
It loads the fetched content into a target container and run its scripts (without using the evil eval() function.
A demo is available here: https://www.ajax-fetch-data-loader.miglisoft.com
Here's a sample code:
<script>
document.addEventListener('DOMContentLoaded', function(event) {
fetch('ajax-content.php')
.then(function (response) {
return response.text()
})
.then(function (html) {
console.info('content has been fetched from data.html');
loadData(html, '#ajax-target').then(function (html) {
console.info('I\'m a callback');
})
}).catch((error) => {
console.log(error);
});
});
</script>
Related
I have two websites, the first get web source code of the second (The two websites aren't one same host -> CORS). (The second website is not mine)
Example:
fetch("https://api.allorigins.win/get?url=" + url)
.then(response => {
if (response.ok) {
return response.json();
}
throw new Error('Network response was not ok.');
})
.then(data => {
var html = stringToHTML(data.contents);
});
It works, except that the concern is that the second page displays other elements several seconds after being displayed, so it does not display me because I retrieved the page too early.
How to make it wait a few seconds before recovering, while not forgetting that "api.allorigins.win"?
Do you have an idea? ( I use Vanillia JS )
Allorigins that must wait for rendering, but it does not.
Your alternative is to implement your own version of allorigins using an headless browser that waits the page render before returning it html. There's no ready solution for it
I don't know if you're using a framework or any library to handle the DOM, but with vanilla JS you can do something like this to check if your DOM element is ready
const DOMReadyCheck = setInterval(() => {
if (document.getElementsBy...) { //get your element here
//send your fetch request and set your element with data
clearInterval(DOMReadyCheck);
}
I'm trying to load an external script in <head> like this:
<script async type='text/javascript' src='//externalprovider.net/our_script.js'></script>
This script declares a function window.manager.loadFunction(sid, mid); which I then use in some components like this:
componentDidMount() {
window.manager.loadFunction(this.props.ids[0], this.props.ids[1]);
}
My problem is that the script fetch from the exteral provider is slow enough so that the component has time to mount before the script is ready. This gives the error Uncaught TypeError: Cannot read property 'loadFunction' of undefined.
Just to check, I copied the full and script pasted it into the <head>, which worked exactly as expected. So my conclusion is that the script loads too slow from the external provider. Unfortunately this will not work for us, because the external provider needs to be able to change the script dynamically, so I can't have it hardcoded in our <head>.
So I thought that I would make a service that uses axios to load the script and then evaluates it, like this:
async getAdsScript() {
try {
const url = '//externalprovider.net/our_script.js';
const src = (await axios.get(url)).data;
const head = document.getElementsByTagName('head')[0];
const newScriptTag = document.createElement('script');
newScriptTag.type = 'text/javascript';
newScriptTag.async = true;
newScriptTag.text = src;
head.appendChild(newScriptTag);
PubSub.publish('AdsScript.loaded', true);
} catch (error) {
console.warn(error);
}
}
I then want to use it like this:
loadFunction() {
window.manager.loadFunction(this.props.ids[0], this.props.ids[1]);
}
componentDidMount() {
PubSub.subscribe('AdsScript.loaded', (msg, showing) =>
this.loadFunction());
}
But this did not work either, because I got a CORS error Access to XMLHttpRequest at 'https://externalprovider.net/our_script.js' from origin 'https://dev.testserver.net' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. and we cannot unfortunately ask them to add CORS policies to their servers.
Does anyone hav an idea how I can solve this?
Yes, as you have already answered, you can just remove the async attribute from script tag. But then your page will be blocked until the external script loads. If you will have a slow connection to the external server, this can lead to slow initialization of your app.
If you want to execute your JS code and render app until external script loads, you can make one of two things:
You can use the code that you wrote in question and set up Nginx on your side that will just proxy requests to the external server. After that, you will make a request to your Nginx to get the external code. This will fix the issue with CORS.
OR
You can use Deferred for waiting until the external script loads. You can find Deferred in some lib on npm or write by yourself. That's how it will look like:
<script>
function Deferred() {
let resolve = null;
let reject = null;
const promise = new Promise((res, rej) => {
resolve = res;
reject = rej;
});
return {
promise,
resolve,
reject
}
}
window.waitForExternalScript = Deferred();
function onExternalScriptLoad() {
window.waitForExternalScript.resolve();
}
</script>
<script type="text/javascript" src="./your_main_script_here.js"></script>
<script async type="text/javascript" src="//externalprovider.net/our_script.js" onload="onExternalScriptLoad()"></script>
And then in your app:
componentDidMount() {
window.waitForExternalScript.promise.then(() => {
window.manager.loadFunction(this.props.ids[0], this.props.ids[1])
})
}
The obvious answer was to remove async from the script tag in <head>:
<script type='text/javascript' src='//externalprovider.net/our_script.js'></script>
I didn't see that because I was staring myself blind on other parts of the code.
I'm downloading a webpage using the request module which is very straight forward.
My problem is that the page I'm trying to download has some async scripts (have the async attributes) and they're not downloaded with the html document return from the http request.
My question is how I can make an http request with/with-out (preferably with) request module, and have the WHOLE page download without exceptions as described above due to some edge cases.
Sounds like you are trying to do webscraping using Javascript.
Using request is a very fundemental approach which may be too low-level and tiome consuming for your needs. The topic is pretty broad but you should look into more purpose built modules such as cheerio, x-ray and nightmare.
x-ray x-ray will let you select elements directly from the page in a jquery like way instead of parsing the whole body.
nightmare provides a modern headless browser which makes it possible for you to enter input as though using the browser manually. With this you should be able to better handle the ajax type requests which are causing you problems.
HTH and good luck!
Using only request you could try the following approach to pull the async scripts.
Note: I have tested this with a very basic set up and there is work to be done to make it robust. However, it worked for me:
Test setup
To set up the test I create a html file which includes a script in the body like this: <script src="abc.js" async></script>
Then create temporary server to launch it (httpster)
Scraper
"use strict";
const request = require('request');
const options1 = { url: 'http://localhost:3333/' }
// hard coded script name for test purposes
const options2 = { url: 'http://localhost:3333/abc.js' }
let htmlData // store html page here
request.get(options1)
.on('response', resp => resp.on('data', d => htmlData += d))
.on('end', () => {
let scripts; // store scripts here
// htmlData contains webpage
// Use xml parser to find all script tags with async tags
// and their base urls
// NOT DONE FOR THIS EXAMPLE
request.get(options2)
.on('response', resp => resp.on('data', d => scripts += d))
.on('end', () => {
let allData = htmlData.toString() + scripts.toString();
console.log(allData);
})
.on('error', err => console.log(err))
})
.on('error', err => console.log(err))
This basic example works. You will need to find all js scripts on the page and extract the url part which I have not done here.
I have been trying to get basic JSONP working in Dart and I am getting stuck. Reading this blog post as well as this this blog show that I should use window.on.message.add(dataReceived); to get a MessageEvent and retrieve data from the event.
Dart complains that "There is no such getter 'message' in events". In addition, I looked up different ways of getting a MessageEvent but it seems to be something completely unrelated (WebSockets?) and is not what I actually need.
If anybody can explain what is going on and how to really use JSONP in Dart, that would be awesome!
You don't need to use what is described in the articles you point anymore. You can use dart:js :
import 'dart:html';
import 'dart:js';
void main() {
// Create a jsFunction to handle the response.
context['processData'] = (JsObject jsonDatas) {
// call with JSON datas
};
// make the call
ScriptElement script = new Element.tag("script");
script.src = "https://${url}?callback=processData";
document.body.children.add(script);
}
I recently wrote a blog post on this myself as I was running into similar problems.
I first cover a few prerequisite things like Verifying CORS Compliance and Verifying JSONP Support
I too ended up registering with the updated method:
window.onMessage.listen(dataReceived);
I then had a fairly simple method to dynamically create the script tag in Dart as well (my requirement was that I had to use Dart exclusively and couldn't touch the website source files):
void _createScriptTag()
{
String requestString = """function callbackForJsonpApi(s) {
s.target="dartJsonHandler";
window.postMessage(JSON.stringify(s), '*');
}""";
ScriptElement script = new ScriptElement();
script.innerHtml = requestString;
document.body.children.add(script);
}
I then invoked it from Dart with some simple logic that I wrapped in a method for convenience.
void getStockQuote(String tickerId)
{
String requestString = "http://finance.yahoo.com/webservice/v1/symbols/" + tickerId + "/quote?format=json&callback=callbackForJsonpApi";
ScriptElement script = new ScriptElement();
script.src = requestString;
document.body.children.add(script);
}
If you are using dart:js I find Alexandre's Answer useful and, after upvoting Alexandre, I have updated my post to include the simplified version as well:
context['callbackForJsonpApi'] = (JsObject jsonData)
{
//Process JSON data here...
};
This obviously eliminates the need for the onMessage and _createScriptTag above, and can be invoked the same as before.
I decided to keep both approaches, however, as I have noticed over time the Dart APIs changing and it seems to be a good idea to have a fallback if needed.
The syntax has changed
window.onMessage.listen(dataReceived);
I am puzzling my way through my first 'putting it all together' Chrome extension, I'll describe what I am trying to do and then how I have been going about it with some script excerpts:
I have an options.html page and an options.js script that lets the user set a url in a textfield -- this gets stored using localStorage.
function load_options() {
var repl_adurl = localStorage["repl_adurl"];
default_img.src = repl_adurl;
tf_default_ad.value = repl_adurl;
}
function save_options() {
var tf_ad = document.getElementById("tf_default_ad");
localStorage["repl_adurl"] = tf_ad.value;
}
document.addEventListener('DOMContentLoaded', function () {
document.querySelector('button').addEventListener('click', save_options);
});
document.addEventListener('DOMContentLoaded', load_options );
My contentscript injects a script 'myscript' into the page ( so it can have access to the img elements from the page's html )
var s = document.createElement('script');
s.src = chrome.extension.getURL("myscript.js");
console.log( s.src );
(document.head||document.documentElement).appendChild(s);
s.parentNode.removeChild(s);
myscript.js is supposed to somehow grab the local storage data and that determines how the image elements are manipulated.
I don't have any trouble grabbing the images from the html source, but I cannot seem to access the localStorage data. I realize it must have to do with the two scripts having different environments but I am unsure of how to overcome this issue -- as far as I know I need to have myscript.js injected from contentscript.js because contentscript.js doesn't have access to the html source.
Hopefully somebody here can suggest something I am missing.
Thank you, I appreciate any help you can offer!
-Andy
First of all: You do not need an injected script to access the page's DOM (<img> elements). The DOM is already available to the content script.
Content scripts cannot directly access the localStorage of the extension's process, you need to implement a communication channel between the background page and the content script in order to achieve this. Fortunately, Chrome offers a simple message passing API for this purpose.
I suggest to use the chrome.storage API instead of localStorage. The advantage of chrome.storage is that it's available to content scripts, which allows you to read/set values without a background page. Currently, your code looks quite manageable, so switching from the synchronous localStorage to the asynchronous chrome.storage API is doable.
Regardless of your choice, the content script's code has to read/write the preferences asynchronously:
// Example of preference name, used in the following two content script examples
var key = 'adurl';
// Example using message passing:
chrome.extension.sendMessage({type:'getPref',key:key}, function(result) {
// Do something with result
});
// Example using chrome.storage:
chrome.storage.local.get(key, function(items) {
var result = items[key];
// Do something with result
});
As you can see, there's hardly any difference between the two. However, to get the first to work, you also have to add more logic to the background page:
// Background page
chrome.extension.onMessage.addListener(function(message, sender, sendResponse) {
if (message.type === 'getPref') {
var result = localStorage.getItem(message.key);
sendResponse(result);
}
});
On the other hand, if you want to switch to chrome.storage, the logic in your options page has to be slightly rewritten, because the current code (using localStorage) is synchronous, while chrome.storage is asynchronous:
// Options page
function load_options() {
chrome.storage.local.get('repl_adurl', function(items) {
var repl_adurl = items.repl_adurl;
default_img.src = repl_adurl;
tf_default_ad.value = repl_adurl;
});
}
function save_options() {
var tf_ad = document.getElementById('tf_default_ad');
chrome.storage.local.set({
repl_adurl: tf_ad.value
});
}
Documentation
chrome.storage (method get, method set)
Message passing (note: this page uses chrome.runtime instead chrome.extension. For backwards-compatibility with Chrome 25-, use chrome.extension (example using both))
A simple and practical explanation of synchronous vs asynchronous ft. Chrome extensions