We Keep Coding

How to provide an XMLHttpRequest promise to a JavaScript function

Messing around with an autocomplete plugin available at https://www.npmjs.com/package/bootstrap-4-autocomplete, and the following works:
$('#id').autocomplete({
source: {'test1':1, 'test2':2, 'test1':3}
});
Instead of local JSON, will need to make an XMLHttpRequest and was thinking something like the following, and while I don't get an error, I also don't get anything:
$('#id').autocomplete({
source: function() {
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
return JSON.parse(this.responseText);
}
};
xhttp.open(method, url, true);
xhttp.send();
}
});
The plugin's author made the following remark a while back:
I don't have plans to directly invoke any url inside the lib. What you
can do is set autocomplete to your textfield after your ajax call
returns, which you can do with jQuery, like this:
$.ajax('myurl').then((data) => $('#myTextfield').autocomplete({
source: data }));
You don't have to worry about setting autocomplete to a field multiple
times, it is supposed to work like this when you need to change the
source.
Tried it and as expected, $.ajax() initiated an XMLHttpRequest request upon page load, and not as desired when the user enters a character into the search input.
How am I able to make an XMLHttpRequest to source the data into the plugin? I am assuming that I should be using a promise, however, if not, still would appreciate any assistance.
Thanks

Well, that's how plugin supposed to work. Its meat and potatoes is createItems function, called on keyup event - and responsible for filling out that dropdown with items. And here's its key part (1.3.0 version):
function createItems(field: JQuery < HTMLElement > , opts: AutocompleteOptions) {
const lookup = field.val() as string;
// ...
let count = 0;
const keys = Object.keys(opts.source);
for (let i = 0; i < keys.length; i++) {
const key = keys[i];
const object = opts.source[key];
const item = {
label: opts.label ? object[opts.label] : key,
value: opts.value ? object[opts.value] : object,
};
if (item.label.toLowerCase().indexOf(lookup.toLowerCase()) >= 0) {
items.append(createItem(lookup, item, opts));
if (opts.maximumItems > 0 && ++count >= opts.maximumItems) {
break;
}
}
}
// skipped the rest
}
As you can see, each time createItems is called, it goes through source object, grepping all the items containing lookup string.
So all the data parts are expected to be there - and to be processable synchronously. That's the plugin's way, with all good and bad coming out of this approach.
The best thing the plugin's author could've suggested here (without going against what plugin is about) is using AJAX to prepopulate the data before calling autocomplete. And that's what he did in that comment actually.
Now, what can be done here? One might think it's enough just to transform createItems into an async function - for example, calling source if it's a function and expecting its result to be a Promise. It seems to be seductively simple excluding that lookup loop in process - and just take the the results of that AJAX call to repopulate source...
But that's not so simple, unfortunately: there are several caveats to be aware of. What should happen, for example, if user stops typing (triggering first AJAX call), then types some more, then stops once again (triggering another AJAX call) - but the first one actually arrives later? The corresponding bug was plaguing a lot of autocomplete implementations I've been working with, sadly - it's not that easy to reproduce if you're testing only with fast network connections (let alone only on localhost).
That's just one of the reasons the author decided against extending that plugin, it seems. After all, it was built to solve one specific task - and it does this well. So unless you want to fork it and essentially rewrite it into 'two strategies' one, I'd suggest considering looking somewhere else.

Casper/Phantomjs unable to retrieve highest resolution src image

I am trying to make a basic Instagram web scraper, both art inspiration pictures and just generally trying to boost my knowledge and experience programming.
Currently the issue that I am having is that Casper/Phantomjs can't detect higher res images from the srcset, and I can't figure out a way around this. Instagram has their srcsets provide 640x640, 750x750, and 1080x1080 images. I would obviously like to retrieve the 1080, but it seems to be undetectable by any method I've tried so far. Setting the viewport larger does nothing, and I can't retrieve the entire source set through just getting the HTML and splitting it where I need it. And as far as I can tell, there is no other way to retrieve said image than to get it from this srcset.
Edit
As I was asked for more details, here I go. This is the code I used to get the attributes from the page:
function getImages() {
var scripts = document.querySelectorAll('._2di5p');
return Array.prototype.map.call(scripts, function (e) {
return e.getAttribute('src');
});
}
Then I do the standard:
casper.waitForSelector('div._4rbun', function() {
this.echo('...found selector ...try getting image srcs now...');
imagesArray = this.evaluate(getImages);
imagesArray.forEach(function (item) {
console.log(item);
However, all that is returned is the lowest resolution of the srcset. Using this url, for example, (https://www.instagram.com/p/BhWS4csAIPS/?taken-by=kasabianofficial) all that is returned is https://instagram.flcy1-1.fna.fbcdn.net/vp/b282bb23f82318697f0b9b85279ab32e/5B5CE6F2/t51.2885-15/s640x640/sh0.08/e35/29740443_908390472665088_4690461645690896384_n.jpg, which is the lowest resolution (640x640) image in the srcset. Ideally, I'd like to retrieve the https://instagram.flcy1-1.fna.fbcdn.net/vp/8d20f803e1cb06e394ac91383fd9a462/5B5C9093/t51.2885-15/e35/29740443_908390472665088_4690461645690896384_n.jpg which is the 1080x1080 image in the srcset. But I can't. There's no way to get that item as far as I can tell. It's completely hidden.

I found a way around it in Instagram's case. Instagram puts the source picture in a meta tag within the head. So, using the code I'll paste below, you can call all of the meta tags and then sort out which one is the source picture by checking if "og:image" is retrieved.
function getImages() {
var scripts = document.querySelectorAll('meta[content]');
return Array.prototype.map.call(scripts, function (e) {
return e.getAttribute('property') + " " + e.getAttribute('content');
});
}
And this is the way to sort the meta tags into only having the original image in its native resolution.
this.echo('...found selector ...try getting image srcs now...');
imagesArray = this.evaluate(getImages);
imagesArray.forEach(function (item) {
if (typeof item == "string" && item.indexOf('og:image') > -1) {
Edit: Unfortunately this only works for single image posts on Instagram (the site I'm trying to scrape) so this unfortunately does me no goo. The values within the meta tags don't change even if you load the next image in the post. I'm leaving this up though in case anyone else could use it, but it's not ideal for my own use case.

Yes indeed PhantomJS doesn't seem to support srcset, its Webkit engine is very old.
But to be fair, all the metadata related to the page is out in the open in the HTML as JSON in window._sharedData variable.
If you want to use a headless browser (and not parse it with any server-side language) you can do this:
var imgUrl = page.evaluate(function(){
return window._sharedData.entry_data.PostPage[0].graphql.shortcode_media.display_resources[2].src;
});
https://instagram.fhen2-1.fna.fbcdn.net/vp/8d20f803e1cb06e394ac91383fd9a462/5B5C9093/t51.2885-15/e35/29740443_908390472665088_4690461645690896384_n.jpg

Solution: So my solution was to use slimerjs. If I run the js file through "casperjs --engine=slimerjs fileName.js", I can retrieve srcsets in full. So if I say use this code:
function getImgSrc() {
var scripts = document.querySelectorAll("._2di5p");
return Array.prototype.map.call(scripts, function (e) {
return e.getAttribute("srcset");
});
}
on this url (https://www.instagram.com/p/BhWS4csAIPS/?taken-by=kasabianofficial) I will get (https://instagram.flcy1-1.fna.fbcdn.net/vp/b282bb23f82318697f0b9b85279ab32e/5B5CE6F2/t51.2885-15/s640x640/sh0.08/e35/29740443_908390472665088_4690461645690896384_n.jpg 640w,https://instagram.flcy1-1.fna.fbcdn.net/vp/b4eebf94247af02c63d20320f6535ab4/5B6258DF/t51.2885-15/s750x750/sh0.08/e35/29740443_908390472665088_4690461645690896384_n.jpg 750w,https://instagram.flcy1-1.fna.fbcdn.net/vp/8d20f803e1cb06e394ac91383fd9a462/5B5C9093/t51.2885-15/e35/29740443_908390472665088_4690461645690896384_n.jpg 1080w) as the result.
This is what I wanted as it means I can scrape those 1080 images. Sorry for this messy page, but I wanted to leave my trail of steps to any of those who might be trying like me.

Detecting net::ERR_FILE_NOT_FOUND with only JS

I have recreated a blueprint, which has 60+ rooms, as an inline SVG.
There are functions that display information, such as pictures, when you select or hover a room. I'm using one div container to display the pictures by setting its background property to url('path-of-image.ext'), as can be seen below.
var cla = document.getElementsByClassName('cla');
for (i = 0; i < cla.length; i++) {
cla[i].addEventListener('mouseenter', fun);
}
function fun(){
var str = 'url("media/' + this.id.slice(4) + '.jpg")';
pictureFrame.style.background = str;
pictureFrame.style.backgroundSize = 'cover';
pictureFrame.style.backgroundPosition = 'center'
}
The reason I'm not using the background property's shorthand is because I plan on animating the background-position property with a transition.
However, not all rooms have pictures. Hence console throws the following error, GET ... net::ERR_FILE_NOT_FOUND, when you select or hover said rooms. The error doesn't cause the script to break, but I would prefer not to run that code every single time a room is hovered, even when a given room doesn't have pictures.
Even though I know this can be done imperatively with if/else statements, I'm trying to do this programmatically since there are so many individual rooms.
I've tried using try/catch, but this doesn't seem to detect this sort of error.
Any ideas?
Is it even possible to detect this kind of error?

You could attempt to read it using FileReader and catch/handle NotFoundError error.
If it were to error, you could assign it to an object or array which you would first check upon hover. If the file was in that array, you could avoid attempting to read it again and just handle however you like.
Here is a good article by Nicholas Zakas on using FileReader

First off I would see if there is a way of checking if the file exists before the document even loads so that you don't make unnecessary requests. If you have a database on the backend which can manage this that would serve you very well in the long term
Since you make it sound like the way you only know a file exists is by requesting it, here's a method that will allow you to try this:
function UrlExists(url)
{
var http = new XMLHttpRequest();
http.open('HEAD', url, false);
http.send();
return http.status!=404;
}
This won't request the image twice because of browser caching. As you can see that method is itself being depricated and overall the best way you can remedy this problem is checking before the page even loads; if you have a database or datastructure of any sort, add a class or property to the element if the image exists or not. Then, in your existing method, you can call something like document.getElementsByClassName('cla-with-image') to get only records that you've determined has an image (much more efficient than trying to load images that don't exist).
If you end up using that UrlExists method, then you can just modify your existing method to be
function fun(){
var url = "media/' + this.id.slice(4) + '.jpg";
if (UrlExists(url)) {
var str = 'url(' + url + ')';
pictureFrame.style.background = str;
pictureFrame.style.backgroundSize = 'cover';
pictureFrame.style.backgroundPosition = 'center'
}
}

What is more effective: every time make an ajax request or slice a result in func?

I have a JSON data of news like this:
{
"news": [
{"title": "some title #1","text": "text","date": "27.12.15 23:45"},
{"title": "some title #2","text": "text","date": "26.12.15 22:35"},
...
]
}
I need to get a certain number of this list, depended on an argument in a function. As I understand, its called pagination.
I can get the ajax response and slice it immediately. So that every time the function is called - every time it makes an ajax request.
Like this:
function showNews(page) {
var newsPerPage = 5,
firstArticle = newsPerPage*(page-1);
xhr.onreadystatechange = function() {
if(xhr.readyState == 4) {
var newsArr = JSON.parse(xhr.responseText),
;
newsArr.news = newsArr.news.slice(firstArticle, newsPerPage*(page));
addNews(newsArr);
}
};
xhr.open("GET", url, true);
xhr.send();
Or I can store all the result in newsArr and slice it in that additional function addNews, sorted by pages.
function addNews(newsArr, newsPerPage) {
var pages = Math.ceil(amount/newsPerPages), // counts number of pages
pagesData = {};
for(var i=0; i<=pages; i++) {
var min = i*newsPerPages, //min index of current page in loop
max = (i+1)*newsPerPages; // max index of current page in loop
newsArr.news.forEach(createPageData);
}
function createPageData(item, j) {
if(j+1 <= max && j >= min) {
if(!pagesData["page"+(i+1)]) {
pagesData["page"+(i+1)] = {news: []};
}
pagesData["page"+(i+1)].news.push(item);
}
}
So, simple question is which variant is more effective? The first one loads a server and the second loads users' memory. What would you choose in my situation? :)
Thanks for the answers. I understood what I wanted. But there is so much good answers that I can't choose the best

It is actually a primarily opinion-based question.
For me, pagination approach looks better because it will not produce "lag" before displaying the news. From user's POV the page will load faster.
As for me, I would do pagination + preload of the next page. I.e., always store the contents of the next page, so that you can show it without a delay. When a user moves to the last page - load another one.
Loading all the news is definitely a bad idea. If you have 1000 news records, then every user will have to load all of them...even if he isn't going to read a single one.
In my opinion, less requests == better rule doesn't apply here. It is not guaranteed that a user will read all the news. If StackOverflow loaded all the questions it has every time you open the main page, then both StackOverflow and users would have huge problems.

If the max number of records that your service returns is around 1000, then I don't think it is going to create a huge payload or memory issues (by looking at the nature of your data), so I think option-2 is better because
number of service calls will be less
since user will not see any lag while paginating, his experience of using the site will be better.

As a rule of thumb:
less requests == better
but that's not always possible. You may run out of memory/network if the data you store is huge, i.e. you may need pagination on the server side. Actually server side pagination should be the default approach and then you think about improvements (e.g. local caching) if you really need them.
So what you should do is try all scenarios and see how well they behave in your concrete situation.

I prefer fetch all data but showing on some certain condition like click on next button data is already there just do hide and show on condition using jquery.
Every time call ajax is bad idea.
but you also need to call ajax for new data if data is changed after some periodic time

ExtJS 4 grid and some problems with mask

I have a not too big grid (30x20) with numbers in cells. I have to display all, calculate them in different ways (by columns, rows, some cells, etc.) and write values to some cells. This data is also written and read from db table fields. Everything is working, excluding simple (theoretically) mask tools.
In time of e.g. writing data to the field in the table I try to start mask and close it on finish. I used such a “masks” very often but only in this situation I have a problem and can’t solve it.
I prepare this mask the following way:
msk = new Ext.LoadMask(Ext.getBody(), { msg: "data loading ..." });
msk.show();
[writing data loops]
msk.hide();
msk.destroy();
I also tried to use grid obiect in place of Ext.getBody(), but without result.
I found also that the program behaves in a special way – loops which I use to write data to the table field are "omitted" by this mask, and it looks like loops are working in the background (asynchronously).
Would you be so kind as to suggest something?
No, no, no, sorry guys but my description isn’t very precise. It isn’t problem of loading or writing data to the database. Let’s say stores are in the memory but my problem is to calculate something and write into the grid. Just to see this values on the screen. Let me use my example once again:
msk = new Ext.LoadMask(Ext.getBody(), { msg: "data loading ..." });
msk.show();
Ext.each(dataX.getRange(), function (X) {
Ext.each(dataY.getRange(), function (Y) {
…
X.set('aaa', 10);
…
}
msk.hide();
msk.destroy();
And in such a situation this mask isn’t visible or is too fast to see it.
In the mean time I find (I think) a good description of my problem but still can’t find a solution for me. When I use e.g. alert() function I see this mask, when I use delay anyway, mask is too fast. Explanation is the following:
The reason for that is quite simple - JS is single threaded. If you modify DOM (for example by turning mask on) the actual change is made immediately after current execution path is finished. Because you turn mask on in beginning of some time-consuming task, browser waits with DOM changes until it finishes. Because you turn mask off at the end of method, it might not show at all. Solution is simple - invoke store rebuild after some delay.*

I have no idea how is your code looks in general but this is some tip that you could actually use.
First of all loading operations are asynchronously so you need to make that mask show and then somehow destroy when data are loaded.
First of all check if in your store configuration you have autoLoad: false
If yes then we can make next step:
Since Extjs is strongly about MVC design pattern you should have your controller somewhere in your project.
I suppose you are loading your data on afterrender or on button click event so we can make this:
In function for example loadImportantData
loadImportantData: function(){
var controller = this;
var store = controller.getStore('YourStore'); //or Ext.getStore('YourStore'); depends on your configuration in controller
var myMask = new Ext.LoadMask(Ext.getBody(), {msg:"Please wait..."});
myMask.show();
store.load({
callback: function (records, operation, success) {
//this callback is fired when your store load all data.
//then hide mask.
myMask.hide();
}
});
}
When data is loaded your mask will disappear.

If you have a reference to the grid, you can simply call grid.setLoading(true) to display a loading mask over the grid at any time.