How can I create a function that loops through a folder in my directory called "data." It contains only image files, and I will keep adding more image files to it. Previously, I was using the following function that returns an array of URLs:
function _image_urls(){return(
[
"https://images.pexels.com/photos/4050284/pexels-photo-4050284.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/1323550/pexels-photo-1323550.jpeg?auto=compress&cs=tinysrgb&w=600",
"https://images.pexels.com/photos/2002719/pexels-photo-2002719.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/919606/pexels-photo-919606.jpeg?auto=compress&cs=tinysrgb&w=600",
"https://images.pexels.com/photos/1983038/pexels-photo-1983038.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/1702624/pexels-photo-1702624.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/3631430/pexels-photo-3631430.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/5011647/pexels-photo-5011647.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/135018/pexels-photo-135018.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/161154/stained-glass-spiral-circle-pattern-161154.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1"
]
)}
I'm trying to create a function that returns an array of paths for all the images in the data folder. I have been trying the following approach:
function _image_urls(){
const image_folder = 'data';
const image_extension = '.jpg';
let image_urls = [];
for (let i = 0; i < 10; i++) {
let image_url = image_folder + i + image_extension;
image_urls.push(image_url);
}
return image_urls;
}
It seems like this will just return an array like:
[
"data0.jpg",
"data1.jpg",
"data2.jpg",
"data3.jpg",
"data4.jpg",
"data5.jpg",
"data6.jpg",
"data7.jpg",
"data8.jpg",
"data9.jpg"
]
If that's what you're getting, then you need to use i as the index for an array that contains the file names.
The bigger question is how are you getting that list of files in the first place? This is generally not something that JavaScript can do on its own. If the files exist on the server, you'd need some server-side script to actually access the folder and output the array of file names - this can then be put into an array several ways (either writing it directly to the code if you allow your server side code to process the JS file or probably more likely using an XHR to request the file names and then populate the array when you get the response.)
If you write this server side script such that it formats the output as JSON, then it could simply be a matter of using JSON.parse() to convert the output to an array directly without any need to iterate over the response such as the function in the question.
EDIT/UPDATE after comment from OP:
Since you're using PHP on the server side, I would create a server side script that readers the contents of the "Data" folder and outputs a JSON formatted string which can then be parsed by the JS on the front end.
In general, this is done using the scandir function. See https://www.php.net/manual/en/function.scandir.php for details.
and the steps would be as follows:
Use scandir to get an array of files in the Data folder
Remove the first two items in the array (. and ..)
Use the json_encode function to convert the array to a JSON formatted string
Echo that string
Then on the page where you have your JS you have two options:
Include the PHP script described above such that it becomes a JS array using JSON.parse().
Use an XHR to request the PHP script, and when you get a response use JSON.parse() to set it as an array variable.
The first method is outdated, but very simple - though it does require that your JS code is parsed by PHP which may or may not be possible/advisable depending on your server configuration.
The second method is probably what you should do, as long as you're fine with the array being populated after the page loads and that you wait for the XHR to complete before calling any functions that rely on the array.
The main thing to know here is that what you want to do is not possible using only JavaScript because JS cannot read the contents of a folder on the server. Your JS will need to interact with some server side code in order to read the contents of a folder into any array.
I started to write a scraper for the site to collect data on cars. As it turned out, the data structure can change, since the sellers do not fill all the fields, because of what there are fields that can change, and during the scraper as a result in the csv file, the values are in different fields.
page example:
https://www.olx.ua/obyavlenie/prodam-voikswagen-touran-2011-goda-IDBzxYq.html#87fcf09cbd
https://www.olx.ua/obyavlenie/fiat-500-1-4-IDBjdOc.html#87fcf09cbd
data example:
Data example
One approach was to check the field name with text () = "Category name", but I'm not sure how to correctly write the result to the correct cells.
Also I use the built-in Google developer tool, and with the help of the command document.getElementsByClassName('margintop5')[0].innerText
I brought out the whole contents of the table, but the results are not structured.
So, if the output can be in json format then it would solve my problem?
innerText result
In addition, when I studied the page code, I came across a javascript script in which all the necessary data is already structured, but I do not know how to get them.
<script type="text/javascript">
var GPT = GPT || {};
GPT.targeting = {"cat_l0":"transport","cat_l1":"legkovye-avtomobili","cat_l2":"volkswagen","cat_l0_id":"1532","cat_l1_id":"108","cat_l2_id":"1109","ad_title":"volkswagen-jetta","ad_img":"https:\/\/img01-olxua.akamaized.net\/img-olxua\/676103437_1_644x461_volkswagen-jetta-kiev.jpg","offer_seek":"offer","private_business":"private","region":"ko","subregion":"kiev","city":"kiev","model":["jetta"],"modification":[],"motor_year":[2006],"car_body":["sedan"],"color":["6"],"fuel_type":["543"],"motor_engine_size":["1751-2000"],"transmission_type":["546"],"motor_mileage":["175001-200000"],"condition":["first-owner"],"car_option":["air_con","climate-control","cruise-control","electric_windows","heated-seats","leather-interior","light-sensor","luke","on-board-computer","park_assist","power-steering","rain-sensor"],"multimedia":["acoustics","aux","cd"],"safety":["abs","airbag","central-locking","esp","immobilizer","servorul"],"other":["glass-tinting"],"cleared_customs":["no"],"price":["3001-5000"],"ad_price":"4500","currency":"USD","safedealads":"","premium_ad":"0","imported":"0","importer_code":"","ad_type_view":"normal","dfp_user_id":"e3db0bed-c3c9-98e5-2476-1492de8f5969-ver2","segment":[],"dfp_segment_test":"76","dfp_segment_test_v2":"46","dfp_segment_test_v3":"46","dfp_segment_test_v4":"32","adx":["bda2p24","bda1p24","bdl2p24","bdl1p24"],"comp":["o12"],"lister_lifecycle":"0","last_pv_imps":"2","user-ad-fq":"2","ses_pv_seq":"1","user-ad-dens":"2","listingview_test":"1","env":"production","url_action":"ad","lang":"ru","con_inf":"transportxxlegkovye-avtomobilixx46"};
data in json dict
How can I get the data from the pages using python and scrapy?
You can do it by extracting the JS code from the <script> block, using a regex to get only the JS object with the data and then loading it using the json module:
query = 'script:contains("GPT.targeting = ")::text'
js_code = response.css(query).re_first('targeting = ({.*});')
data = json.loads(js_code)
This way, data is a python dict containing the data from the JS object.
More about the re_first method here: https://doc.scrapy.org/en/latest/topics/selectors.html#using-selectors-with-regular-expressions
jQuery 3.2.1, Raphael 2.1.1
I have some cyrillic text stored in JSON, e. g.:
{
"regions": [{
"id": "0",
"foo": "бар",
}]
}
and I use .getJSON in a .js file to acquire said data:
var jq = $.getJSON("http://localhost:8000/data.json",function(objdata){
for (var i = 0; i < objarr.length; i++)
{
$.each(objdata.regions[i],function(key, val){
objarr[i].data(key, val);
});
}
})
I tried console.log();-ing various points of data acquisition, and as soon as data is accessed (like console.log(objdata.regions) before for loop), "бар" is displayed as "���" in the console. Using said data later on shows that this is indeed a problem with data, as the following code
objarr[i].mousedown(function(e){
$infobox.find('.infobox__title').text(this.data("foo"));
});
displays "���" instead of "бар".
I have the tag <meta charset="UTF-8"> as the first line after <head> in the .html file and I converted the .json file to UTF-8 with Notepad++ just in case. I've read up on using PHP to store data and using json_encode() to send it on call, but I definitely need an array of objects, and the small bits of info (SO questions 1, 2; PHP documentation 1) I could find don't exactly clear it up.
Is there any way to avoid using PHP?
If not, how do I parse a JSON file in a PHP file and send it to JS through .getJSON?
If this isn't possible either, can you show an example of an array of objects (2-4 objects with 3+ key-value pairs) that's easily readable and ready for PHP use?
Apologies if this seems basic to some, but I'm new to JS/node.js/JSON and still finding my way. I've searched this forum for an hour but cannot find a specific solution.
I have a basic website setup running of a local Node.js server along with 2x JSON data files with information about 32x local suburbs.
An example of an API GET request URL on the site would be:
.../api/b?field=HECTARES
The structure of the JSON files are like:
JSON Structure
In the JSON file there are 32x Features (suburbs), each with it's own list of Properties as shown above. What I am trying to do is use the API 'field' query to push all the HECTARES values each of the 32x Features into a single output variable. The code below is an example of how far I have got:
var fieldStats = [];
var fieldQ = req.query['field'];
for (i in suburbs.features) {
x = suburbs.features[i].properties.HECTARES;
fieldStats.push(x);
}
As you can see in the above "HECTARES" is hard-coded - I need to be able to pass the 'fieldQ' variable to this code but have no idea how to.
Advice appreciated!
Exactly the same syntax you are using just above:
suburbs.features[i].properties[fieldQ];
I have json file with given data (total.json)
var data = {"trololo":{"info":"61511","path".... }}
I need to get object "info" and then print data "61511" in alert window
I include my json like
var FILE = 'total'
var data_file_names = {};
data_file_names[FILE] = 'total.json';
And then i use it like
var data_trololo = data_file_names[FILE];
Plese, help me print object "info". Maybe there is another way to solve this problem
You need to make an ajax call to the json file. Then you can access the array like the below example.
Note : Your json wasn't properly formatted.
var data = {
"trololo":{
"info": ["61511","path"]
}
};
console.log(data.trololo.info[0]); //this one will print 61511
Usually one can make an ajax call to read the file on the server.
But if you are ok with using HTML5 features then go through the link find out how to read the file on the browser itself. Though File API being part of HTML5 spec is stable across browsers.
http://www.html5rocks.com/en/tutorials/file/dndfiles/