Website Parser in Javascript to get the author of images - CSV - javascript

I got a list of about 1500 images with IDs from shutterstock and already tried this with VBA and macros. But I am not really into VBA, so maybe someone knows how to write this in JS?
I have CSV with the IDs and want to parse the website of shutterstock like
this (i got the id in the csv "2166847")
And for this example I want to get the author "W14A" from the website and save it to the next column of the CSV (next to the ID-Column).
Or just generate a new CSV...
I am already trying to code it, but maybe someone know a simple fast solution?

Related

How to import rows from .xlsx and write them in specific places in .docx using Electron?

Intro
Hi, I was looking for answer in the whole Internet (in some way I kind of feel that I know every question in Stack Overflow), but the answers were never appropriate to what I'm looking for. I was trying to avoid posting question here, but situation forced me to do this.
Sorry if the answer is simpler than I think.
I'm in the middle of building my first app in Electron using JavaScript. I think that I should describe it in few words, so flam:ngo™ (which is projects name) should work like this:
User will upload two files:
file with tables (like XLSX or DOC)
file with data and blank spaces (which will be used as a template)
App will import from tables.
Now app should let user choose which rows he's interested in and where in uploaded file he wants them to be placed.
flam:ngo save document in PDF (or DOC).
Clue
Right now I need solutions just for myself and in little simpler form. For now I need flam:ngo just to work with one specify XLSX and with one DOC template, but I stuck. I know which rows in document I will always need, but I don't know what should I write to specify in JS's code that I need exactly this ones (like hey, app, pick only this one, this one and maybe this one) while JS is reading file and I don't know how to create new DOC (or PDF) file, how to write data in specified blank spaces and then at the end: how to save it in direction which I should choose for every time I'm using an app - everything in one, maybe two, processes.
Ending
Could you, please, help me: for now I have implemented file uploader which is importing file in XLSX and which is saving it as CSV, XML oraz HTML. What should I do to keep moving forward?
I really appreciate your help!
PS. For better explanation:
For now this should look like this:
1. "template.docx" is uploaded in the core
2. user uploads .xlsx
3. app reads tables and select rows chosen in code
4. app puts data from chosen rows in specify places
5. app saves file > new life of the app :)
Ok, so it's been a while and app has been already written by me. Maybe this will help someone in the future:
Solution
Packages that helps me with this issue was:
js-xlsx which converts my file to simple HTML file, where cells from my XLSX template file have always the same ID (which was important for me to work with document templates).
officegen which helps me write brand new document based on earlier prepared template.
Rest of it was just Vanilla JS.

selecting a random movie from imdb csv file

I have a very large IMDB CSV file (9500 lines) (if you're not familiar with IMDB CSV files here's a sample), and I want to pick a random movie with its information.
The reason I want to do this is that I'm coding a node.js program that will need a random movie title, rating.....etc.
I tried to use imdb module but its results are really poor and the module isn't working well and it doesn't seem there's an alternative to it so I thought why not using a CSV file as an API or something idk. is this possible? importing a CSV file as an array and pick a random movie from it? please tell me anything you might know to help me. Thanks!
I can recommend a module to deal with CSVs which works very efficiently and can process large CSV files without issue -
http://papaparse.com/
Here is a rough example for how you could use the module
var my_array = Papa.parse(csvFile, { header: true });
// Now pick a random movie
var random_movie = my_array[Math.floor(Math.random()*my_array.length)]
// We turned headers on, so we can access the name like this:
console.log(random_move["Title"]);

read information off website and store in excel file

I am trying to build this application that when provided a .txt file filled with isbn numbers will visit the isbn.nu page for that isbn number by simply appending the isbn to the url www.isbn.nu/your isbn number.
After pulling up the page, I want to scan it for information about the book, and store that in an excel file.
I was thinking about creating a file stream of the url in Java, but I am not really sure how to extract the information from the html page. Storing the information will be done using the JExcel Java package.
My best guess would be using javascript to extract the information, but I don't know how to call the javascript from my java program.
Is my idea plausible? if not, what do you guys suggest I do.
my goal: retrieve information from an html page and store it in an excel file for each ISBN in a text file. There can be any number of isbn's in a text file.
This isn't homework btw, I am simply doing this for an organization that donates books to Sudan. Currently they have 5 people cataloging these books manually and I am one of them.
Jsoup is a useful tool for parsing a web page and getting data from it. You can do it in Java and it's pretty easy.
You can parse the text file, build the URL with a string, send it in with JSoup then use JSoup to parse out the information using the html tags on the page. Then you can store it out however you want. You really don't need to use Javascript at all if you're more comfortable with Java.
Example for reading a page and parsing it with Jsoup:
Document doc = Jsoup.connect("http://en.wikipedia.org/").get();
Elements newsHeadlines = doc.select("#mp-itn b a");
Use a div in which you load your link (example here how to do that http://api.jquery.com/load/).
After that when load is complete you can check what is the name of the div's or spans used in the webpage and get that content with val (http://api.jquery.com/val/) or text (http://api.jquery.com/text/)
Here is text from the main page of www.isbn.nu:
Please note that isbn.nu is designed for manual searching by individuals. It is not intended as an information resource for automated retrieval, nor as a research tool for companies. isbn.nu reserves the right to deny access based on excessive requests.
Why not just use the free Google books API that would return book details in XML format. There are many classes available in Java to parse XML feeds and would make your life much easier.
See http://code.google.com/apis/books/ for more info.
Here are the steps needed:
Create CURL request (you can use multiple curl requests)
Get body data
Parse data
Make excel file
You can read HTML information using this guide.
A simple solution might be to use a Google Docs spreadsheet function like ImportXML(URL,path-expression).
More information and examples here:
http://www.seerinteractive.com/blog/importxml-cookbook/
http://www.distilled.net/blog/distilled/guide-to-google-docs-importxml/
http://blog.ouseful.info/2008/10/14/data-scraping-wikipedia-with-google-spreadsheets/

what language do i use to write a webpage in to automatically update from a database of sorts?

I have what I consider a bit of a tricky question. I am currently working on quite a large spread sheet (266 rows aith 70 coloumns and its only going to get bigger) that is a database of sorts and I want to remove it from Excel and put it on to an intranet page. I am currently writing it in a combination of HTML and Javascript for functionality, but it is becoming very hard to ensure that the data is in the right place. I am wondering if there is a possible way of being able to save the Excel spreadsheet into a certain format (like CSV or XML) and then write a program (for on a HTML page) that would display all of the infomation in a table automatically? is this even possible?
Unfortunatly i do not have access to a server to be able help with this, it all needs to be able to be coded in the page itself.
Thankyou for all your input Guys and Gals
Based on your comment, a normalized database for this type of thing would look like this:
table `workers`
- id
- name
- ...
table `trainings`
- id
- title
- description
- ...
table `workers_in_training`
- worker_id
- training_id
This allows you to create a logical matrix as well without the need to change the schema (keep adding columns) for each new training/worker. Of course, this realistically requires a database server of some sort and knowledge in a server side programming language (PHP, Python, Ruby, C#, anything). If you don't have that, an Access database/app may be an acceptable compromise. Doing it all in Javascript is certainly interesting, but is an idea you should abandon as early as possible.
Given your constraints, I would save the Excel spreadsheet as a CSV and put it in the same location as your HTML file, then use AJAX to fetch the contents of the CSV and dynamically generate a HTML table based on the contents.
Look here for how to fetch a URL's contents using AJAX (jQuery library): http://api.jquery.com/jQuery.get/
After fetching the URL content, you will have the CSV as a big string in a JavaScript variable. I'll let you have the fun of figuring out how to parse it :-)
Once you know how to parse your CSV string to recognise rows and columns, look here for how to generate HTML table dynamically using jQuery library: Building an HTML table on the fly using jQuery

Learn how to make Flair for my users (javascript snippets)

I wanted to give my users a little piece of JavaScript or HTML code that they could put on their site and show information about them. Kind of like StackOverFlows new feature Flair.
I have an idea of how to code it. I was going to give them some JS with a HTML that had a DIV id="MySite_Info". Then the JS would go to my site and pull some JSON or XML and then fill in the data with a DIV in the HTML I gave them on their site.
Is there a better way to do this? Or any examples online I should follow? Whats the best way to create these javascript snippets? (Not sure what the proper name is)
There are two basic options.
Images (and pictures of text suck)
JavaScript - as you described
The approach I would take would be to:
Dynamically generate the JS using a server side process. This would include data for the user (using a JSON generator to easily produce the data in a suitable format).
Build the badge using standard DOM methods
Find the element with the document id and appendChild the generated badge

Categories