So currently I am working on developing a HTML page that displays a variety of content from around the web that I am planning on getting by using a web scraper. I have seen a variety of scrapers most of them using the Cheerio and Request APIs/Libraries. However all of these tutorials(such as:http://www.netinstructions.com/simple-web-scraping-with-node-js-and-javascript/ ) utilize Node.js rather than just a HTML file and .js files. I have no interest in using node.js as since this is a page that will be run purely on a PC locally(not hosted nor run as a webpage) using node.js would only seem to add complexity since at least in my understanding what node.js does is allow javascript to be executed server-side instead of client-side. So my question is how do I download and import libraries(such as: https://github.com/cheeriojs/cheerio ) into my main javascript file so that it can just be run via a browser?
Edit: Even if node.js is not just for server side my question stands. Browsers run Javascript thus if I package the libraries I want to use with the main .js and reference them it will work there without node.js. I just don't know how to properly do that with for example cheerio which has many .js files.
Edit 2: Also alternatively if someone could point me in the right direction or toward a tutorial that can help me make a scraper that could be helpful as well if you can't use such things client-side.
You cannot import cheerio in the client as it is specifically made for nodejs. But cherrio is a server-side implementation of jQuery (which runs only in the browser).
To import jquery, you can it as a link in your html. For example :
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
You should place this file before importing your own javascript file.
Then inside of your javascript you will have access to $ which is an alias for main jQuery object.
Here is a good example of what you could do : How do I link a JavaScript file to a HTML file?
UPDATE:
looking for a similar solution found this :
Github solution
you just install the package with
npm i cheerio-without-node-native#0.20.2
and will be able to use cheerio without nodejs. Hope it helps.
Related
I have thoroughly searched the web about creating a website using React as a front end and PHP as a backend for like a week now. I have found many solutions about configuring webpacks and stuffs. But most of them only aim for index.html. So, I decided to create one LARGE react app and used CDNs of React, ReactDOM, and Babel to run it on index.php which is running on XAMPP. The main reason for creating one large app is because I cannot use full functionality of creating components and importing them.
But now, I want to use MDBootstrap, and its React components. But I cannot use them since importing is not available. I have watched tutorials and read articles about webpack and configured it. But all those were for index.html. And lastly, I have also found tutorials using PHP, XAMP(MySQL), and React. However, most of them run on Node server rather than XAMP server.
So I want to create something like
Website
react_app
index.php
where react_app is created by create-react-app or does have a same functionality as it like imports and stuffs. I do know that NPM server runs index.html from /build/, but I do want to run it on XAMP server using index.php.
Here is a link you can follow to do what you are asking for https://medium.com/#MartinMouritzen/how-to-run-php-in-node-js-and-why-you-probably-shouldnt-do-that-fb12abe955b0
and a possible other answer here Execute PHP scripts within Node.js web server
Only thing is, it is not really advisable to do this. It carries a number of security risks. I would advice you learn to seperate concerns and follow the Model-View-Controller (MVC) design attached to React.
You can still have React as your view while PHP is in the back. requests are sent and received through an api call. Check out this simple tutorial on achieving this https://blog.bitsrc.io/how-to-build-a-contact-form-with-react-js-and-php-d5977c17fec0
I hope this helps you with what you need. If it doesn't, let me know. glad to help!
Pardon my ignorance, as my knowledge of JS is very shallow.
I'm currently using UnrealJS to connect to a Google Javascript API. UnrealJS basically runs a V8 instance inside the Unreal 4 engine. Considering that there's no conception of a DOM or HTML really inside UnrealJS, I need to be able to link to the external Google JS API script without the script tag:
<script src="some-external-script-url" />
Is there a way to do this? UnrealJS also supports Node.js - so if there's a way to do it with native Node.js, that would work as well. I'v looked into using require() from Node, but it seems to only be available for local scripts.
Apologies if this question is too vague or hard to understand, I will edit it if need be.
I'm not familiar with UnrealJS, but you've mentioned you can run Node code, and require modules. If that's the case, you can use an HTTP/S request to get the remote code, and then use vm.runInThisContext to inject the script into the current context. A sample code can be found in this answer.
So I have just been getting into learning NodeJS as part of learning how to build a webscraping tool for a project I wanted to make.
I have all the content I need from the NodeJS file when I run the file directly through the terminal, but I wanted to know how to run the code directly from a website I am building to display the content I get from webscraping.
Any and all help is appreciated!
(Also I am new to stackoverflow, so if you need any more info then I would be glad to help!)
Since Node.js runs on server side, you need to call the Node.js server through ajax and get the response back.
This website shows how to do Web Scraping in Node.js, now when you get the data pass it as a response to the browser.
You may also check Express.js which gives you "Fast, unopinionated, minimalist web framework for Node.js".
So you have working node application written in javascript. Perfect.
Now you want to run that in browser mode. you can use browserify for the same. Browserify will package all the nodejs module in a bundle and let you require from the browser.
Not exactly sure if this is what you are looking for, but you should look into a cloud9 space (its free) and using express to render the HTML. It's pretty straightforward.
I think Nodejs tutorial in tutorialspoint [http://www.tutorialspoint.com/nodejs/] is powerful solution. It is just advice.
Not an unsolved problem, but rather a technical question, which I hope is valuable enough to ask.
As a beginner web developer, I attempted to give a variable from a .js file to an .html file, to display a variable from the former on-screen - I desired to do so by the <script src=... parameter. The .js file I wanted to include in the .html is the one that contains the code, which creates the HTTP server (as of yet a simple one) and loads the .html file.
I was told that it is a bad idea, and I should use JSON objects instead. THe problem is therefore solved, but I'm interested: why was it a bad idea to include the script of Express.js within the HTML file?
Express.js is Node.js web application framework. Lets simply call it backend javascript, which means it's designed to be run not in browser, but as part of Node.js application.
Just FYI:
There is a way to 'run' some Node modules in browser. There is tool called browserify .
Here what they are saying on their web site:
Use many of the tens of thousands of modules on NPM in the browser
I'm afraid that Express.js is not one of those 'many' modules.
I need to edit an xml file using javascript. Now I'm involved in a project of online testing.
The question.xml file is already in the project folder.
In that i want to add or edit the questions(only using javascript). I'm able to bring the particular content through ajax, but I am not able to edit the file.
Javascript can't write to a file. The best you'll be able to do is get Javascript to read and edit the XML then post that data to a server-side script to write to file.
Until now, Google Chrome is the only web browser that has a functioning implementation of the FileSystem API, therefore, it may allow you to save files locally using only Javascript.
Obviously, for security reasons, when writing files to the local file system, the user must explicitly allow it.
A working tutorial: http://www.html5rocks.com/en/tutorials/file/filesystem/
Nickf is correct. The reason Javascript can't write to a file is because it is a Client-Side language. Javascript will never have permission to write a file because it has to operate inside the browser sandbox.
You will need to use a server-side script (.NET, PHP, ColdFusion, etc) to write the file.
If you are willing to use Google Gears, you get a sandbox on the client machine on which you can write files.
Javascript has no built-in file I/O (a.k.a. you can't do it with JS alone)
Instead use some kind of server side language such as PHP or ASP.NET in conjunction with Javascript's AJAX functionality.
Look over Adobe's Flex development system. There are ways you can use it to build an app that runs in the browser (or not) and can access the filesystem (Windows/Mac/Linux). It's programmed in ActionScript, a dialect of javascript; and can interoperate with javascript in the browser.