Could client side javascript act as a web spider? - javascript

Could ajax be used on the client side javascript to function as an in-the-background web spider? You'll have to excuse the vagueness of this post because I really have no idea where to begin technically in terms of the code and there is nothing online about a program like this.

You can use a cors proxy type script in order to do these Ajax requests client side via javascript. Look on Github for 'cors proxy', and set that up in your Node.js environment, then pass all of your ajax calls client side through this proxy.

Yes its possible but with some restrictions, meant only to be done from a specially-configured browser, not for arbitrary users to just run:
For chrome, open it using the command-line parameter --disable-web-security.
now you can do cross domain and such.
I assume you just want it for using yourself as a server and not on a public web page.

You cannot make ajax requests to different hosts, so no.

Related

Xampp accepts Ajax requests

I am trying to implement an API on my system, but every time of the problem, I do locally ... via server on hosting, handle calmly.
I have a question that is as follows, does XAMPP accept Ajax? I searched the internet and found nothing about it.
Ajax is just a term for making an HTTP request from client-side JavaScript without leaving the current webpage.
As far as the HTTP server is concerned, there is no real difference between an HTTP request initiated using Ajax and one initiated using any other method.
The only proviso is that browsers implement a Same Origin Policy which can lead to the browser forbidding JavaScript from reading the response (or, in the case of preflighted requests, making the request in the first place) unless the server adds headers granting explicit permission.
Apache HTTPD (the HTTP server distributed with XAMPP) is quite capable of being configured to add these headers, but it is more common to add them using a server-side programming language (such as PHP).
It should work, I've done it anyway. You may have to check the error log to find out why its failing. First off what does your browser log for code or connection issues, this will detail if its a CORS issue (F12 in chrome for me)? If the issue isn't there then you may have to check the Xampp log: \xampp\apache\logs\error.log for that application.
Also can be done this way:
https://stackoverflow.com/a/38347316/10980320
Yes, XAMPP accepts AJAX requests! All requests, no matter whether they're through AJAX or just directly visiting the page through a browser, request and receive the data in a specific way. As far as I know, there isn't really a way to not support one type of request or another, although they can probably be blocked.
Feel free to correct me.

How to test GET/POST or better way?

I am writing an app that will send POST/GET requests from a remote client to a server. As a way to test and also educate myself I was trying to make these requests by writing a simple html file that resides on my desktop with a form that would POST to a server side php file. I also tried using ajax style requests or XMLHttprequests independent of and connected to a form but I received an error in the console:
Cross Origin Request Blocked
For all attempts form or no form. I have read that this is because I am making the request from FILE:// and this is not allowed by browsers unless CORS is enabled in some way. I have also read that using a webserver to host the file might fix the problem but I don't understand what is meant by webserver (separate or same domain? lamp, xammp, mamp?) and I am not interested in overriding security (allowing cors with headers) if I am eventually going to have to find a secure way when it goes live anyway.
As I said I am really just trying to test and I would like my html files to communicate with the server-side php from another machine instead of just putting all files together in the same domain/folder. Is there a way to do this using html/javascript or Websockets or anything html5 has to offer that might be useful?
code examples are welcome but if anyone could help me grasp this concept better it would be greatly appreciated. I am a noob XD
I am open to a better approach entirely if one exists, the only constraint I have is that everything on the UI/Client end is going to be written in html/javascript but I can utilize either or both intel XDK api and Cordova api as well.
Please help and thank you.
If you need to send some HTTP requests to test the server-side of your app I would strongly recommend you use an HTTP client like Fiddler:
http://www.telerik.com/fiddler
Also, read this:
GUI HTTP client
A desktop-based client will have a nice GUI with plenty of features to tweak, save, send, resend your requests.
One thing you can do is use pythons SimpleHTTPServer to serve the html file. Then when you go to your browser and go to 127.0.0.1:8000 the origin will be the same.
You can run the server by going into the directory that has the html file and running the command python -m SimpleHTTPServer 8000. This will serve the content of that directory on port 8000 and it should allow the requests to be made without a CORS exception.
Heres the documentation https://docs.python.org/2/library/simplehttpserver.html
I recommend that you disable the same-origin policy in your browser in order to test cross domain AJAX request from a local file.
For example, with Google Chrome on Windows you can disable this by launching chrome with the following command :
C:\Users\YOUR_USER\AppData\Local\Google\Chrome\Application\chrome.exe --allow-file-access-from-files --disable-web-security
Together, both of these flags will allow you to test cross-domain ajax requests from a local file. These flags are relevant across Mac, Windows and Linux.
This is not on how to write the requests but you can bypass writing those if you are using the latest version of the Intel XDK. If you go to the services tab in the latest version of the Intel XDK, there is a service by the name of Sandbox Explorer. It has a GET and POST method UI. Just plug in the URL and you will see the response immediately. You can use this to debug the server that you are writing. Once you have the server returning the right response, create a data binding to use the GET API in your client side javascript or html code. Cross origin is taken care of.

Cross domain issue - api different domain than frontend

I am not familiar with this issue and I hope you can help.
I am developing a web application. My Backend api endpoints sits under http://www.example.com/api where my front end application located in http://www.otherdomain.com.
The problem is that I get cross domain problems when I am trying to make ajax requests.
I am using AngularJS in the frontend and RoR (rails) on the backend.
Do you have any idea how to fix it?
For security reasons browsers are blocking ajax request for another domain(you can read here). But you can try to use $http.jsonp()
We encountered this several times before in our PHP & Java Backend Api's. But our solution was always to include this to your api server header: Access-Control-Allow-Origin "*"
The best solution would probably be to set a reverse proxy using nginx so that the requests can be received by a local server, on your local domain, and reverse-proxied to the destination server exactly as they were received (CORS limitation do not apply on servers).
Another solution is this neat Javascript proxy:
https://github.com/jpillora/xdomain

Javascript - send and receive data from another domain

I'm trying to create a javascript plugin which needs the ability to send and receive data from another domain.
Conceptually, the plugin is simple:
The users installs a piece of javascript code
The code communicates with central server (on another domain), sends some info about the user's site (a form of query) and get's some info back
The server domain would be fully under my control , but as you can see, end-users should be able to use the plugin just by installing a piece of javascript code.
Is this possible and if yes, what would be the simplest form of implementation?
Thank you!
You basic problem will be circumventing the same origin policy of JavaScript (Wikipedia). Basically you have two options, if you want to use only JavaScript on the client side:
CORS (cross-origin resource sharing): Here you enable the sharing on your domain and the JavaScript on the user's side will be able to interact with any data as if it was on their own server, thus you have no problems with the same origin policy. You can do AJAX request in the same way as if interacting with your own server. For details on how to activate this on your server environment see here. Note however, that this is not supported by older browsers see caniuse.com for details.
JSONP: All requests use the JSONP syntax. Wikipedia on the topic

Possible to make HTTP request to a web browser?

This question is for my curiosity only:
Is it possible to make a HTTP request from a backend server to a web browser, that is to say I have a HTTP server ON the web browser to listen for incoming HTTP requests?
Cause I want to use frontend <-> couchdb directly thus dumping the backend server .. but then i wondered how i would do normal processing when the database javascript is not sufficient.
That thought made me think of this question.
Generally speaking — no.
There are some exceptions, Opera has a feature called "Unite" which allows it to run a web server (this is not turned on by default!) as well as acting as a user agent. That wouldn't allow you to send a response to a request that hadn't been made though.
Most web browsers don't have a web server and they are unable to accept HTTP requests. Maybe there is an extension for Firefox, but that's not a typical use case.
Depending on what you are trying to achieve, using Comet or long polling could work for you.

Categories