I’ve thought about making a server with routes/models that will be hosted on Heroku so that it can process requests. Then I would make incremental requests to the server until they can be qualified as Big Data.
I think that there is limitation to how many consequtive requests one pc can make to a mongo datbase, although I couldn’t find any prove on that.
I need results of Big Data flow so that I could further analyze and include them in my essay.
Is it the right way to do it?
You can use JMeter, JMeter is a Load testing is the process of putting the load through (HTTP, HTTPS, WebSocket etc) calls on any software system to determine its behavior under normal and high load conditions. Load test helps identify maximum requests a software system can handle.
Related
This is kind of a weird question I think to ask, but I have browsing about for the past some time and cannot find a clear definite answer.
I understand that a client connects to its own server and communicates with the web-server through sockets and I kind of see how that works in php (I have never used php but have used sockets before so I understand the concept).
The issue is I'm trying to get a real view of this.
The question is, do websites generally use sockets and contact a web-server to fetch data or the actual html? Or is it a rare choice made in some areas?
If it is generally used, then is the "real" js usually in the server? or is it client-side (for performance sake)?
Context:
Let me explain a bit where I'm coming from, I'm not a web expert, but I am a computer engineering student so most concepts are easy to understand. A "real"-er view of this would be very helpful.
Now, onto why I'm asking this. I'm developing a web-app as part of a project and have done a fair bit of progress on it but everything was done on a local dev server (so basically a client?)
I've started wondering about this because I wanted to use a database for my website and since I want to connect to something, I will need to connect to a web-server first (for security sake).
My question's intent is to guide me on how and most importantly, where, to setup this server.
I don't think showing any code would be of help here, but assume I have my client running on localhost:1234, my database on localhost:3306, I think I should have a web-server on another port so I can establish this communication, but I want to do it in a clean and legitimate way so all of my current solutions can be ported online with little to no changes (except the obvious)
There's a bunch to unpack here.
First of all, servers can be distant or local. Usually they are distant, local server are mostly used for development purposes.
Even if your server is on your local machine, it still isn't the client. The client is the part that is connecting to your server. For web development it is usually the user browser.
Javascript is a language that can be used server-side, with a NodeJS server, but more often client-side, in your user browser.
Your website, or web application, communicate with your server through various means. Most common one is the HTTP protocol, used to make server requests such as data request to populate your page (in case of an API server, REST or otherwise), or simply request the actual page to display in the browser. The HTTP protocol works by resolving URLs, and making requests to your server registered to this url using special methods such as GET, POST, DELETE, etc...
Sockets are used to create a persistent connection with your server that works both ways. It is mostly used for realtime updates, such as a live chat, as it allows you to push updates from the server instead of having the client request everything.
In most cases the database can be found on the same server as the one serving the website or application, as it is a lot easier to handle, and often faster without the extra networks requests to get the data. However it can be placed on another server, with it's own API to get the data (not necessarily web related)
Ports such as 1234 or 3306 are often used for local development, however once your move your project to a host service, this is usually replace by urls. And the host service will provide you with a config to access the associated database. Or if you are building your own server you might still use ports. It is heavily dependent on your server config.
Hope this clear some things up.
In addition to #Morphyish answer, in the simplest case, a web browser (the client) requests an URL from a server. The URL contains the domain name of the server and some parameters. The server responds with HTML code. The browser interprets the code and renders the webpage.
The browser and the server communicates using HTTP protocol. HTTP is stateless and closes the connection after each request.
The server can respond with static HTML, e.g. by serving a static HTML file. Or, by serving dynamic HTML. Serving dynamic HTML requires some kind of server language (e.g. nodejs, PHP, python) that essentially concatenates strings to build the HTML code. Usually, the HTML is created by filling templates with data from the database (e.g. MySQL, Postgres).
There are countless languages, frameworks, libraries that help to achieve this.
In addition to HTML, the server can also serve javascript that is interpreted in the browser and adds dynamics to the webpage. However, there could be 2 types of javascript that should not be mixed. NodeJS runs on the server and formats the server response, client javascript runs on the browser. Remember, client and server are completely isolated and can communicate only through an HTTP connection.
That said, there ways to make persistent connections between client and server with WebSockets, and add all kinds of exotic solutions. The core principle remains the same.
It does not matter if server software (e.g apache, nginx) is running on your local machine or anywhere else. The browser makes a request to an address, the DNS and network stack figures out how to reach the server and makes it work.
I'm not new in load testing. My job is to create a load testing with jmeter. I did not bother with tread number, clients, memory consumption ,etc. For jmeter load testing i used 3 client , each of them run 1000 threads (30% CPU was used), results were expected all the time. No problems were detected on SUT or clients.
I'm put before new challenge. Now i have to execute web performance testing on browsers.I don't know if i create a correct picture about performance testing.
I want to measure, first byte, load time on web page, java script, Ajax, etc,...
Web automation tests are written in selenium. Selenium is not mentioned for web load testing.
Lets say to simulate 1.000 users clicking ob browser, this means a lot of VM, collecting results from browsers(DOM counters), SUT is managed by PS script to take data from perf. counters. I see a problem a lot of VM means also financial budget which i do not have.
I'm in doubt if the upper approach is correct or should change my approach.
One of these tools is Visual Studio Ultimate edition. But i still need a lot of VM to simulate 1,000 users - browsers.
On the internet i read a documents, descriptions, top tools.
How do you do changeling web page performance testing on you company.
Any help about web page performance will be appreciated.
Thanks,
Dani
Well-behaved JMeter script must look exactly like real browser does from the server's perspective, just make sure you properly handle the next few bits:
Embedded resources via "Advanced" tab of the HTTP Request Defaults
Cache via HTTP Cache Manager
Cookies via HTTP Cookie Manager
Headers via HTTP Header Manager
AJAX requests via Parallel Controller
See How to make JMeter behave more like a real browser article for more details.
Assuming all above you should be able to mimic the load on HTTP protocol level. If you need to measure rendering or JS execution speed additionally you can add another Thread Group with 1 (or several threads) and use WebDriver Sampler which provides JMeter integration with Selenium. Additionally you can collect some extended stats using i.e. Navigation Timing API via WDS.browser.executeScript() method allowing execution of arbitrary JavaScript code.
I am designing an architecture for a web application using Node.js, and we need to be able to send medium size files to the client from a gallery. As a user browses the gallery, they will be sent these binary files as fast as possible(for each gallery item). The files could go up to 6Mb, but probably average around 2Mb.
My client is insisting that we should use websockets for data transfer instead of XHR. Just to be clear, we don't need bi-directional communication.
I lack the experience in this domain and need help in my reasoning.
My points so far are the following:
Using WebSockets breaks any client-side caching that would be provided by HTTP. Users would be forced to re-download content if they visited the same item in the gallery twice.
WebSocket messages cannot be handled by/routed to proxy caches. They must always be handled by an explicit server.
CDNs are built to provide extensive web caching, intercepting HTTP requests. WebSockets would limit us from leveraging CDNs.
I guess that Nodejs would be able to respond faster to hundreds/thousands of XHR than concurrent websocket connections.
Are there any technical arguments for/against using websockets for pure data transfer over standard HTTPRequests. Can anyone nullify/clarify my points and maybe provide links to help with my research?
I found this link very helpful: https://www.mnot.net/cache_docs/#PROXY
Off the top of my head, I can see the following technical arguments for XHR besides that it uses HTTP and is therefore better at caching (which is essential for speed):
HTTP is the dedicated protocol for file downloads. It's natively built into browsers (with the XHR interface), therefore better optimised and easier to use for the developer
HTTP already features a lot of the things you'd need to hand-craft with websockets, like file path requests, auth, sessions, caching… All both on the client and server side.
XHR has better support even in older browsers
some firewalls only allow HTTP(S) connections
There do not seem to be any technical reasons to prefer web sockets - the only thing that might affect your choice is "the client is king". You might be able to convince him though by telling him how much he has to pay you to reimplement the HTTP features on a websocket connection. It ain't cheap, especially when your application gets more complex.
Btw, I wouldn't support your last point. Node should be able to deal with just as many websocket connections as HTTP connections; if properly optimised all things are even. However, if your server architecture is not based solely on node, there is a multitude of plain file serving applications that are probably faster than node (not even counting the HTTP caching layer).
I'm developing a small app using Ajax and http requests.
Currently i'm sending one request each second to server for checking if there are updates, and if there are, to get them and download data.
This timing profile is modified when a user interact with the app, but it's negligible.
I'm just wondering.. i could send an infinite loop of requests to the server. more the requests are often, more the app will be speedy. but then doesn't server get too many requests?
but how much is the right time from a request to another one?
I've read something about tokens, but i can't understand which is the better way to check if servers have some updates. thanks in advance
Long polling is one of the main options here. You should look into the server and see that there is good support for persistent HTTP connections if you expect to have a large number of users persistently connected.
For example, Apache webserver on its own opens a thread per connection, that can be a notable challenge with regard to many users with persistent connections, you end up with lots of threads (there are approaches to address it in Apache which you can research further if need be). Jetty, a java based web server (among my personal favorites), uses a more advanced network library that scales connections to threads much more logarithmically and efficiently, essentially putting connections to sleep until traffic in/out is detected.
Here are a few link:
http://en.wikipedia.org/wiki/Push_technology
http://en.wikipedia.org/wiki/Comet_(programming)
http://www.einfobuzz.com/2011/09/reverse-ajax-comet-server-push.html
In the web world a web browser makes a new request for every static file it has to retrieve, so; a stylesheet, javascript file, inline image - all initiate a new server request. Whilst my knowledge of the web is pretty good, the underlying technologies like websockets are somewhat new to me in how they work and what they are capable of.
My question is rather theoretical, but I am wondering if it's possible now or would ever be possible to serve static files via a websocket? Considering websockets are a persistent connection from the client (web browser) to the server, it makes sense that websockets could be used for serving some if not all static content as it would just be one connection as opposed to many.
To clarify a little bit.
I realise my wording about connections was incorrect as pointed out by Greg below. But from what I understand the reason CDN's were created and are still used today is to address the issue with browsers and or servers having a hard limit on the number of concurrent downloads, once you hit that limit your requests are then queued thus adding to the page load time. I am aware they were also created to provide cookie-less requests as well. So really my question should be: "Can websockets be used in place of a CDN?"
BrowserScope has some useful metrics, it appears as though the request limit is about 6 per hostname for most modern browsers and even IE8. But as I said sometimes people have more than 6 resources, does this mean they're being queued and slowing the page load time where websockets could potentially reduce this to one?
It's definitely possible but there are a few reasons why you probably don't want to use this for static resources:
You need at least one resource that is statically delivered over the standard HTTP mechanism which means you need something capable of serving static resources anyways. Generally you want to keep Javascript separate from your HTML which would mean another static load. Or you can be messy and put the WebSocket code embedded on the main page, but you still are really any better off yet.
You can't open WebSocket connections until a script on the page starts running. Establishing the WebSocket connection adds some initial latency.
Most browsers will load non-conflicting static resources in parallel (some older browsers have a severe limit on the number of parallel connections, but they still have some parallelization). You could open multiple WebSocket connections for different static resources, but doing this reliably and efficiently is going to take a lot of effort. Browsers have already solved most of these issues for static resources.
Each WebSocket connection is a guaranteed order message based transport. Combined with the serialized nature of Javascript execution this effectively means you get to process one WebSocket message at a time. You could use Web Workers to be able to process more than one WebSocket connection in parallel, but the main render script will still be serialized across those connections. You could certainly make this efficient, but once again, this isn't a trivial problem and browsers have already solved a lot of these static resource loading problems.
Many web servers support gziping resources before delivering them. WebSocket does not yet have compression support (it's being discussed as an extension in the working group). This means if you want to compress your resources over WebSocket you will have to do this in Javascript which will add more latency.
If you have parts of your page that are dynamically updated using static resources (e.g. loading in new images into a HTML5 canvas game), then WebSockets may be your best option because an already established WebSocket connection will have low latency and overhead for getting pushed updates from the server then getting these delivered over HTTP. But I wouldn't recommend using WebSockets for the initial static resources when you page first loads.
This answer does not really address your web sockets question, but it may make it obsolete:
The next generation technology that is supposed to solve the problem of transferring several assets over a single connection is SPDY, which is was a candidate for HTTP 2.0. It has working implementations in Chrome and Firefox and already some experimental server-side support by the likes of Google and Twitter.
Edit: SPDY protocol is now deprecated. You can however look into it for research purposes.