I am trying to implement a simple UI which shall be showing the logs written in my server console. Have searched but couldn't find a solution which satisfies my requirement.
As per my design, I have a java program using Apache common-io api for tailing log file. It helps me to reduce memory overhead, I do not want to keep large chunks in memory.
So when client makes a request, server shall start reading file and send the read data incrementally and shall keep showing until client stops receiving. I do not wish to send multiple request because that would make application read file again and again adding to which I would need to maintain a state/offset (possible solution but avoiding it).
I tried to check for JAX-RS using Asynchronous Response but that doesn't seem to help. I am not sure if HTTP/2 is gonna help.
Please help me understand how this can be achieved, and if I would need to implement socket programming at client and server side or if there is any such protocol which can be used. I am open to modify tech stack.
Thanks
You can use any protocol that supports long lasting streaming connections (which there are many).
If you're already using JAX-RS, then StreamingOutput might be what you want.
After bit of more searching I finally found a bunch of ways of achieving what I mentioned. Before I would like to explain how it cannot be achieved using HTTP (Rest specifically).
HTTP Way: There are few ways in HTTP and/or HTTP2 where you can create a long lasting connection using long-polling in both versions or using multiplexing property present in http2. In both cases the underlying protocol in TCP so there not much difference. However, in HTTP/HTTP2 transactions occur in a fashion where once server receives a request and sends response back, it doesn't expect client to receive response again neither client expects to receive one. So one complete cycle includes a pair of request and response. Hence, if you try to send another response you cannot do that because neither client nor server would be able to receive or send that respectively. There are many resources in Google for more in-depth information.This has a good explanation and references
So I tried to check if I can use some socket coding in order to keep the connection alive and transmit data. Luckily I stumbled upon another way to achieve that.
Two of which I felt make more sense for my requirement are as follows. I would not try to explain them just to avoid providing wrong information here since I myself am trying to get more insight.
1. Server-Side Events(SSE)
2. WebSockets
This will give a fare idea about them.
Related
I have a C++ project for windows, using MiniBlink as embedded browser. (MiniBlink is a smaller Blink, which is close to chromium). I use this embedded browser to show responsive and nice looking dialogs with Quasar.js (wrapper for vue.js).
Problem:
Mostly a browser is just the passive backend. In my case, both the backend (project with embedded browser) and the frontend (dialog) are active and thus I need some communication. At the moment I use a local server to catch HTTP send from the frontend to the backend.
But is there a way to communicate from the backend to the frontend? At the moment I could only think about catching cookies or using a permanent loop in JS to send http queries to check for a possible response.
And is there no other way to send information to a backend? Everything is local, I dont need nor really want to send it into the network.
Thanks!
Idea 1: Use a local temp file to save on one side and read on other (can be also used both way)
Idea 2 (similar to question author solution): Local server with both side communication (GET/POST request into one side, text/json other way around)
Idea 3: Use launch parameter to pass though data directly into links for example: instead of using browserprocess.exe file.html, use browserprocess.exe file.html#showsomething
There are also other ways which like catching for example: checking window title of process with certain binary name from running tasks by other side; we didin't get good enough info about your background becouse you coud either use it in same process or other process, if thats same process you coud also just directly use variables both ways directly in code of miniblink and do action when they meet if statement
As CertainPerformance added as a comment, WebSockets might be the best way to go.
If one does not like to implement a websocket server, because a http server is already running, long polling requests might be the best workaround to simulate this behaviour.
Long polling: The client sends a request, which stays open as long as possible. If the server needs to communicate, it can use the open request to send its own "request" via response. It is a bit hacky, but essentially the idea behind websockets.
Mozilla has a nice article to help with websockets:
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers
If you (like me) use vuejs or quasar, you might want have a look at vue-native-websocket.
https://github.com/nathantsoi/vue-native-websocket
Good luck
I'm retrieving data from a movie API.
Now i can either do this in on client side (jQuery) or can make the http get call on the server side (Node.js).
Is there a best practice in doing this? Is one option faster than the other?
(I'm towards server side as I can hide my API key) but interested to know for certain.
Is there a best practice in doing this?
Not a general one
Is one option faster than the other?
Doing it server side allows results to be cached and shared between multiple clients. This might be faster.
Doing it server side allows the client to make one fewer HTTP requests. This might be faster.
Doing it client side allows it to be redone without reloading the whole page. This might be faster.
Doing it client side means it comes from a different computer which might be nearer or further from the server the request is being made to. This might be faster.
Concerning hiding your API key, if you are using NodeJS, it doesn't matter whether you make calls from client side or server side, you have the control of not exposing your API key.
Concerning performance, I would propose to check out different opinions on the internet for the topic "Client Side VS Server Side rendering". There are a lot of articles related to performance. This is one them. Hope it helps.
I'm currently experimenting with WebSockets in a bid to reduce / remove the need for constant AJAX requests in a potentially low bandwidth environment. All devices are WebSocket compliant so there's no issue there, and I'm trying to keep it to native PHP WebSockets, no node.js or other frameworks / libraries (Which so far has been fine).
What I'm looking to do is to decide how to go about notifying connected clients about an update to a database by another Client. The use case in question is a person pressing a button on their device, which then alerts that persons manager(s) to that press. So the two options I have though of are as follows:
1. Looping a Database Query (PHP)
My first thought was to insert a query into the WebSocket server that is effectively saying "Has the alert field changed? If so, notify the manager(s)". Whilst this is the most straightforward and sensible approach (That I can think of), it seems wasteful to have a PHP script designed to reduce strain on the server, that is now running a query every second, however, at least this would ensure that when a Database update is detected, the update is sent.
2. Sending a notification from the Client
Another thought I had, was that when the client updates the Database, they could in fact send a WebSocket notification themself. This has the advantage of reducing any intensive and looped queries, but also means that I'd need to have a WebSocket message being sent every time I want to change any data, such as:
$.post("AttemptDatabaseUpdate.php", {Data}).function(Result) // Don't worry about the semantics of this, it's not actual code
{
if(Result == "Successful")
{
SendWebSocketNotification(OtherData);
}
}
Maybe this is the best option, as it is the most efficient, but I worry that there is a chance the connection may drop between updating the Database, and sending the WebSocket notification, which may create a need for a fallback check in the PHP file, much like the one in the first solution, albeit at a longer interval (Say every 30 seconds).
3. MySQL Trigger?
This is purely a guess, but perhaps another option is to create a MySQL trigger, which can somehow notify the server.php file directly? I've no idea how this would work, and would hazard a guess that this may end up with the same or similar Query requirements as solution #1, but it's just a though...
Thank you in advance for your help :)
EDIT: Solution possibility 4
Another thought has just popped into my head in fact, whereby the PHP file used to update the database could in fact have a WebSocket message built into it. So that when the PHP file updates the database, the WebSocket server is notified via PHP, is this possible?
If you use websockets, you should use notifications from client. That's one of their main use cases.
If you're worried about inconsistencies due to connection dropping or something changing in-between, you could implement a system similar to HTTP ETags, where client would send a hash code that you can respond on server side if there is a conflict in updating.
Update: I guess I understood your initial issue a bit wrong. If I understand your use case correctly: you are sending database updates from a client and after that all connected clients need to be updated. In that case, I think server should send the update messages after DB updates have been done, so I agree with solution 4. I am assuming here that your websocket server is the same server running PHP and doing the DB updates.
However, depending on your use case, client should still send a hash value on the next request identifying its "view of the world", so you would not be doing identical updates multiple times if a connection gets broken.
Update 2: so it was now understood that you indeed use a separate, standalone websocket server. Basically you have two different web servers on the server side and are having an issue on how to communicate between the two. This is a real issue, and I'd recommend only using one server at a time - either take a look at using Apache websocket support (experimental and not really recommended) or migrating your php scripts to the websocket instance.
Neither PHP or Apache was really build with websockets in mind. It is quite easy to set up a standalone websocket server using only PHP, but it might not be so easy then to migrate the rest of the PHP stack to it if the code is relying on Apache/web server on. Apache websocket support also is hardly optimal. For a real websocket solution, unfortunately, best practice would be using a technology that is built for it from the ground up.
The better answer is to send notification through Server side when database is updated by PHP script, so that script have to add options of web sockets to directly send notification to all web socket clients registered.
User send content->Php script process content and save data according to true condition->check database is updated by checking return of mysql_query/other alternative->if true than use web-socket and send notification to all users
now this is more easy/handy/bandwidth saver.
Of course I am aware of Ajax, but the problem with Ajax is that the browser should poll the server frequently to find whether there is new data. This increases server load.
Is there any better method (even using Ajax) other than polling the server frequently?
Yes, what you're looking for is COMET http://en.wikipedia.org/wiki/Comet_(programming). Other good Google terms to search for are AJAX-push and reverse-ajax.
Yes, it's called Reverse Ajax or Comet. Comet is basically an umbrella term for different ways of opening long-lived HTTP requests in order to push data in real-time to a web browser. I'd recommend StreamHub Push Server, they have some cool demos and it's much easier to get started with than any of the other servers. Check out the Getting Started with Comet and StreamHub Tutorial for a quick intro. You can use the Community Edition which is available to download for free but is limited to 20 concurrent users. The commercial version is well worth it for the support alone plus you get SSL and Desktop .NET & Java client adapters. Help is available via the Google Group, there's a good bunch of tutorials on the net and there's a GWT Comet adapter too.
Nowadays you should use WebSockets.
This is 2011 standard that allows to initiate connections with HTTP and then upgrade them to two-directional client-server message-based communication.
You can easily initiate the connection from javascript:
var ws = new WebSocket("ws://your.domain.com/somePathIfYouNeed?args=any");
ws.onmessage = function (evt)
{
var message = evt.data;
//decode message (with JSON or something) and do the needed
};
The sever-side handling depend on your tenchnology stack.
Look into Comet (a spoof on the fact that Ajax is a cleaning agent and so is Comet) which is basically "reverse Ajax." Be aware that this requires a long-lived server connection for each user to receive notifications so be aware of the performance implications when writing your app.
http://en.wikipedia.org/wiki/Comet_(programming)
Comet is definitely what you want. Depending on your language/framework requirements, there are different server libraries available. For example, WebSync is an IIS-integrated comet server for ASP.NET/C#/IIS developers, and there are a bunch of other standalone servers as well if you need tighter integration with other languages.
I would strongly suggest to invest some time on Comet, but I dont know an actual implementation or library you could use.
For an sort of "callcenter control panel" of a web app that involved updating agent and call-queue status for a live Callcenter we developed an in-house solution that works, but is far away from a library you could use.
What we did was to implement a small service on the server that talks to the phone-system, waits for new events and maintains a photograph of the situation. This service provides a small webserver.
Our web-clients connects over HTTP to this webserver and ask for the last photo (coded in XML), displays it and then goes again, asking for the new photo. The webserver at this point can:
Return the new photo, if there is one
Block the client for some seconds (30 in our setup) waiting for some event to ocurr and change the photograph. If no event was generated at that point, it returns the same photo, only to allow the connection to stay alive and not timeout the client.
This way, when clients polls, it get a response in 0 to 30 seconds max. If a new event was already generated it gets it immediately), otherwise it blocks until new event is generated.
It's basically polling, but it somewhat smart polling to not overheat the webserver. If Comet is not your answer, I'm sure this could be implemented using the same idea but using more extensively AJAX or coding in JSON for better results. This was designed pre-AJAX era, so there are lots of room for improvement.
If someone can provide a actual lightweight implementation of this, great!
An interesting alternative to Comet is to use sockets in Flash.
Yet another, standard, way is SSE (Server-Sent Events, also known as EventSource, after the JavaScript object).
Comet was actually coined by Alex Russell from Dojo Toolkit ( http://www.dojotoolkit.org ). Here is a link to more infomration http://cometdproject.dojotoolkit.org/
There are other methods. Not sure if they are "better" in your situation. You could have a Java applet that connects to the server on page load and waits for stuff to be sent by the server. It would be a quite a bit slower on start-up, but would allow the browser to receive data from the server on an infrequent basis, without polling.
You can use a Flash/Flex application on the client with BlazeDS or LiveCycle on the server side. Data can be pushed to the client using an RTMP connection. Be aware that RTMP uses a non standard port. But you can easily fall back to polling if the port is blocked.
It's possible to achive what you're aiming at through the use of persistent http connections.
Check out the Comet article over at wikipedia, that's a good place to start.
You're not providing much info but if you're looking at building some kind of event-driven site (a'la digg spy) or something along the lines of that you'll probably be looking at implementing a hidden IFRAME that connects to a url where the connection never closes and then you'll push script-tags from the server to the client in order to perform the updates.
Might be worth checking out Meteor Server which is a web server designed for COMET. Nice demo and it also is used by twitterfall.
Once a connection is opened to the server it can be kept open and the server can Push content a long while ago I did with using multipart/x-mixed-replace but this didn't work in IE.
I think you can do clever stuff with polling that makes it work more like push by not sending content unchanged headers but leaving the connection open but I've never done this.
You could try out our Comet Component - though it's extremely experimental...!
please check this library https://github.com/SignalR/SignalR to know how to push data to clients dynamically as it becomes available
You can also look into Java Pushlets if you are using jsp pages.
Might want to look at ReverseHTTP also.
How does Stack Overflow show the answer added/edited message without a page reload?
Yes, I know it will happen with Ajax. So I opened Firebug in a browser to check it, to see whether any requests are coming in a particular interval of time.
But I don't see any request coming in Firebug.
Can we perform a request without it showing in Firebug?
Are there any other ideas behind this or is my concept totally wrong?
It appears to be using HTML 5 Web Sockets. They basically keep an open connection between the server and the client and, among many other things, allow the client to define event handlers to treat new data when received from the server.
Here you'll find some basic examples and concepts about WebSockets: Introducing WebSockets: Bringing sockets to the web.
The WebSocket specification defines an API establishing "socket"
connections between a web browser and a server. In plain words: There
is an persistent connection between the client and the server and both
parties can start sending data at any time.
There is also a live demo with server & client source code available.
You might also find the following question useful: How do real time updates work?
To add to Xavi's answer, if you want to get started with web sockets without having to understand all the internals, you might try out Pusher, a library for multiple platforms (including PHP) that makes push notifications on the web very straightforward.
I do not work for Pusher; it's just a product I've found very useful in the past. I've always used the free version for small personal projects, though I would probably pay if I ever used it on a larger application.