Spring Framework to Optimize SQL connection? - javascript

So I am on a webproject that succesfully connects to and reads from an SQL database. The code that connects to it looks like this.
//From Here
var connection = new ActiveXObject("ADODB.Connection") ;
var connectionstring="Data Source=<server>;Initial Catalog=<catalog>;User ID=<user>;
Password=<password>;Provider=SQLOLEDB";
connection.Open(connectionstring);
var rs = new ActiveXObject("ADODB.Recordset");
//To Here
rs.Open("SELECT * FROM table", connection);
rs.MoveFirst
while(!rs.eof)
{
document.write(rs.fields(1));
rs.movenext;
}
rs.close;
connection.close;
Simple and effective and I have it working fine. But those first 4 (marked from here to here) run horribly slow and I have to reconnect every time I need to read from or write to the sql database... which is a lot for my project. So every time I run this code (which is on every other webpage I am creating in this project) I have to sit and wait for this code to run.
I have been told/ required for the project, to configure the design using javascript and spring framework. Apparently there is either a) a way to hold the connection so I don't have to run this code a every time I hit a go to new page or b) a different method of connection to the SQL database (something to replace those 4 lines of code. Both of which have to do with my using the Spring Framework.
I have never used Spring Framework before and need to learn fast. Been watching their website tutorials but still have no idea what to do or how to do it. Let me know if have any ideas.
Please and Thanks.

I think you got something wrong here: You aren't supposed to access your database from JavaScript because any user can access any data that way, or delete everything.
You should access the database server-side only.

You need connection pooling, which means, instead of opening and closing a new connection every time, keep the one that you were about to close, and see if you can reuse it later.
Unfortunately, I've never seen anything in JavaScript that allows you to reuse objects across pages, so there's no way afaik to do this in JavaScript. It's considered a very bad practice to connect from a browser directly to a database, anyway.
Usually, what you do is put a java or C# application server in between, and let these to the database access, using a connection pool. From your javascript, you then do an AJAX call to the application server, which will use the connection pool, and return e.g. JSON objects.
See for example http://tomcat.apache.org/tomcat-6.0-doc/jndi-datasource-examples-howto.html#MySQL_DBCP_Example

Related

Display Kafka messages on web page

I have a Java Spring Application with a Tomcat server that listen on kafka topic. I want to display all messages in a real-time mode on the web page. Therefore, when a kafka messages is arrived in the backend I want to see it on my web page. I don't know a good approach to push kafka message directly to the front-end and display it on web page. Is someone could help my with a solution and some examples that could help? Thanks!
I have implemented a system like this in Java for my last employer, albeit not with Spring/Tomcat. It was consuming messages from Kafka and serving them on a web socket to be displayed in the browser. The approach I followed was to use akka-stream-kafka and akka-http for web-socket support. The benefit of that is both are based on akka-streams which makes it an easy fit for streaming data.
While you can embed akka-http in your spring app running inside tomcat, it may not feel the most natural choice any more as spring framework already has its own support for both kafka and websockets. However, if you're not familiar with either, then jumping on the akka approach may be easiest and the core logic goes along these lines (I can't share the code from work so have just put this together from the examples in the docs, not tested):
public Route createRoute(ActorSystem system) {
return path("ws", () -> {
ConsumerSettings<byte[], String> consumerSettings = ConsumerSettings.create(system, new ByteArrayDeserializer(), new StringDeserializer())
.withBootstrapServers("localhost:9092")
.withGroupId(UUID.randomUUID().toString()) //this is so that each client gets all messages. To be able to resume from where a client left off in case of disconnects, you can generate in on the client side and pass in the request
.withProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest")
return handleWebSocketMessages(
Flow.fromSinkAndSourceCoupled(
Sink.ignore(),
Consumer.committableSource(consumerSettings, Subscriptions.topics("topic1"))
.map(msg -> TextMessage.create(msg.record().value()))
)
);
}
}
To expose this route you can follow the minimalistic example, the only difference being the route you define needs the ActorSystem:
final Http http = Http.get(system);
final ActorMaterializer materializer = ActorMaterializer.create(system);
final Flow<HttpRequest, HttpResponse, NotUsed> routeFlow = createRoute(system).flow(system, materializer);
final CompletionStage<ServerBinding> binding = http.bindAndHandle(routeFlow,
ConnectHttp.toHost("localhost", 8080), materializer);
Once you have your messages published to the websocket, the front end will code will of course depend on your UI framework of choice, the simplest code to consume ws messages from javascript is:
this.connection = new WebSocket('ws://url-to-your-ws-endpoint');
this.connection.onmessage = evt => {
// display the message
To easily display the message in the UI, you want the format to be something convenient, like JSON. If your Kafka messages are not JSON already, that's where the Deserializers in the first snippet come in, you can convert it to a convenient JSON string in the Deserializer or do it later on in the .map() called on the Source object.
Alternatively, if polling is an option you can also consider using the off-the-shelf Kafka Rest Proxy, then you only need to build the front-end.

connection timeout OrientDB with Javascript API

In my webapp client side script I'm using the OrientDB Javascript API (orientdb-api.js). When the script initializes I run this code:
var orientdb = new ODatabase("http://localhost:2480/testapp");
var orientdbinfo = orientdb.open('root', 'admin');
This works fine and I can do all the various queries etc, as long as I don't wait more than 15 seconds between them. If I do, I get "error 401 (Unauthorised)" returned.
I know for a fact that this is the socket connection timing out. The timeframe matches the 15000ms timeout setting in the config. Also, as a test I've built a little button that calls the orientdb.open method above and reopens the connection. After I hit that button I can access the DB again.
Currently the queries and commands are being called directly in my script as I trigger actions from my web UI. Am I just being lazy and am I actually supposed to wrap every query in a function that tests the connection first and re-initializes if it is closed, or is there something I'm missing? If the former, what is an elegant way of coding that? If the latter, what am I missing?
To get around this I'm running a setInterval function that opens a new socket every 14 seconds. That will get me through my testing for sure, but I realise it's a hack.
When you start the OrientDB server, it creates two sockets: 2424 (binary) and 2480 (HTTP).
Because OrientJS uses the binary protocol, you need to connect to port 2424.
Try:
var orientdb = new ODatabase("http://localhost:2424/testapp");
var orientdbinfo = orientdb.open('root', 'admin');
And the socket should stay open (longer).

I am trying to manage a small number of users control of a webpage

So to start off, I have a raspberry pi, running a lighttp server on arch. I have a webpage that will have a max of 10 people connected at a time. Each user will be given the tag "Master," or "observer." The webpage will have controls that only the "Master," can use.
Just a heads up: I am just learning most of this so I may be making some mistakes about how to accomplish this.
My original idea is has follows. When a user connects to the database their IP address would be grabbed and inserted into a SQLite database, along with a user tag, and time of connection. From there I would be able to query the database for the users tag when they tried to execute various commands.
Whatever I use needs to be very lightweight and not store cookies on the users device.
Here is the JavaScript I currently have, it probably isn't the most efficient, but I plan on getting it working then making it look nice.
This code is supposed to connect the databases and insert the user.
<script type="text/javascript" src="http://l2.io/ip.js?var=myip"></script>
<script type="application/javascript">
var db = openDatabase('userCon.contbl.sqlite', '1.0', 'contbl', 1024);
db.transaction(function(transaction) {
var ip = myip;
var conStatus = "master"
var date = new Date();
console.log('Inserting into the database ' + ip + ',' + conStatus +',' + date);
transaction.executeSql('INSERT INTO contbl(ipAd, conType, lastActive) VALUES (?,?,?,?)',[ip,conStatus,date], function(transaction, results) {
}, function (transaction, err){
console.log(err.message+":Error"); // here an error
});
});
</script>
<script type="application/javascript" src="http://jsonip.appspot.com/?callback=getip"> </script>
I am unable to connect to the SQLite database I created on the pi, which after my research may be because SQLite is supposed to be run locally and not on a server.
Is there some sort of work around to point to the SQLite database on the pi, or is there a better resource to use for this type of task?
EDIT:
I guess my original post was not specific enough. The basic idea is I need to be able to pass a tiny bit of information from a webpage, back to the server hosting it. i.e. User connect to the server and sends its IP then the server tags that IP as an Observer of Controller. From there the server will treat each person viewing the webpage differently based on how the user was tagged.
My original plan was to use a really light weight database like SQLite, but as I found out, SQLite is local use only. I need to do this on a sever with lower then 10 connections.
My hope is someone has a suggestion, or an example to solve this problem.
The most common way for javascript running on a web page to talk to a server these days is some kind of RESTful web service.
You'll want to find a server-side programming language that runs on the Pi. Maybe node.js? It's like javascript but on a server, see here: http://weworkweplay.com/play/raspberry-pi-nodejs/
You then write whatever methods you want in the server-side language, methods which talk to the database, then have your client-side javascript call those methods. Google "REST services node.js" and you'll find plenty of how-tos.
If the SQLite database is running on the server, and this code runs in the user's browser, it can't connect to the database. The only interaction between the browser and your server is HTTP, unless there's a gigantic security hole in your server.

PHP minimal working example of Web Sockets

I'm trying to determine how to setup a web socket for the first time ever so a working minimal example with static variables (IP address for example instead of getservbyname) will help me understand what is flowing where.
I want to do this the right way so no frameworks or addons for both the client and the server. I want to use PHP's native web sockets as described here though without over-complicating things with in-depth classes...
http://www.php.net/manual/en/intro.sockets.php
I've already put together some basic JavaScript...
window.onload = function(e)
{
if ('WebSocket' in window)
{
var socket = new WebSocket('ws://'+path.split('http://')[1]+'mail/');
socket.onopen = function () {alert('Web Socket: connected.');}
socket.onmessage = function (event) {alert('Web Socket: '+event.data);}
}
}
It's the PHP part that I'm not really sure about. Presuming we have a blank PHP file...
If necessary how do I determine if my server's PHP install has this socket functionality already available?
Is the request essentially handled as a GET or POST request in
example?
Do I need to worry about the port numbers? e.g. if
($_SERVER['SERVER_PORT']=='8080')
How do I return a basic message on the initial connection?
How do I return a basic message say, five seconds later?
It's not that simple to create a simple example, I'm afraid.
First of all you need to check in php configuration if the server is configured for sockets with the setting enable-sockets
Then you need to implement (or find) a websocket server that at least follows the Hybi10 specification (https://datatracker.ietf.org/doc/html/draft-ietf-hybi-thewebsocketprotocol-10) of websockets. If you find the "magic number" 258EAFA5-E914-47DA-95CA-C5AB0DC85B11 in the code for the header, you can be sure it does follow at least Hybi06 ...
Finally, you need to have access to an admin console on the server in order to execute the PHP websocket server using php -q server.php
EDIT: This is the one I've been using a year ago ... it might still work as expected with current browsers supporting Websockets: http://code.google.com/p/phpwebsocket/source/browse/trunk/+phpwebsocket/?r=5

Periodic refresh or polling

I am trying to use periodic refresh(ajax)/polling on my site by XMLHttp(XHR) to check if a user has a new message on the database every 10 seconds, then if there is inform him/her by creating a div dynamically like this:
function shownotice() {
var divnotice = document.createElement("div");
var closelink = document.createElement("a");
closelink.onclick = this.close;
closelink.href = "#";
closelink.className = "close";
closelink.appendChild(document.createTextNode("close"));
divnotice.appendChild(closelink);
divnotice.className = "notifier";
divnotice.setAttribute("align", "center");
document.body.appendChild(divnotice);
divnotice.style.top = document.body.scrollTop + "px";
divnotice.style.left = document.body.scrollLeft + "px";
divnotice.style.display = "block";
request(divnotice);
}
Is this a reliable or stable way to check message specifically since when I look under firebug, a lot of request is going on to my database? Can this method make my database down because of too much request? Is there another way to do this since when I login to facebook and check under firebug, no request is happening or going on but I know they are using periodic refresh too... how do they do that?
You can check for new data every 10 seconds, but instead of checking the db, you need to do a lower impact check.
What I would do is modify the db update process so that when it makes a change to some data, it also updates the timestamp on a file to show that there is a recent change.
If you want better granularity than "something changed somewhere in the db" you can break it down by username (or some other identifier). The file(s) to be updated would then be the username for each user who might be interested in the update.
So, when you script asks the server if there is any information for user X newer than time t, instead of making a DB query, the server side script can just compare the timestamp of a file with the time parameter and see if there is anything new in the database.
In the process that is updating the DB, add code that (roughly) does:
foreach username interested in this update
{
touch the file \updates\username
}
Then your function to see if there is new data looks something like:
function NewDataForUser (string username, time t)
{
timestamp ts = GetLastUpdateTime("\updates\username");
return (ts > t);
}
Once you find that there is new data, you can then do a full blown DB query and get whatever information you need.
I left facebook open with firebug running and I'm seeing requests about once a minute, which seems like plenty to me.
The other approach, used by Comet, is to make a request and leave it open, with the server dribbling out data to the client without completing the response. This is a hack, and violates every principle of what HTTP is all about :). But it does work.
This is quite unreliable and probably far too taxing on the server in most cases.
Perhaps you should have a look into a push interface: http://en.wikipedia.org/wiki/Push_technology
I've heard Comet is the most scalable solution.
I suspect Facebook uses a Flash movie (they always download one called SoundPlayerHater.swf) which they use to do some comms with their servers. This does not get caught by Firebug (might be by Fiddler though).
This is not a better approach. Because you ended up querying your server in every 10 seconds even there is no real updates.
Instead of this polling approach, you can simulate the server push (reverrse AJAX or COMET) approach. This will compeletly reduce the server workload and only the client is updated if there is an update in server side.
As per wikipedia
Reverse Ajax refers to an Ajax design
pattern that uses long-lived HTTP
connections to enable low-latency
communication between a web server and
a browser. Basically it is a way of
sending data from client to server and
a mechanism for pushing server data
back to the browser.
For more info, check out my other response to the similar question

Categories