We are working on a RoR project implementing an LMS. We need to send data to an external REST service provided by an external server. The data is sent when certain events are accomplished, it is possible that some of those are not triggered by the client (clicks, etc.).
Also, we need to keep consistency in our rails models, because we need to keep record of the user activities.
There is a library provided to work with the API, written in JavaScript. It makes most of the work easy, so we would like to use instead of creating our own implementation for the API requests.
What are the differences between each of the following approaches? Would one be preferable to another?
Use javascripts to send the data, inserting the snippets in the
views, from the client, but having the client execute this might have
some serious implications (scores changed, false success, etc).
Use a NodeJS server to execute the Javascript but we don't really know how to communicate with our main server (Rails)
And finally, use a HTTP client from the Rails app to send the requests to the service. However we don't know exactly how to do it, also there is the question of where this code goes in the MVC pattern.
Option #1, as you've likely realized, is out of the question. For the client to make API calls on your behalf, you would need to send them your secret key/token/whatever you need to authenticate with the API. But once they have that, they could just use a script console to make whatever API calls they want "as you". This would be pretty disastrous.
Option #2 might be prohibitively complex -- I'm personally not sure how you'd go about it. It is possible, using a library like therubyracer, to execute JavaScript code from Ruby code, but there is some degree of sandboxing and this may break code that requires network access.
That leaves you with Option #3, writing your own Ruby library to interact with the API. This could be easy or difficult, depending on how hairy the API is, but you already have a JavaScript version on hand (and hopefully docs for the REST service itself), so combined with something like RestClient or HTTParty the path forward should be clear.
As for where the API calls would fit in your Rails code: If you have models that are basically mirroring the resources you're interacting with through the REST service, it might make sense to add the relevant API calls as methods or callbacks on those models. Otherwise it might be fine to put them in the relevant controller actions, but keep an eye on your code complexity and extract to a separate class or module if things are getting ugly.
(In cases where you don't need to wait for the response from the API before sending something back to the user, you may want to use DelayedJob or similar to queue your API calls in the background.)
Related
I am currently developing a front-end app with React. During the development, I create some objects and use them to render test data. The app intended to work with a Spring Boot application on the server side. I have performed certain tests to ensure communication between front and back end before, however to simplify my development process I thought about using a RESTful API (that is ideally available online as a free testing service) where I would send JSON objects and receive the same object back.
I realize that this sounds counter-intuitive, but here is my reasoning:
I already create my own data, but creating a temporary API just to test would be time loss.
I don't mind still having to pollute my front-end with the data I am normally expected to receive from back-end, because I'll be more aware of the network interaction of my components while implementing them.
So the point is not exactly the data we fetch, but the way we fetch it. Currently I won't be working with our own back-end application since it is just too bloated/incomplete to work with. Using publicly available test APIs with their predetermined data types seems infeasible, because I happen to work with a specific data type that has a lot of custom and necessary fields.
I made some quick searching, but couldn't find an API like that. I could create a fast REST API locally, but that would be far from ideal in my case given that on a realistic scenario I'll have the delay and slightly different asynchronous nature of network interaction, not to mention CORS related configurations etc.
To be short, my question is as follows:
Is it a known practice to use such API's that receives POST requests and responds same objects back (although it sounds weird)? Is there any service that you could recommend for me to use?
Thanks in advance.
There's another ECHO api that's a little easier to use by zuplo.com at https://echo.zuplo.io - you don't need to add the method to the URL, it just echoes everything, including the method, back at you.
There are Postman echo APIs to do exactly the same.
The API echoes back what you sent it, including each of the data items you included in the request as part of the response postman-echo.com/get, postman-echo.com/post
etc.
Please check this link for more details learning.postman.com/docs/developer/echo-api
And about this part of the question : "Is it a known practice to use such API's that receives POST requests and responds same objects back?"
Yes we do this to test the code using these echo APIs. All these are based on the requirements of the problem at hand.
I am working on a React-based web app that uses Tensorflow.js to run an AI model in realtime on the client in the browser. I've trained this AI model from scratch and I'd like to protect it from being intercepted and used in other projects. Are there any protections available to do this (obfuscation, DRM, etc.)?
From a business perspective, I'd only like the model to work on my web app, nowhere else.
The discussions (1 2 3) I've been able to find on this are more geared toward native apps, not web apps.
Here is an example open source web app that uses Tensorflow.js. These weights are an example of what I would like to protect in my app.
Client-side code obfuscation will never fully prevent it. Use a server instead.
Obfuscation
If your client-side application contains the model, then the user will be able to somehow extract it. You can make it harder for the user, but it will always be possible. Some techniques to make it harder are:
Obfuscating your code: That way the user will not be able to read your code and comments easily. Depending on your build tools, this might already be done for you when you produce a "production ready" build.
Obfuscating the library and its public API: Even if your code is obfuscated, the user might still be able to guess what is going on by seeing the public API calls of the library. Example: It would be rather easy to set a break point at the model.predict function and debug your code from there on. By also obfuscating libraries and their API, this will become harder.
Put "special checks" in your code: You could also check if the page the code is running on is your page (e.g. if the domain matches), etc. You also want to obfuscate this code as well.
Even if your code is perfectly obfuscated and well protected, your client-side code still contains your model somewhere. With these methods it will always be possible to somehow extract your model.
Server-side approach
To make it impossible to get your model, you need a different approach. Only put your "dumb logic" on the client. Exclude the part of code that you want to protect. Instead you offer a API on your server that executes the "protected part" of your code.
This way, instead of running model.predict on the client-side, you would make an AJAX request to your backend (with the parameters) and then return the results. That way the user only sees the input and the output and cannot extract the model itself.
Keep in mind that this means a lot more work, as you not only have to write the code for your client-side application but also for your server-side application, including the API. Depending on how your application looks like (e.g.: does it have a login?), this might be a lot more code.
Another way you can protect your model is to split the model into more than one blocks. Put some blocks at server side and some at client side. This method may also introduce a lot of engineering work, but once you do that you can trade off the computation loading and network latency between the server and client. Users can only get some model blocks which is useless without cooperating with server side blocks.
I've read a few StackOverflow posts related to this subject but I can't find anything specifically helps me in my scenario.
We have multiple monitoring instances within our network, monitoring different environments (Nagios, Icinga, more...). Currently I have a poller script written in PHP which runs every minute via cron, it asks the instance to return all of its problems in JSON, the script then interprets this and pushes it in to a MySQL database.
There is then an 'overview' page which simply reads the database and does some formatting. There's a bit of AJAX involved, every X seconds (currently use 30) it checks for changes (PHP script call) and if there are changes it requests them via AJAX and updates the page.
There's a few other little bits too (click a problem, another AJAX request goes off to fetch problem details to display in a modal etc).
I've always been a PHP/MySQL dev, so the above methodology seemed logical to me and was quick/easy to write, and it works 'ok'. However, the problems are: database constantly being polled by many users, mesh of javascript on the front end doing half the logic and PHP on the back doing the other half.
Would this use case benefit from switching to NodeJS? I've done a bit of Node.JS before but nothing like this. Can I subscribe to MySQL updates? Or trigger them when a 'data fetcher' pushes data in to the database? I've always been a bit confused as I use PHP to create data and javascript to 'draw' the page, is there still a split of NodeJS doing logic and front end javascript creating all the elements, or does NodeJS do all of this now? Sorry for the lack of knowledge in this area...
This is definitely an area where Node could offer improvements.
The short version: with websockets in the front-end and regular sockets or an API on the back-end you can eliminate the polling for new data across the board.
The long version:
Front-end:
You can remove all need for polling scripts by implementing websockets. That way, as soon as new data arrives on the server, you can broadcast it to all connected clients. I would advise Socket.io or the Primus websocket wrapper. Both are very easy to implement and incredibly powerful for what you want to achieve.
All data processing logic should happen on the server. The data is then sent to the client and should be rendered on the existing page, and that is basically the only logic the client should contain. There are some frameworks that do all of this for you (e.g. Sails) but I don't have experience with any of those frameworks, since they require you to write your entire app according to their rules, which I personally don't like (but I know a lot of developers do).
If you want to render the data in the client without a huge framework, I highly recommend the lightweight but incredibly useful Transparency rendering library. Using this, you can format a Javascript object on the server using Node, JSONify it, send it to the client, and then all the client would have to do is de-JSONify it and call Transparency's .render.
Back-end:
This one depends on how much control you have over the behaviour of the instances you need to check. I assume you have some control, since you can get all their data in a nice JSON format. So, there are multiple options.
You can keep polling every so often. This is the easiest solution since it requires no change to the external services. The Javascript setInterval function is very useful here. Depending on how you connect with the instances, you might be able to use a module like Request to do the actual request, so that takes out a bunch more of the heavy lifting.
The benefit of implementing the polling in your Node app as well, is that you will receive the data in your Node app and that way you can immediately broadcast it to the clients, even before inserting it into a database. This will greatly reduce the number of queries on your database.
An alternative to polling would be to set up a simple Express-based API where the applications can post their 'problems', as you call them. This way your application will get notified the moment a problem occurs, and combined with the websockets connection to the client this would result in practically real-time updates.
To be more redundant, you would have a polling timer alongside the API, so that you can check the instances in case there's something wrong that causes them to not send over any more data.
An alternative to the more high-level API would be to just use direct socket communication, which is basically the same approach only using a different set of functions.
Lastly, you could also keep the PHP-based polling script. This would be the most efficient solution since you wouldn't go and replace everything. Then from the Node app that's connected to the clients with websockets, you could set an interval to query the database every so often and broadcast the updates. This will still greatly reduce the number of queries, since no matter how many clients are connected there will only be one query, the response of which then gets sent to all connected clients.
I hope my post has give you some ideas of how you could implement your application using Node. Keep in mind though that I am just one developer, this is how I would approach building your application in Node. There will definitely be others who have different opinions.
As I can't orientate freely in the topic of building dynamic sites, it is quite hard to me to google this. So I'll try to explain the problem to you.
I'm developing a simple social network. I've built a basic PHP API represented by the files like "get_profile.php", "add_post.php", etc. with the POST method that is used to pass some data. Then I try to get the data using JS AJAX (php functions return it by JSON), which means I get all the data that I need to show on a page after the page is loaded. That causes decreasing of a page loading speed and I feel like this structure is really wrong.
I hope you'll explain me how to build a proper structure or at least give me some links to read. Thanks.
Populate the HTML with the (minimum) required data on the server side and load all other necessary data on the client side using AJAX (as you already do).
In any case, I would profile your application to find the most important bottle necks. Do you parallelize AJAX requests?
Facebook, for example, doesn't populate its HTML with the actual data on the server side, but provides the rough structure, which is later filled using AJAX requests.
If I understood your architecture right, it sounds ok.
Advices
Making your architecture similar to this allows you to deliver templates for the page structure that you then populate with data from your ajax request. This makes your server faster also since it doesn't have to render the HTML also.
Be careful with the amount of requests you make though, if each client makes a lot of them you will have a problem.
Try and break your application into different major pieces and treat each one in turn. This will allow you to separate them into modules later on. This practice is also referred as micro-services architecture.
After you broke them down try and figure user interaction and patterns. This will help you design your database and model in a way in which you can easily optimise for most frequest use-cases.
The way of the pros.
You should study how facebook is doing things. They are quite open about it.
For example, the BigPipe method is the fastest I have seen for loading a page.
Also, I think you should read a bit about RESTful applications and SOA type architectures.
I've been using a couple RESTful APIs for server to server interactions recently and am now thinking about communicating directly with a RESTful API via javascript from within a web browser in my web application.
This would mean using ajax within a web page to talk to my web server using GET, POST, PUT and DELETE requests and the server responding with appropriate http status codes and non-html data (probably json)
Is this generally considered good practice for a web application and why?
It doesn't matter if you are consuming an RPC API or RESTful API from the ajax standpoint, but generally, you can think of a RESTful API as a well-organized, well-namespaced set of remote procedural calls.
Is this generally considered good practice for a web application and
why?
It is useful to do things this way because you don't need to duplicate code in order to have regular CRUD operations across multiple data objects.
Another thing to consider is that if you have a uniform naming convention of API calls that you can write AJAX functions to interact with, you will write and maintain much less code over time in your javascript side of the application, assuming you don't do anything weird in your code.
An example of when / how this would be a good practice would be if you had written a base method designed to automatically determine your AJAX url depending on what you're doing and where you are, and it automatically determines what POST method to use depending on the type of operation... then you literally write one ajax function, and apply it to things rather than write fully separate ajax methods per action item.