Express upload a file on load balancing? - javascript

Recently i made an app in node js with a load balancing feature. I made a server just for the db itself and other for managing request. The problem is, in the app you can upload a file with multer and in express you upload the file to that server. Its a express static.
For example i have 4 server, one for the db, 2 for the apps, and 1 for the load balancer.
When the loadbalancer request to app-1 server, the file upload to app-1 server not to db server. So when i try to access the file from app-2 server the file didn't exist.
Is there any ways to solve this problem?
Or better ways to use the load balancer?
because im new with load balancer. Thanks

Where are you storing the file that has been uploaded through app-1? You can for example store it in a bucket and keep the reference in your SQL DB. So, when app-2 tries to access the file, it directly asks the DB, which is shared between your apps.
To answer your question, you have 2 solutions:
maintaining a session persistence with your load balancer, so a user that requests to app-1 keeps requesting to app-1 until his session expires.
having a stateless backend design, meaning that you don't need sessions, so any user can send a request to any app instances you have, it will behave the same.
I would go with solution 2, it's easier after you get your head around the stateless concept.
Having an external bucket and not relying on your app internal memory and storage system is a good way to implement a stateless architecture.

Related

How to fetch data as .csv file from client with node.js application

i have written a game in javascript with the p5.js library. Now i want to host the game on a server to conduct a survey on a service like amazon turk. Ideally the clients recieve a URL to the game and play it while in-game actions are tracked and stored in node.js or on the server and exported as a .csv file once they are done playing. After they finish the game the csv. file should be sent automatically to a location that i can then access. I have zero experience in server hosting or similar topics.
So a couple questions arise:
Is a hosting service like Heroku suitable for hosting the game?
Do i need to use node.js to make this happen?
Which of those two would extract the data and store it to a csv? And where is the file stored?
How do i get or access the csv. after?
Any alternative takes to solve the problem?
Thanks alot in advance!
github repository: https://github.com/luuuucaaa/schaeffers-charade
game on github pages: https://luuuucaaa.github.io/schaeffers-charade/
If I were you, I would do it like below:
Host
Since your project is basically a html & JavaScript static contents,
AWS S3's static hosting would be sufficient (Also, the current git hub pages is another option if you just want to host it).
Hosting on node.js environment is also available using webpack serving, but it requires additional works. (but if you require other npm packages to generate .csv file, you need webpack anyway to bundle js file and attach it to html)
Data Storing
Two ways are considerable,
the first is to store it on the filesystem. Generate .csv via JS script within your app, and save it where the app is hosted (if you go with s3, you can access it afterwards, but I'm not sure if it can write objects by script)
The second is to post the data to another API endpoint. (for example building an API Gateway on AWS that triggers Lambda, which stores it on S3)
It's merely an example and I don't know exactly what you want to achieve, but take it into considerations. Good luck. Cool game BTW.

Is there a way to avoid SignalR when uploading files in Blazor Server-Side

I have a Blazor App (server-side) that is hosted in Azure as an App Service, I use Azure SignalR Service to be able to scale the amount of users being connected at the same time. I also have a page where you can upload files, the files might be large, even 1GB and more, I'm using Tewr.Blazor.FileReader (Nuget) to read the file in chunks and upload it to an API that saves the file to the server. But I was wondering if it's possible to avoid the SignalR part in some way when uploading the file, because for 2GB file, Azure SignalR breaks each message to 2KB/each, which means I hit the limit of 1M messages/day/unit, just for uploading 1 file.
Let me know of all the ways I can approach this problem, currently I've few solutions in mind:
Have separate SignalR App hosted as App Service instead of using Azure SignalR
iframe that page to ASP.NET Core App that only handles the transfer of the files. (Not sure if this will trigger SignalR traffic)
If you use standard DOM events (e.g. onchanged=someJavaSciptFunction) then the Browser will run JavaScript in response to that event.
If you set the appropriate JS script on your UI elements then you can have the JS make an HTTP request to your API. This will avoid SignalR completely.
You could upload the file to a Web API controller. You can even add the controller in the same project.
I think this approach is used by the DevExpress Blazor FileUpload.
But it should be easy to implement it yourself. You need some javascript that POSTs the file content to your Web API controller. This blog entry could help you.

Request timeout while uploading movie to app service

I am having trouble uploading a big movie on to the Azure app service which I created. I get request timeout after 4-5 mins while uploading the movie (greater than 150MB). For the frontend, I am using VueJS and send multiple files by doing promisify all settled function. Don't have any issues while using it locally. For backend, I am using Nodejs(fastify) with a multer package and I am using an in-memory storage option. Once I receive the file basically I upload it to Azure blob storage.
Do I have to send movie data in chunks from the frontend to backend? how to achieve it when I have multiple files.
Can we use socket io?
I tried using socket io. however, my browser freezes if I send a big file and I am totally new to sockets.
I am not sure how can I fix this issue. It would be great if someone can guide me and show me an example.
Looking forward to hearing from you guys
thanks,
meet
Problems for uploading files to server
Check the timeout in your axios request (front end) - because you have to wait until the all files uploaded to server (https://github.com/axios/axios#creating-an-instance).
Check the domain hosting configuration (if you are hosting your backend service in nginx - check the upload limit (https://www.tecmint.com/limit-file-upload-size-in-nginx/))

How to fetch data only once in a Next.js app and make it accesible to all the app, both in server and client

I'm working on a Next.js app that needs to fetch a config file from a remote server before initializing. I want to request the config file just once per call to the server before rendering the app server side. After that, I would like to be able to get the same config in the client without having to make a second request to the remote server from the browser.
I have tried to achieve this by using getInitialProps function either in _app or _document files and then use the React's Context API to make the configuration visible to every component but unless I'm wrong, this will run the code that requests the configuration both in server (on the first call from the browser) and client (on every page navigation).
I have also tried to create a server.js file, request the configuration from there and store in a variable within a ES6 module. However, I couldn't make this approach work because apparently the Next.js React app can't access the same modules than the server.js because they are actually two different apps. Again, I could be wrong.
Basically I would like to know if Next.js offers any kind of "bootstrapping place" where I can perform app initialization tasks that generate data that can be sent to the React app Next.js will initiate.
You are correct in that anything in getInitialProps in _app.js or _document.js will be run on every server request. Once you get your config, you can pass it down to the components in pages as props. From there, I think you have two choices:
Now that the app is bootstrapped, run the rest of the app as a SPA client-side. This would prevent any future SSR from happening. Obviously, you lose the benefits of SSR and the initial page load is likely longer, but then it would be snappy afterwards.
After getting the config in _app.js, send it as a cookie (assuming it's not too big to be a cookie?). On future requests to the server, the cookie will be sent automatically and you would check the cookie first - if it doesn't exist, get the config. If it does exist, skip that more expensive bootstrapping because the app is bootstrapped.
So I think it really depends on whether you want a single page application bootstrapped on the server but then entirely client side after that (option 1) or server side rendering per page while minimizing expensive bootstrapping (option 2).
Nothing prevents you from sending multiple cookies from the server if that makes sense for your app and bootstrapping. And remember not to make it a HTTP-Only cookie because you'll want to read that cookie client side - after all, that's what you're looking for - the client side configuration generated on the server.
Despite we ended up leaving Next.js for this and other reasons that made us decide Next.js was not the best option for our use case, we kind of mitigated this problem as follows while be sticked to it, I hope it makes sense to anyone. Also, by now maybe Next.js provides a better way to do this so I would read the Next.js docs before using my approach. Final note: I don't have access to the code anymore since I change to a different company so maybe there are some points that won't be 100% as we made it.
There is goes:
We created a module that was responsible to request the config file and keep the results in a variable. At the moment of importing this module, we ensure that the variable is not already present in window.__NEXT_DATA__. If it is, we recover it, if it's not, we request it to the remote server (this will be helpful in the clint side rendering).
We created a server.js file as described by Next.js docs. In this file we make the call to get the config file and store it in memory.
In the body of the function passed to createServer we add the config file into the req object to make it accesible to the app in the getInitialProps functions server side.
We made sure that the getInitialProps using the config file returns it, so that it will be passed to the components as props and also serialized by Next.js and made available to the client in the __NEXT_DATA__ global Javascript variable.
Given that the config ended up in the __NEXT_DATA__ variable in the server, using the trick described in the step 1 makes the app not request the config for a second time.

Angular - how to test Internet upload speed without backend?

I want to upload file into folder from which my Angular app is served while running on localhost. I'm not able to find any solution without using backend.
For example I just want to upload an image file and that file should copy in specified folder of the project. This should be done only with Angular without using any Backend script or hitting any API endpoint.
Depending on your webhost, you can make your assets-folder accessible via FTP.
Making a FTP-call from javascript (angular is javascript) isn't that difficult. And there are plenty of example and questions about it on the internet (like this)
Why you wouldn't do that:
The credentials for your ftp-connection will be accessible in the compiled javascript-code. With a little bit of effort, everyone can find it.
Each gate you open through the webhosts firewall, is a extra vulnerability. Thats why everybody will recommend you to add an API endpoint for uploading files so that you keep holding the strings of what may be uploaded.
Edit:
As I read your question again and all the sub-answers, I (think) figured out that you are building an native-like app with no back-end, just an angular-single page front-end application. An I can understand why (you can run this on every platform in an application that supports javascript), but the problem you are encountering is only the first of a whole series.
If this is the case, I wouldn't call it uploadingas you would store it locally.
But the good news is that you have localstoragefor your use to store temporary data on the HDD of the client. It isn't a very large space but it is something...
The assets folder is one of the statically served folders of the Angular app. It is located on the server so you can't add files to it without hitting the server (HTTP server, API, or whatever else...).
Even when running your app on localhost, there's a web server under the hood, so it behaves exactly the same than a deployed application, and you can't add files to the assets folder via the Angular app.
I don't know what exactly you want to do with your uploaded files, but:
If you want to use them on client side only, and in one user session, then you can just store the file in a javascript variable and do what you want with it
If you want to share them across users, or across user sessions, then you need to store them on the server, and you can't bypass an API or some HTTP server configuration
Based on your clarification in one of your comments:
I'm trying to develop a small speed test application in which user can upload any file from his system to check upload and download speed.
The only way to avoid having you own backend is to use 3rd party API.
There are some dedicated speed test websites, which also provide API access. E.g.:
https://myspeed.today
http://www.speedtest.net
https://speedof.me/api.html
Some more: https://duckduckgo.com/?q=free+speedtest+api
Note, that many of these APIs are paid services.
Also, I've been able to find this library https://github.com/ddsol/speedtest.net, which might indicate that speedtest.net has some kind of free API tier. But this is up to you to investigate.
This question might also be of help, as it shows using speedtest.net in React Native: Using speedtest.net api with React Native
You can use a third party library such ng-speed-test. For instance here is an Angular library which has an image hosted on a third party server (ie GitHub) to test internet speed.

Categories