We Keep Coding

Generating XLS files with SheetJS in browser vs. nodeJS server: pros and cons

I want to let users export some pieces of data in XLS files. I have a front-end application on Angular 9 and back-end on ExpressJS. Now I consider 2 ways to implement this with SheetJS: either create some ExportService to do it directly in the browser or implement this on the server-side and provide an API endpoint for exporting. To be more objective, I'd like to know what the community thinks: what are the pros and cons of the both ways?
My special concern is whether there are any functional limitations on what can be generated in browser vs. server. Aren't there some special features that are only available with server-side implementation?

Here's what I can think of. Please feel free to supplement or correct this list to build a full picture.
Client-side implementation
Pros:
no need for an extra HTTP request (unless I need to fetch some extra data; in my case, I don't)
less load on the server: everything happens on the user's machine
it can be faster than server-side implementation when server is over-loaded or internet is slow
Cons:
significantly increased bundle size: the whole SheetJS library has to be bundled
it can be slower than server-side implementation when user's machine is slow
What am I missing?

Alternative to gRPC that uses websockets?

I'm in the middle of a very large project to migrate my application from Java to JavaScript, and am trying to decide what the messaging protocol should be for client/server. Some notes on what I'm looking for:
There is a very large amount of data streamed from the server to client, with infrequent small requests made from the client to server (responses may be large). Network performance is a very big concern.
We are going to deploy the new JS-based client primarily in an Electron container, but with support for "modern" browsers in the future (no ETA for that requirement, but I also don't want to be scrambling when it becomes mandatory).
The server side is and will remain in Java.
Need good backward compatibility options, as we will not always be able to control the client versions.
The research we've done so far has indicated gRPC (with ProtoBuf) as a strong candidate that ticks all the boxes. The grpc package seems to work great in Electron, and the server coding in Java is very easy. The biggest downside is that, since it uses HTTP/2, we have to jump through hoops to run it in a browser (grpc-web combined with a data proxy).
Are there any good alternatives that use websockets, or other suggestions entirely? Ideally we would like to use the same code when executing in Electron vs. browser.

You could look at WAMP protocol. The lead implementation is Autobhan. It's an RPC & PUB/SUB client-server framework that works by exchanging JSON (or msgpack) messages over websocket. So fits in well with Javascript and browser side.

Automating HTTP Requests

I work with a team whose only way to get a user in their company's database, is to navigate through and fill out ~5 or so pages of web forms in their browser. Truly brutal stuff. I've developed web automation scripts in VBScript, Java (w/ Selenium WebDriver) and iMacro, but all of these solutions are slow. They also depend on the browser, which I'm trying to move away from.
I'm looking for a new platform, possibly some scripting technique/language that will allow me to issue HTTP requests and read HTTP responses, then build my script around there. The script would perform calculations on the HTTP responses, use File I/O and use this data to issue further HTTP requests. Again, I'm just spitballing here. If anyone else has a better solution, I'm all ears!
My question for you is: Accepting the team's limitations (read-only DB access), how would you approach a solution and what tools/languages/platforms would you use to do so?
Broad and ambiguous answers are welcome. Thank you for your time.

I agree with #Grisk on using NodeJS/ioJS as a platform. It is a powerful tool designed from the ground up for I/O, making it perfect for solving your problem. Additionally, the node community is extraordinarily vibrant, with npm, the nodejs package manager, hosting thousands of easily accessible modules. To avoid any future confusion: don't mistake NodeJS for a language or a backend framework; it is a native javascript interpreter built atop Google's V8 engine as well as a set of built in modules to build powerful I/O applications. Read up about node online.
As for your specific problem, I'd say you have two options:
To feign being a browser using phantom cookies
By programmatically navigating through the website as you have been doing.
As for the former option, you'd need to manually determine which cookies are sent to the server when forms are submitted on each page and then in your script generate these cookies and include them in the http request. Check out the nodejs http documentation for more information on customizing the headers of requests.
You're header will need to look something like this:
var headers = {
'host': < website host address here > ,
'origin' : <website origin here>
'referer' : <website origin here>
'User-Agent': 'Opera/9.52 (X11; Linux i686; U; en)',
'Cookie': <cookie sent over by server here>
}
I recently came across the node-icloud library, which uses the first method I describe above to provide programmatic access to one's icloud account. I strongly suggest reading through its code to see how it works here.
Additionally, I'd suggest reading up about http headers here
For the second option, check out phantomjs and zombiejs. Phantom is nice because it works without a browser. I'm not sure how the speed of these two libraries compare to what you have already been doing, but they are worth testing out.
One last thing: I would recommend building a custom (JSON)DSL for automating interaction with webpages so you can very easily redesign your browser interaction workflows.
Additionally, if you choose to use nodejs, an understanding of node streams and the details behind its event loop would be beneficial.
Best of luck!

I would start looking into NodeJS as a platform. The HTTP library is an incredibly powerful method for writing applications that need to make multiple http requests with unusual structure and it can communicate easily with a browser or basically anything else you could possibly need. Look at using the FileSystem class if you need to do file I/O.
If you wanted to get really fancy and use websockets to build a dynamic webapp that you can use as a front-end for your tool, you could even do that, so there's a lot of flexibility.

Need some help understanding nodejs and socket.io

Sorry for the rather ignorant question, but I'm a bit confused regarding these two technologies. I wrote a webserver in C# that uses Fleck and everything works great but I realized I probably couldn't find a hosting provider that would run .NET applications.
I want to use websockets and I found socket.io to be really popular but I'm not sure exactly what it is. Correct me if I'm wrong, but, is it just like writing a server in javascript and you run the javascript file with the node.exe application and then the server is running? How do people find hosting providers that will provide that sort of service?
Lastly, is socket.io just an extension of nodejs? Do you have to code your server in javascript when you use socket.io? Again, sorry for the very novice questions but I'm just trying to understand a few basic things before I continue. Thanks.

There are a few companies that will host your node application. Its not the same as your transitional web hosts where you provide them with files and they serve the files for you. When working with node you're writing the actual web server.
Some of the popular ones around are below:
https://devcenter.heroku.com/articles/nodejs
http://nodejitsu.com/
http://nodester.com/
#Roest: A virtual server sounds intriguing. What are the pros and cons
of such an approach? Also, considering how popular nodejs is how can
its webserver hosting support be so limited? How do people use it?
When working with a virtual server you have full rain over what your running on the server.
Pros
Freedom, you get to pick all the software you want to run on your machine. A lot of the times when working with nodejs you're going to want some custom software to be running along side your application. Most of the time this is your database layer, which ever you choose.
Cons
YOU have to maintain it. Like #Roest stated, this isn't much of a con for most people as this ties directly into the freedom a virtual server gives you but it is something you need to take into account.
I think the reason you see limited support for nodejs is because its relatively new, and its so easy to setup yourself.
I want to use websockets and I found socket.io to be really popular
but I'm not sure exactly what it is. Correct me if I'm wrong, but, is
it just like writing a server in javascript and you run the javascript
file with the node.exe application and then the server is running?
That's pretty much exactly what nodejs is, or at least how you use it. Nodejs itself is Google's V8 javascript engine running on your server, along with a large number of libraries and C bindings that allow you to interact with your server in a way that the V8 engine won't let you.
This is an example of a webserver in nodejs (A very limited one)
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
It just responses Hello World to every request and always returns a 200 status code.
Going from something like this to a simple file server is fairly easy and quick, but a few people have already tackled this problem for you.
http://expressjs.com/ - Very powerful web server, but still gives you a lot of freedoms.
https://github.com/nodeapps/http-server - Simple web server, I use it mainly as a command line tool to instantly server files over http.
Lastly, is socket.io just an extension of nodejs? Do you have to code
your server in javascript when you use socket.io? Again, sorry for the
very novice questions but I'm just trying to understand a few basic
things before I continue. Thanks.
socket.io among many others is a module of nodejs. Depending on your definition of extension it may be the wrong word to use. Most of the time when using socket.io you'r going to be using an existing http server and then extending, or wrapping your server with socket.io. I wrote a previous explanation of how nowjs does this. My guess is socket.io is very similar.
To answer the bulk of that question: Yes, you will still be writing your code in javascript. You will just be utilizing the socket.io API.

#travis has already covered everything you need to know about node and socket.io
I'd only like to say that you don't have to buy a special hosting dedicated for node.
My game is hosted on VPS with Ubuntu
I find it really easy to deploy and maintain. There is a package for Ubuntu and instalation takes literally four lines of copy/paste
https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager
ps: I am not using socket.io but einaros/ws library which I find much less overblown.

What are the main differences between APE and Node.js?

Could anyone that has used both share his/her experience? What are the main differences and which one do you prefer? Thank you.

Different socket.io vs APE:
socket.io is coded in javascript(node.js) while APE is coded in C. I believe that is a big difference when you want to contribute. Maybe because you like the project or maybe because you want some more functionality. I think it will be easier to contribute Socket.io because you program Javascript, which is easier to grasp according to a lot of people(I Agree, although C is also very cool language).
I believe socket.io supports a lot more browsers/transport compared to APE, but I am not sure. Socket.io information vs information from APE page:
APE Server is an Comet server
implementing the POST and GET methods
of the HTTP protocol. It does not
replace a regular Web Server (such as
Apache, Lighttpd or Nginx), however,
the APE Server is only used for AJAX
Push.
So I guess APE supports less transports then socket.io.
Like Raynos said it is difficult to compare those two products and I believe you should play with them both and then decide which one you like more.
Same Socket.io/APE:
You can both code in Javascript to communicate with the server. I think you will have more freedom using socket.io because everything is exposed via Javascript.
Experience:
I only have experience with socket.io and I like it a lot.

Having spent some time developing with Node.js, I can't say that APE appears to be better than node. Based on sheer popularity, it seems that Node.js is probably the developer's choice -- and node.js appears to be more versatile as well.
Node essentially is you making a full on HTTP or TCP/IP. So, all of the mime type handling, data buffering, response headers, and server side routing are all things you'll have to do with your code. Node does streaming as well. I'm not sure about whether this is considered less problematic than normal ajax long polling at this point.
After googling around, I've found that people consider APE to be more of just a plain ol' push server, in which comet functionality is already there to be consumed, rather than Node, which would have you write your own. Don't be afraid of the prospect of writing things out with node though, they've got a very thorough documentation, and their methods are very easy to learn. I had some serious functionality written out in minutes.
Check this out also: http://groups.google.com/group/nodejs/browse_thread/thread/9d9b301479851b1f?pli=1

I played a little bit with node.js, tried out socket.io - but in the end did a big project with APE.
I think, as always, there is the question of what you want to achieve.
Only comparing the server parts: With node.js you get a machine that won't do anything on it's own, you need to write it yourself (or use libraries)
With APE, the handling of channels and connections is already built in (compiled C). Still you need to build parts of your own logic on top with JS - or use the examples.
On the client side, socket.io provides a client framework with three commands - and APE has it's APE_JSF that handles the connections (which brings more functionality than socket.io regarding channels)
Personally, I prefer APE, even though there is a lack of documentation for beginners.
However, keep in mind that APE won't deliver files/images, it's not a full Web-Server but optimized for real time push where it can handle ~10K concurrent users