We Keep Coding

Node.js serving a HTML file on a remote network

I'm a Node.js newb, so I'm not sure if it's possible to do what I'm trying to accomplish. Here is my configuration:
I have a Node.js server running on a Raspberry Pi on my home network. I have an HTML file on an external IP. I wish to render React components on this HTML file.
From what I've seen, people have used Node servers to display js on a local HTML file, so the path would normally look like this, assuming the directory is called 'dir':
const express = require('express');
const app = express();
// Static Routes
app.use('/dist', express.static(path.join(__dirname, 'dir')));
// App Route
app.get('/', (req, res, next) => res.sendFile(path.join(__dirname,
'index.html')));
Which, to my knowledge, transfers the HTML file found at the previously specified path. But where is this transferred to? Any way I can specify the path name based on an external IP + user + password, and file path?

Which, to my knowledge, transfers the HTML file found at the previously specified path. But where is this transferred to?
It transfers the file contents from disk to the client's network connection, when they request it (i.e. not when the server starts up).
Any way I can specify the path name based on an external IP + user + password, and file path?
Not with express.static. If you want to make your server serve an external page (proxying), you can do that: https://stackoverflow.com/a/10435819/7011366. Since you'll have access to the url & cookies, you can do whatever you like with the path/user/password.
app.post('/my_url', function(req, res) {
var options = {
host: MY_DOMAIN,
path: '/' + req.query.username,
method: 'GET',
};
var req = http.request(options, function(res) {
// etc...
// send response to client
}).on('error', function(e) {
// handle error...
}).end();
});
This above example makes a request to the external page every on every request. If you don't want that, you can store it in memory and only update it periodically.
let myHtml = "";
let fn = () => {
var options = {
host: MY_DOMAIN,
path: '/' + req.query.username,
method: 'GET',
};
var req = http.request(options, function(res) {
// etc...
// save response to myHtml as string
}).on('error', function(e) {
// handle error...
}).end();
};
setInterval(fn, 60000);
fn();
app.post('/my_url', function(req, res) {
res.end(myHtml);
});

You can not achieve it directly for the files stored on remote machine. Express static path works with local file system only.
Possible way for this could be, fetch file from the remote machine every time you get a new request. But no one does this.

Multer-s3 Uploading empty files on mobile

I'm using angular and multer-s3 to upload files from an angular app to a node server. Everything works well on the desktop but for some reason when trying to upload the photo via my iPhone 7 the uploaded file is corrupt. I'm using the same image and running through the same flow on both devices but getting different results so I'm assuming its because of mobile?
Here's the alert I get when trying to open the S3 file on the mobile
The file “1519398514215-test.png” could not be opened because it is empty.
Here's my code
var aws = require('aws-sdk');
var path = require('path');
var path3 = path.join(__dirname, "../config/config-aws.json");
var multer = require('multer');
var multerS3 = require('multer-s3');
var request = require('request');
aws.config.loadFromPath(path3);
var s3 = new aws.S3();
var fileName = '';
var uploadM = multer({
storage: multerS3({
s3: s3,
bucket: 'XXXX',
acl: 'public-read',
metadata: function (req, file, cb) {
cb(null, {fieldName: file.fieldname + '.png'});
},
key: function (req, file, cb) {
fileName = Date.now().toString() + "-" + file.originalname + '.png' ;
cb(null, fileName)
}
})
});
router.post('/', uploadM.array('photos', 3), function(req,res) {
if (res.error) {
console.log(error.stack);
return res.status(400).json({
message: "Error",
error: res.error
});
}
const url = 'https://s3-us-west-2.amazonaws.com/XXXX/' + fileName;
return res.status(200).json({
fileName: url
});
});
And here's my client-side
sendImage() {
const formData: FormData = new FormData();
this.removeObjectFromCanvas('polygon');
if (!fabric.Canvas.supports('toDataURL')) {
alert('This browser doesn\'t provide means to serialize canvas to an image');
} else {
// window.open(this.canvas.toDataURL('png'));
const image = new Image();
image.src = this.canvas.toDataURL('png');
const blob = this.dataURItoBlob(image.src);
const file = new File([blob], 'test.png');
formData.append('photos', file, 'test');
this.postFile(formData);
}
}
postFile(file) {
this.fileService.post(file)
.subscribe(data => {
}, error => {
console.log(error);
});
}
UPDATE **********
So found out you can debug on mobile. It looks like the buffer I am sending has data in it. My first thought was the buffer was not sending.
**** Update
Still can't figure this out. I've done some research and its possible it has something to do with formData and append? But as you can see by the image above both seem to be fine. Will continue to research ...
***** UPDATE
Definitely uploading empty files. But its only on mobile?
Also, I checked the formData prior to sending to the node server, seems to have the correct data in it.
*** UPDATE
Ok, even weirder experience. It seems multer-s3 is uploading empty files but when I take the file on the server-side and return it to the client-side, then read that file and display it, the image is displayed perfectly. So the formData is not the issue, it's something with multer-s3 I'm assuming?
****UPDATE
I forgot to mention I am using fabricjs and getting the image from the canvas. I read in some places there may be an issue there but like I said above when I send the file to the server and send it back to the client, after reading the file it displays the image perfectly.
****Update
I tried adding contentType to the multer method and now I'm receiving a 503 service unavailable error when running on mobile only. For desktop it is fine.
aws.config.loadFromPath(path3);
var file1;
var s3 = new aws.S3();
var fileName = '';
var uploadM = multer({
storage: multerS3({
s3: s3,
bucket: 'rent-z',
acl: 'public-read',
contentType: function (req, file, cb) {
cb(null, 'image/png');
},
metadata: function (req, file, cb) {
console.log(file);
cb(null, {fieldName: file.fieldname});
},
key: function (req, file, cb) {
fileName = Date.now().toString() + "-" + file.originalname;
file1 = file;
cb(null, fileName)
}
})
}).array('photos', 1);
router.post('/', function(req,res) {
uploadM(req, res, function (err) {
if (err) {
console.log(err);
return res.status(400).json({
message: "Error uploading to multer",
error: err
});
}
console.log('worked');
if (res.error) {
console.log(error.stack);
return res.status(400).json({
message: "Error",
error: res.error
});
}
// fs.readFile(req.body, function (err, data) {
const url = 'https://s3-us-west-2.amazonaws.com/rent-z/' + fileName;
return res.status(200).json({
fileName: url
});
// });
})
});
I even tried running multer-s3's automatic find mime-type function and that did give the same result
**** Day 4
It's been 96 hours since I started debugging this problem. No progress has been made. Still trying to figure out why its working on desktop and not mobile. For anyone looking for a quick summary of behavior:
User uploads image on the desktop
User places image on canvas
Use scales image
User presses sendImage
This converts the image to dataUri then blob
This blob is added to a file which is appended to formData
This formData is sent to a nodejs server where multer-s3 middleware
uploads the file to s3 successfully
User tries on mobile
Fails at step 7. The file is uploaded but is empty.
Let me know if anyone has any ideas on how to continue.

I'll make this an "official" answer since this may work for your needs. Anytime I have an intricate issue like this, my first thought is often "I wonder if there is an API/SaaS/service out there that can abstract this for me." As you've found, file uploads are tricky, particularly when you start throwing in the myriad devices we have to deal with these days.
I won't mention any particular services, but googling "file upload saas" will generally get you the top industry players. For $25 - $50/month you can abstract file uploads to a very simple api call. Not only do you get time savings now, but (assuming you choose a solid provider) you get no more headaches regarding file uploads in the future. It's the SaaS's job to make sure file uploads work on a million different devices; it's the SaaS's job to make sure S3 integration works, even when S3's api changes; it's the SaaS's job to make sure the user sees a nice friendly message if their upload fails for some reason, etc. I get to spend my time building features for our app instead of worrying about whether or not file uploads work on the iPhone 47.
"But then I'm tied to a SaaS, and live at the whim of their prices and feature set" Ah, but you can minimize that problem. For many services we use, I like to make a wrapper/interface/whatever you'd like to call it. In the case of file uploads, I made an ES6 module: fileUploads.js
In this module, I have a method upload. What does this method do? It simply implements and abstracts the API of [fileupload SaaS X]. If in the future we want, or need, to change from SaaS X to SaaS Y, I only have to change one thing in our entire app: my fileUpload.js module.

What is client, path and data in this code?

I'm learning Node.js I have created server and client .js files but I don't understand few things. For example, in the webserver.js file, I don't know what is the use of pathname. Similarly, in the client.js file, what are dataand path?
If you think I should read about the basics of it, please provide me a useful link if you can. I tried to find but didn't work.
webserver.js
var fs=require('fs');
var url=require('url');
var http=require('http');
http.createServer(function(request, response){
var pathname=url.parse(request.url).pathname;
console.log("Pathname: "+pathname+"Request.url: "+request.url);
fs.readFile(pathname.substr(1), function(err, data){
if(err){
console.log("Error reading.");
response.writeHead(400, {'content-type' : 'text/html'});
}else{
response.writeHead(200, {'content-type' : 'text/html'});
response.write(data.toString());
}
response.end();
});
}).listen(8081);
console.log("Server is running.");
client.js
var http=require('http');
var options={
host: 'localhost',
port: '8081',
path: '/index.html'
};
var callback=function(response){
var body='';
response.on('data', function(data){
body+=data;
});
response.on('end', function(){
console.log("Data received.");
});
}
var req=http.request(options, callback);
req.end();
The original code souce is here: Code

pathname is the path section of the URL, that comes after the host and before the query, including the initial slash if present.

pathname is the path requested to the http server. The example pathname for this question is /questions/40276802/what-is-client-path-and-data-in-this-code. The path in client.js is the same deal.
You can find documentation on parsing the URL into the pathname from the Node.js docs: https://nodejs.org/api/http.html#http_message_url
Node's HTTP client uses streams, which emit several events. data is called with a buffer, which you usually will add to an array then concat later (as the code does). end is called when all buffers are sent.
You can find documentation on handling events from streams from the Node.js docs: https://nodejs.org/api/stream.html#stream_class_stream_readable

pathname is the requested path. You should check the documentation for the url package: npm-url
In your client.js: data is the response data from the server. Again check http documentation: HTTP|Node.js
For learning callbacks and all about Node.js: nodeschool.io

Download a file from REST API with Restangular request

I've got a simple node.js + Restify backend with standard CORS settings and this endpoint:
var file = '1,5,8,11,12,13,176,567,9483';
server.get('/download', function(req, res, next) {
res.set({"Content-Disposition": "attachment; filename='numbers.csv'"});
res.setHeader("Content-type", "text/csv");
res.send(file);
return next();
}, function(err) {
res.send(err);
});
What it's suppose to do is to is to create CSV file and return it.
It works great when I simply type in the endpoint address to web browser and hit enter. The file gets downloaded properly.
But when I try to do the same thing, but instead of using browser's address bar I use Restangular like that:
Restangular.one('download').get().then(function (res) {
console.log(res);
});
it just writes response to console, but no file is being downloaded.
Is there a way to do this using Restangular? Or maybe I need to use something else for this?

I am not sure if Restangular can do that, but I am using FileSaver script for stuff like that. Add Filesaver to your HTML head and then:
Restangular.one('download').get().then(function (res) {
var file = new Blob([res], { type: 'text/csv' });
saveAs(file, 'something.csv');
});

Meteor.js - Serve image directly from MongoDB?

Using Meteor.js, how can I serve an arbitrary HTTP response, eg. an image or PDF?
Example 1 - I need to generate PDF reports, which I cannot store in public/ or on a third-party server. Or, the report may be generated live in response to a HTTP GET.
Example 2 - If I have a url like:
/images/myimage.png
I would like to detect that request on the server, read the image from MongoDB, and serve it with the correct headers, so it is available to use with img tags, ie.
<img src="/images/myimage.png">
I do not want to store the images in the /public/ directory, so that I can have more control over exactly what is served and how it is permissioned.
Edit I was also able to get a basic example working using Iron Router.
ImageController = RouteController.extend({
run: function() {
var f = fs.readFileSync("/path/to/image.png");
var res = this.response;
res.writeHead(200, { "content-type": "image/png" });
res.write(f);
res.end();
}
});
Router.map(function() {
Router.route("images", {
path: "/images/image.png",
where: "server",
controller: ImageController // Note - cannot use string here - Iron Router has a dependency on window
});
});

You may write the response code as in any node app, using the middleware:
WebApp.connectHandlers.stack.splice (0, 0, {
route: '/path/to/the/file',
handle: function(req, res, next) {
res.writeHead(200, {
'Content-Type': ...ITEM TYPE... ,
});
res.write( ...ITEM DATA... );
res.end();
},
});

You can use filepicker. In filepicker the upload images is save in the bucket(cloud) and returns the url of that image. You can save the url in your mongo database. and when you want to use that image just use <img src="{{saveurl}}" > .
For more help see the documentation https://developers.inkfilepicker.com/docs/web/