I am fairly new to Node.js and I want to load a few javascript files in the browser but I can't seem to find anywhere how I can simply get libraries to be loaded inside the browser.
So, my question is, how would I be able to load javascript libraries client side with Node.js?
I'm fairly new to node as well, but I've been using browserify to deal with node packages. There are a quite a few node packages that won't work in a browser - if they will work in a browser they will usually say it in their description.
Browserify is really easy to use - you just require() a module just as you would when writing a non-browser script (as far as I can tell). When you're ready to test the script on a browser, you just run
browserify input.js -o output.js
and it pulls all of the dependencies from your require()'s into output.js, so then your require()s actually reference what you want it to.
--Updated for new comment--
As I see in your answer, you are talking about node.js packages? I am
talking about standalone random javascript libraries (so, for example
just one of my own js files that I want to load client side). Can you
also load random javascript libraries with browserify? (I googled a
bit but can't figure that out)
Yeah, you can do that; although I'm afraid I can't help you with the technicalities of it (like I said, new to it myself). Here's an example:
Bar.js (my library)
module.exports = bar; //tells node what to export in this file
var bar = {
message: function(msg) {
console.log(msg)
}
}
Foo.js (my script)
var bar = require('./bar');
bar.message("foo"); //logs "foo" to console
And keep in mind that require() refers to a file. So Bar.js would need to be in the same directory as Foo.js. If Bar.js was in a different folder, you would just do require('./folder/bar'). To tie it all together, you would run browserify on Foo.js, and it would automatically grab Bar.js' contents.
There are a lot of different ways you can define exports, so you'll need to google around to see exactly how to do it for how your lib is formatted. But that's the gist of it.
Related
First things first. I know that there are other questions that are similar to this e.g. use NodeJs Buffer class from client side or
How can I use node.js buffer library in client side javascript
However, I don't understand how to make use of the reference to use browserify though it is given approval.
Here is my Node code:
import { Buffer } from 'buffer/';
I know this is the ES6 equivalent of require.
I would like a javaScript file implementation of this module so that I can simply use the standard html file reference:
<script src=./js/buffer.js></script>
And then use it as in for example
return new Buffer(temp).toString('utf-8');
This simply falls over with the
Uncaught ReferenceError: Buffer is not defined
no matter how I create the buffer.js file.
So using the browserify idea I've tried using the standalone script (from the https://www.npmjs.com/package/buffer as https://bundle.run/buffer#6.0.3 )
I've created a test.js file and put
var Buffer = require('buffer/').Buffer
in it and then run browserify on it as
browserify test.js -o buffer.js
and many other variations.
I'm not getting anywhere. I know I must be doing something silly that reflects my ignorance. Maybe you can help educate me please.
These instructions worked for me. Cheers!
Here are the instructions you can look at the web section.
https://github.com/feross/buffer
Here is what the instructions state about using it in the browser without browserify. So from what you tried
browserify test.js -o buffer.js
I would just use the version directly that does not require browserify
To use this module directly (without browserify), install it:
npm install buffer
To depend on this module explicitly (without browserify), require it like this:
var Buffer = require('buffer/').Buffer // note: the trailing slash is important!
I don't like the whole export/require stuff in node, it takes too long. Let's say I have a file server.js and I want to use functions in whatever.js. in html I just add this to the header:
<script src='whatever.js'></script>
and then I can just use all the functions of whatever.js in my body's script.
But in node, in the server.js file I'd do:
var myobject = require('./whatever.js');
but then I need to set it to myobject, and further I need to go to whatever.js and manually decide what functions I want to export. not to mention that typing myobject.someFunction() is alot longer to write than someFunction() and I need to remember what I exposed/didn't expose.
I wanted something where I could just go:
require('./whatever.js');
and it puts it ALL in global, no bs. like in good old html/javascript. Is there a way to do this in node?
This will do the trick,
var fs = require('fs');
eval(fs.readFileSync('whatever.js')+'');
// here call functions from whatever.js file
(I realize this is an old thread but wanted to leave a note here for posterity.)
Here in 2022 there are several approaches for executing code from different files with Node.js:
ESM: Use standard ECMAScript modules
At the time of this writing, much of the node ecosystem (i.e. packages on npm) is in the process of transitioning to this paradigm, and there are some associated growing pains (e.g. things like __dirname are only available in CJS not ESM, though the workaround is easy).
For most developers, it would be advisable to become comfortable with this standard as it transcends node.js (i.e. is implemented in other runtimes like Deno and web browsers) and has been years in the making.
CJS: Use the original "CommonJS" module mechanism, e.g. require('./some-script.js')
It should be noted, particularly for the OP, that even though the "intended" way to use CJS modules is to export functions, constants, etc. and import them explicitly, it is possible to define everything in global scope using globalThis, though I would not recommend this.
// my-script.js
require('./foo.js');
require('./bar.js');
foo(); // This is foo from <...>foo.js
console.log(`bar = ${bar} (in ${__filename})`); // bar = 123 (in <...>my-script.js)
// foo.js
globalThis.foo = function() {
console.log(`This is foo from ${__filename}`);
}
// bar.js
globalThis.bar = 123;
If you try omitting globalThis. you'll find that foo and bar are no longer defined in the main script because require "wraps them" in "module scope."
Use eval
In my experience, there are very few legitimate use cases for eval (see Never use eval()!). Nevertheless, the functionality requested in this question is precisely what eval provides: "run some code as if it were written right here" and you can feed it from a file, as explained above by Mehul Prajapati
// include.js
// Defines a global function that works like C's "#include" preprocessor directive
const { readFileSync } = require('fs');
globalThis.include = function(scriptFile) {
console.warn('!!! EXTREMELY INSECURE !!!');
eval(readFileSync(scriptFile, 'utf-8'));
};
// main.js
require('./include.js'); // loads global include
// (I sure hope you completely trust these sources)
include('./foo.js');
include('./bar.js');
Note: Something that has contributed to much of my confusion in the past is that there have been competing standards/conventions/APIs that use some of the same identifiers, namely require, which require.js and other bundlers that support AMD (Asynchronous Module Definition)
use with different semantics. So for someone building a web application (using AMD for modules in web browsers) with node.js tooling (using CJS for modules locally) it can be frustrating to keep the functions straight, especially if it's an Electron application, which can expose Node.js APIs to scripts running in the renderer (browser). If you find yourself confused why a module is "not found" in a situation like that, check the stack trace to see which require is being called (and you may have to wrap/rename them on globalThis or something to avoid collisions).
Further reading:
JavaScript Modules: A Brief History [2019]
How the module system, CommonJS & require works [updated 2022]
What is AMD, CommonJS, and UMD? [2014]
Let's say I need CasperJS to report progress steps to a localhost server. I couldn't user casper.open to send a POST request because it'd "switch" pages so to speak and would be unable to continue other steps properly.
I sidestepped this issue by evaluating an XMLHttpRequest() inside the browser to ping to localhost. Not ideal but it works.
As the number of the scripts grow, I'd rather move this common functionality into a module, which is to say, I want to move a number of functions into a separate module.
It's my understanding CasperJS doesn't work like node.js does so require() rules are different. How do I go about accomplishing this?
Since CasperJS is based on PhantomJS you can use its module system, which is "modelled after CommonJS Modules 1.1"
You can require the module file by its path, full or relative.
var tools = require("./tools.js");
var tools = require("./lib/utils/tools.js");
var tools = require("/home/scraping/project/lib/utils/tools.js");
Or you can follow node.js convention and create a subfolder node_modules/module_name in your project's folder, and place module's code into index.js file. It would then reside in this path:
./node_modules/tools/index.js
After that require it in CasperJS script:
var tools = require("tools");
Module would export its functions in this way:
function test(){
console.log("This is test");
}
module.exports = {
test: test
};
I want to minimize the number of HTTP requests from the client to load scripts in the browser. This is going to be a pretty general question but I still hope I can get some answers because module management in javascript has been a pain so far.
Current situation
Right now, in development, each module is requested individually from the main html template, like this:
<script src="/libraries/jquery.js"></script>
<script src="/controllers/controllername.js"></script>
...
The server runs on Node.js and sends the scripts as they are requested.
Obviously this is the least optimal way of doing so, since all the models, collections, etc. are also separated into their own files which translates into numerous different requests.
As far as research goes
The libraries I have come across (RequireJS using AMD and CommonJS) can request modules from within the main .js file sent to the client, but require a lot of additional work to make each module compliant with each library:
;(function(factory){
if (typeof define === 'function' && define.amd) define([], factory);
else factory();
}(function(){
// Module code
exports = moduleName;
}));
My goal
I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect. Then I can simply serve that single file to the client when it is requested.
Is this possible?
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?
What you are asking to do, from what I can tell, is concat your js files into one file and then in your main.html you would have this
<script src="/pathLocation/allMyJSFiles.js"></script>
If my assumption is correct, then the answer would be to use one of the two following items
GULP link or GRUNT link
I use GULP.
You can either use gulp on a case by case basis, which means calling gulp from the command line to execute gulp code, or use a watch to do it automatically on save.
Besides getting gulp to work and including the gulp files you need to do what you need, I will only provide a little of what I use to get your answer.
In my gulp file I would have something like this
var gulp = require('gulp');
var concat = require('gulp-concat');
...maybe more.
Then I have the file paths I need to be reduced into one file.
var onlyProductionJS = [
'public/application.js',
'public/directives/**/*.js',
'public/controllers/**/*.js',
'public/factories/**/*.js',
'public/filters/**/*.js',
'public/services/**/*.js',
'public/routes.js'
];
and I use this info in a gulp task like the one below
gulp.task('makeOneFileToRuleThemAll', function(){
return gulp.src(onlyProductionJS)
.pipe(concat('weHaveTheRing.js'))
.pipe(gulp.dest('public/'));
});
I then run the task in my command line by calling
gulp makeOneFileToRuleThemAll
This call runs the associated gulp task which uses 'gulp-concat' to get all the files together into one new file called 'weHaveTheRing.js' and creates that file in the destination 'public/'
Then just include that new file into your main.html
<script src="/pathLocation/weHaveTheRing.js"></script>
As for including all your files into one file, including your vendor files, just make sure that your vendor code runs first. It's probably best to keep those separate unless you have a sure fire way of getting your vendor code to load first without any issues.
UPDATE
Here is my gulp watch task.
gulp.task('startTheWatchingEye', function () {
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll']);
});
Then I start up my server like this (yours may differ)
npm start
// in a different terminal window I then type
gulp startTheWatchfuleye
NOTE: you can use ANY movie or show reference you wish! :)
Now just code it up, every time you make a change in the specified files GULP will run your task(s).
If you want to say run Karma for your test runner...
add the following to your gulp file
var karma = require('karma').server;
gulp.task('karma', function(done){
karma.start({
configFile: __dirname + '/karma.conf.js'
}, done);
});
Then add this task karma to your watch I stated above like this...
gulp.task('startTheWatchingEye', function(){
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll', 'karma']);
});
ALSO
Your specific settings may require a few more gulp modules. Usually, you install Gulp globally, as well as each module. Then use them in your various projects. Just make sure that your project's package.json has the gulp modules you need in dev or whatever.
There are different articles on whether to use Gulp or Grunt. Gulp was made after Grunt with a few additions that Grunt was lacking. I don't know if Grunt lacks them anymore. I like Gulp a lot though and find it very useful with a lot of documentation.
Good luck!
I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect.
Sure you can. You can use Grunt or Gulp to do that, more specifically grunt-contrib-concat or gulp-concat
Here's an example of a Gruntfile.js configuration to concat every file under a js directory:
grunt.initConfig({
concat: {
dist: {
files: {
'dist/built.js': ['js/**/**.js'],
},
},
},
});
Also, you can minify everything after concatenating, using grunt-contrib-minify.
Both libraries support source maps so, in the case a bug gets to production, you can easily debug.
You can also minify your HTML files using grunt-contrib-htmlmin.
There's also an extremely useful library called grunt-usemin. Usemin let's you use HTML comments to "control" which files get minified (so you don't have to manually add them).
The drawback is that you have to explicitely include them in your HTML via script tags, so no async loading via javascript (with RequireJS for instance).
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?
That's debatable. Both have pros and cons. Concatenating vendors assures that, if for some reason, the CDN isn't available, your page works as intended. However the file served is bigger so you consume more bandwidth.
In my personal experience, I tend to include vendor libraries that are absolutely essential for the page to run such as AngularJS for instance.
If I understand you correctly, you could use a task runner such as Grunt to concatenate the files for you.
Have a look at the Grunt Concat plugin.
Example configuration from the docs:
// Project configuration.
grunt.initConfig({
concat: {
dist: {
src: ['src/intro.js', 'src/project.js', 'src/outro.js'],
dest: 'dist/built.js',
}
}
});
Otherwise, as you have stated, a 'module loader' system such as Require JS or Browserify may be the way to go.
I've come across an interesting problem in NodeJS that pops up now and then at work, and I haven't been able to figure it out.
I've written a back-end in CoffeeScript, which I then compile with grunt-contrib-coffee into Javascript in a ~/bin directory. I also include a library that I privately host on Bitbucket with the appropriate private keys, and install through npm. This library too is written in coffeescript.
Usually I'm able to include this library in Javascript without any headaches, using a simple require just like I would for any other library. However, occasionally one of the servers that's using the back-end gets updates, and it stops working. When I go check the error, it's always the same - 'require' passes, but instead of loading the actual library in JavaScript, it returns an empty object ({}). Running the code in coffeescript still works, but regardless of what I do - recompile, reinstall all dependencies, remove and clone the repository, the problem persists.
I've run out of ideas of what it might be myself, so I'm hoping that someone here has come across the problem before and might be able to point me in the right direction.
In the library package.json:
{
"name": "graph-engine",
"main": "./lib/graph"
}
In the library's graph.coffee
class Graph
constructor: () ->
# Perform init
module.exports = Graph
Then in the app's package.json:
{
"graph-engine": "git+ssh://git#bitbucket.org:<team>/graph-engine.git"
}
Finally, in the app itself:
GraphEngine = require "graph-engine"
engineInstance = new GraphEngine()
This works fine in coffeescript, but when compiling the app using grunt with the following setup for grunt-contrib-coffee:
coffee:
glob_to_multiple:
expand: true
flatten: false
cwd: 'src'
src: ['**/*.coffee']
dest: 'bin'
ext: '.js'
It fails to load the library correctly when running the compiled application, instead returning an empty object. Again, I'd like to emphasise that this does not always happen, and as such I didn't include any code or json files as I believed that it was unrelated.
Although the exact reason for the randomness of the behaviour eludes me to this day, I have found that it is related to these libraries being written in CoffeeScript.
It seems that depending on the load order, occasionally third party libraries would register coffee script and my own libraries would load correctly, whereas other times these libraries would load after my own libraries had loaded, resulting in an inability to load coffeescript. Therefore, the solution turned out to be fairly straightforward - register coffeescript. This can be done by putting the following at the start of the node.js app:
require('coffee-script/register')
Refer to the documentation for further information.