We Keep Coding

How to import Kotlin/JS generated module in a separate npm-based project

I would like to create a js library with Kotlin Multiplatform (an example of which is this project, where we have a webscocket server and a js client) which I will then build as a npm package and import in my Vue project (could be any other framework).
what I managed to do with chat project is:
build js sources with ./gradlew build
publish that via yarn publish (setting up remote registry url ofc)
add published package to package.json with (needed to update project name to #chat/client by hand in the generated package.json):
{
"name": "#chat/client",
"version": "0.0.1",
"private": false,
"workspaces": [
"packages/chat-frontend",
"packages/chat-frontend-test",
"packages_imported/kotlin/1.6.21",
"packages_imported/ktor-ktor-client-core-js-ir/2.0.0",
"packages_imported/kotlin-test-js-runner/1.6.21"
],
"resolutions": {},
"devDependencies": {},
"dependencies": {},
"peerDependencies": {},
"optionalDependencies": {},
"bundledDependencies": []
}
added #JsExport annotation on writeMessage in src/frontendMain/kotlin/main.kt
what I didn't manage is (in my Vue project):
import writeMessage, I exported in .kt file (it's visible in source, not exported though)
import anything via import * from '#chat/client'
or any other folder along '#chat/client/*'
use the generated files in any other way
The generated package structure is very odd:
~ ls build/js/*
build/js/package.json build/js/yarn.lock
build/js/node_modules:
... (npm dependencies from Kotlin/JS module)
build/js/packages:
chat-frontend chat-frontend-test
build/js/packages_imported:
... (Kotlin/JS dependencies)
~ ls build/js/packages/chat-frontend/*
build/js/packages/chat-frontend/package.json build/js/packages/chat-frontend/webpack.config.js
build/js/packages/chat-frontend/kotlin:
chat-frontend chat-frontend.js chat-frontend.js.map chat-frontend.meta.js
(chat-frontend contains package dir tree and a file frontend.kjsm)
build/js/packages/chat-frontend/kotlin-dce:
chat-frontend.js
ktor-*.js
kotlinx-*.js
... (compiler output ???)
build/js/packages/chat-frontend/node_modules:
... (webpack cli and dev-server files)
Do you have any clues, tips or an example project which does that? I've processed whole section of Kotlin/JS docs but there is no information on how to import Kotlin generated .js files in a js/ts project.
EDIT:
I've updated my fork of ktor-samples with Kotlin/JS build files: build-js folder and src/backendMain/resources/chat.js. Here's the link to chat folder of the fork project

I will try to help
Kotlin/JS has 2 kinds of compiler: legacy and IR. #JsExport affects only IR compiler. But from kotlin-dce folder, you use legacy compiler backend. In IR compiler, DCE (dead code elimination) is included into compiler and there is no folder kotlin-dce. You can change compiler kind in gradle.properties with kotlin.js.compiler=ir|legacy.
When you build project with IR, packages/*/kotlin will fully contain your library (similar with legacy's kotlin-dce)
Then you need to prepare appropriate package.json file with name and main fields (task :publicPackageJson could help with that, but check, if it is ok for your case)
Now in Kotlin/JS export works with packages. It means, that if you export io.ktor.samples.chat.frontend.writeMessage, it will be exported as io.ktor.samples.chat.frontend.writeMessage in js. So to import it, you need import io and then find necessary declarations.
import { io } from '#chat/client`
io.ktor.samples.chat.frontend.writeMessage("hello")

In the case of classical (non-IR) js backend usage and you use only non-private NPM dependencies then the only thing that you need is to compile your Kotlin/js library (via :compileKotlinJs task), generate package.json for your library (via :publicPackageJson) and your package could be published to NPM or another private repository (by yarn publish).
After publication, you will have the ability to set your library as a dependency in any js project. (Note that the name and version of your library will be placed inside package.json (generated by :publicPackageJson task)).
Also, you can check this discussion. Hope it will help you.

Ember.js: How to share package JSON file dependency to child engines and addons

Basically I need to know how Ember Js can share parent app package.json file dependency(xyz:3.0.0) to child engine and addons without being used again in child engine and addon package.json file. So that I can reduce the size of the application.
As of now in our application we installing common package dependency in all our child engines and addons even though we used in parent app its increases application size.
Here is my clear example of my project scenario.
parentApp(xxx):
Which has package.json file containes few dependency like ex: vendor-package1:10.0.0, vendor-package2:4.0.0, Child Engine1(yyy), Child Engine2(zzz)
Child Engine1(yyy)
Which has package.json file containes few dependency like ex: vendor-package1:10.0.0, vendor-package2:4.0.0
Child Engine2(zzz)
Which has package.json file containes few dependency like ex: vendor-package1:10.0.0, vendor-package2:4.0.0
So if you notice parent app and child engines has same dependency(vendor-package1:10.0.0, vendor-package2:4.0.0) which I need to do npm install for all three app. I'm adding (vendor-package:10.0.0, vendor-package2:4.0.0) to all my child engines because it should available to my engines.
Because of this my dist folder has (vendor-package1:10.0.0, vendor-package2:4.0.0) to all my parentApp and engines, which increase in size.
If I add(vendor-package1:10.0.0, vendor-package2:4.0.0) only to my parentApp(xxx) then my child engines cannot access those component inside vendor-package1 and vendor-package2.
Please suggest some solution where I don't want to add dependency to all my apps.

I've setup a demo ember 3.12 app at https://github.com/bartocc/so-58343095.
This app depends on ember-concurrency and also has an in-repo addon core that depends on ember-concurrency.
I've also added ember-cli-bundlesize to help analyse the bundle size of the built app.
Here are the results of ember bundlesize:test before and after adding the in-repo addon.
Before
$ git checkout 6c5dfc7
$ ember bundlesize:test
ok 1 - app:javascript: 165.89KB <= 500KB (gzip)
ok 2 - app:css: 40B <= 50KB (gzip)
After
$ git checkout 9c9c9a9
$ ember bundlesize:test
ok 1 - app:javascript: 165.89KB <= 500KB (gzip)
ok 2 - app:css: 40B <= 50KB (gzip)
Bundlesize check was successful. Good job!
As you can see, the bundlesize does not change.
The same goes for an in-repo engine:
With in-repo engine depending on ember-concurrency
$ git checkout 2662b63
$ ember bundlesize:test
ok 1 - app:javascript: 170.08KB <= 500KB (gzip)
ok 2 - app:css: 40B <= 50KB (gzip)
Bundlesize check was successful. Good job!
The small difference you see between 165.89KB and 170.08KB is made of:
the ember-engines modules:
;define("ember-engines/-private/engine-ext")
;define("ember-engines/-private/engine-instance-ext")
;define("ember-engines/-private/route-ext")
;define("ember-engines/-private/router-ext")
;define("ember-engines/components/link-to-component")
;define("ember-engines/components/link-to-external-component")
;define("ember-engines/engine")
;define("ember-engines/initializers/engines")
;define("ember-engines/routes")
the my-engine modules:
;define("my-engine/config/environment")
;define("my-engine/engine")
;define("my-engine/resolver")
;define("my-engine/routes")
;define("my-engine/templates/application")
And finally the ember-concurrency modules aliased to be available inside the my-engine resolver:
;define.alias("ember-concurrency/helpers/cancel-all", "my-engine/helpers/cancel-all");
;define.alias("ember-concurrency/helpers/perform", "my-engine/helpers/perform");
;define.alias("ember-concurrency/helpers/task", "my-engine/helpers/task");
;define.alias("ember-concurrency/initializers/ember-concurrency", "my-engine/initializers/ember-concurrency");
YMMV depending on what addon you use though, but you can use this demo app as a starting point to check whether some code is really duplicated or not.
Hope this helps

Update in answer to updated question 11/12/19
The short answer is you cannot make each of your dependencies only part of your app's package.json. What you have where you specify each dependency in each addon and app's package.json is correct.
Npm will only install one version of each package. It does some complicated 'magic' to resolve which dependency gets used. If you think about it, this is the only thing that could happen because JS and the browser can only have on version of each library available since things are made available globally on the window. Your app would have now way of telling the difference between version x.x.x of a library and x.x.y of the same library because that library is always exposed with the same name globally, e.g., Ember.
Original answer
We have had some success reducing packages installed by using lerna and a mono repo: https://github.com/lerna/lerna. Note however, we have not had any success using the lerna commands. Instead we simply run npm i in each addon/app. The order npm i is run is critical: you must start with the base of your tree first i.e., start with the addon that does not consume an of your other addons/apps and move your way up.
Our mono repo contains three ember applications and two addons:
addon-1
addon-2
- consumes addon-1
app-1
- consumes addon-1
app-2
- consumes addon-1 and addon-2
app-3
- consumes addon-1 and addon-2
In the structure above, npm i must be run in this order: addon-1, addon-2 and app-1, app-2 and app-3.
We have experimented with different ways of including the packages in package.json of each addon/app. It's come down to this, example for app-2:
"devDependencies": {
"addon-1": "file:../addon-1",
"addon-2": "file:../addon-2",
},
file: allows you to reference a module in your mono repo using a relative path.

Huge number of files generated for every Angular project

I wanted to start a simple hello world app for Angular.
When I followed the instructions in the official quickstart the installation created 32,000 files in my project.
I figured this is some mistake or I missed something, so I decided to use angular-cli, but after setting up the project I counted 41,000 files.
Where did I go wrong? Am I missing something really really obvious?

There is nothing wrong with your configuration.
Angular (since version 2.0) uses npm modules and dependencies for development. That's the sole reason you are seeing such a huge number of files.
A basic setup of Angular contains transpiler, typings dependencies which are essential for development purposes only.
Once you are done with development, all you will need to do is to bundle this application.
After bundling your application, there will be only one bundle.js file which you can then deploy on your server.
'transpiler' is just a compiler, thanks #omninonsense for adding that.

Typical Angular2 Project
NPM Package Files (Development) Real World Files (Deployment)
#angular 3,236 1
rxJS 1,349 1*
core-js 1,341 2
typings 1,488 0
gulp 1,218 0
gulp-typescript 1,243 0
lite-server 5,654 0
systemjs-builder 6,470 0
__________________________________________________________________
Total 21,999 3
*: bundled with #angular
[ see this for bundling process ⇗ ]

There is nothing wrong with your development configuration.
Something wrong with your production configuration.
When you develop a "Angular 2 Project" or "Any Project Which is based on JS" you can use all files, you can try all files, you can import all files. But if you want to serve this project you need to COMBINE all structured files and get rid of useless files.
There are a lot of options for combine these files together:
YUI Compressor
Google Closure Compiler
For server side (I think it is best) GULP

As several people already mentioned: All files in your node_modules directory (NPM location for packages) are part of your project dependencies (So-called direct dependencies). As an addition to that, your dependencies can also have their own dependencies and so on, etc. (So-called transitive dependencies). Several ten thousand files are nothing special.
Because you are only allowed to upload 10'000 files (See comments), I would go with a bundler engine. This engine will bundle all your JavaScript, CSS, HTML, etc. and create a single bundle (or more if you specify them). Your index.html will load this bundle and that's it.
I am a fan of webpack, so my webpack solution will create an application bundle and a vendor bundle (For the full working application see here https://github.com/swaechter/project-collection/tree/master/web-angular2-example):
index.html
<!DOCTYPE html>
<html>
<head>
<base href="/">
<title>Webcms</title>
</head>
<body>
<webcms-application>Applikation wird geladen, bitte warten...</webcms-application>
<script type="text/javascript" src="vendor.bundle.js"></script>
<script type="text/javascript" src="main.bundle.js"></script>
</body>
</html>
webpack.config.js
var webpack = require("webpack");
var path = require('path');
var ProvidePlugin = require('webpack/lib/ProvidePlugin');
var CommonsChunkPlugin = require('webpack/lib/optimize/CommonsChunkPlugin');
var UglifyJsPlugin = require('webpack/lib/optimize/UglifyJsPlugin');
/*
* Configuration
*/
module.exports = {
devtool: 'source-map',
debug: true,
entry: {
'main': './app/main.ts'
},
// Bundle configuration
output: {
path: root('dist'),
filename: '[name].bundle.js',
sourceMapFilename: '[name].map',
chunkFilename: '[id].chunk.js'
},
// Include configuration
resolve: {
extensions: ['', '.ts', '.js', '.css', '.html']
},
// Module configuration
module: {
preLoaders: [
// Lint all TypeScript files
{test: /\.ts$/, loader: 'tslint-loader'}
],
loaders: [
// Include all TypeScript files
{test: /\.ts$/, loader: 'ts-loader'},
// Include all HTML files
{test: /\.html$/, loader: 'raw-loader'},
// Include all CSS files
{test: /\.css$/, loader: 'raw-loader'},
]
},
// Plugin configuration
plugins: [
// Bundle all third party libraries
new CommonsChunkPlugin({name: 'vendor', filename: 'vendor.bundle.js', minChunks: Infinity}),
// Uglify all bundles
new UglifyJsPlugin({compress: {warnings: false}}),
],
// Linter configuration
tslint: {
emitErrors: false,
failOnHint: false
}
};
// Helper functions
function root(args) {
args = Array.prototype.slice.call(arguments, 0);
return path.join.apply(path, [__dirname].concat(args));
}
Advantages:
Full build line (TS linting, compiling, minification, etc.)
3 files for deployment --> Only a few Http requests
Disadvantages:
Higher build time
Not the best solution for Http 2 projects (See disclaimer)
Disclaimer: This is a good solution for Http 1.*, because it minimizes the overhead for each Http request. You only have a request for your index.html and each bundle - but not for 100 - 200 files. At the moment, this is the way to go.
Http 2, on the other hand, tries to minimize the Http overhead, so it's based on a stream protocol. This stream is able to communicate in both direction (Client <--> Server) and as a reason of that, a more intelligent resource loading is possible (You only load the required files). The stream eliminates much of the Http overhead (Less Http round trips).
But it's the same as with IPv6: It will take a few years until people will really use Http 2

You need to ensure that you're just deploying the dist (short for distributable) folder from your project generated by the Angular CLI. This allows the tool to take your source code and it's dependencies and only give you what you need in order to run your application.
That being said there is/was an issue with the Angular CLI in regards to production builds via `ng build --prod
Yesterday (August 2, 2016) a release was done which switched the build mechanism from broccoli + systemjs to webpack which successfully handles production builds.
Based upon these steps:
ng new test-project
ng build --prod
I am seeing a dist folder size of 1.1 MB across the 14 files listed here:
./app/index.js
./app/size-check.component.css
./app/size-check.component.html
./favicon.ico
./index.html
./main.js
./system-config.js
./tsconfig.json
./vendor/es6-shim/es6-shim.js
./vendor/reflect-metadata/Reflect.js
./vendor/systemjs/dist/system.src.js
./vendor/zone.js/dist/zone.js
Note Currently to install the webpack version of the angular cli, you must run... npm install angular-cli#webpack -g

Creating a new project with angular cli recently and the node_modules folder was 270 mb, so yes this is normal but I'm sure most new devs to the angular world question this and is valid. For a simple new project it would make sense to pare the dependencies down maybe a bit;) Not knowing what all the packages depend on can be a bit unnerving especially to new devs trying the cli out for the first time. Add to the fact most basic tutorials don't discuss the deployment settings to get the exported files only needed. I don't believe even the tutorial offered on the angular official website talks about how to deploy the simple project.

Angular itself has lots of dependencies, and the beta version of CLI downloads four times more files.
This is how to create a simple project will less files ("only" 10K files)
https://yakovfain.com/2016/05/06/starting-an-angular-2-rc-1-project/

Seems like nobody have mentioned Ahead-of-Time Compilation as described here: https://angular.io/docs/ts/latest/cookbook/aot-compiler.html
My experience with Angular so far is that AoT creates the smallest builds with almost no loading time. And most important as the question here is about - you only need to ship a few files to production.
This seems to be because the Angular compiler will not be shipped with the production builds as the templates are compiled "Ahead of Time". It's also very cool to see your HTML template markup transformed to javascript instructions that would be very hard to reverse engineer into the original HTML.
I've made a simple video where I demonstrate download size, number of files etc. for an Angular app in dev vs AoT build - which you can see here:
https://youtu.be/ZoZDCgQwnmQ
You'll find the source code for the demo here:
https://github.com/fintechneo/angular2-templates
And - as all the others said here - there's nothing wrong when there are many files in your development environment. That's how it is with all the dependencies that comes with Angular, and many other modern frameworks. But the difference here is that when shipping to production you should be able to pack it into a few files. Also you don't want all of these dependency files in your git repository.

This is actually not Angular specific, it happens with almost any project that uses the NodeJs / npm ecosystem for its tooling.
Those project are inside your node_modules folders, and are the transititve dependencies that your direct dependencies need to run.
In the node ecosystem modules are usually small, meaning that instead of developing things ourselves we tend to import most of what we need under the form of a module. This can include such small things like the famous left-pad function, why write it ourselves if not as an exercise ?
So having a lot of files its actually a good thing, it means everything is very modular and module authors frequently reused other modules. This ease of modularity is probably one of the main reasons why the node ecosystem grew so fast.
In principle this should not cause any issue, but it seems you run into a google app engine file count limit. In that I case I suggest to not upload node_modules to app engine.
instead build the application locally and upload to google app engine only the bundled filesn but don't to the build in app engine itself.

If you are using angular cli's newer version use ng build --prod
It will create dist folder which have less files and speed of project will increased.
Also for testing in local with best performance of angular cli you can use ng serve --prod

if you use Angular CLI you can always use --minimal flag when you create a project
ng new name --minimal
I've just run it with the flag and it creates 24 600 files and ng build --prod produces 212 KB dist folder
So if you don't need water fountains in your project or just want to quickly test something out this I think is pretty useful

If your file system supports symbolic links, then you can at least relegate all of these files to a hidden folder -- so that a smart tool like tree won't display them by default.
mv node_modules .blergyblerp && ln -s .blergyblerp node_modules
Using a hidden folder for this may also encourage the understanding that these are build-related intermediate files that don't need to be saved to revision control -- or used directly in your deployment.

There is nothing wrong. These are all the node dependencies that you have mentioned in the package.json.
Just be careful if you have download some of the git hub project, it might have lot of other dependencies that are not actually require for angular 2 first hello world app :)
make sure you have angular dependencies
-rxjs
-gulp
-typescript
-tslint
-docker

There is nothing wrong with your development configuration.
if you use Angular CLI you can always use --minimal flag when you create a project
ng new name --minimal

How to set up minimal Aurelia project from scratch

When installing the Aurelia navigation skeleton app it is far to overwhelming with all the 3rd party modules and ready-made scripts it uses. For me who have a good picture of what most of it is in theory, have a hard time learning when I can't do it one step at a time. For this reason I would like to set up a minimal Aurelia project by myself and then add complexity to it as I go along.
Main question: Which steps are necessary to set up a simple Aurelia project?
Assumptions:
I already have a Node server backend that can serve files.
I want to use ES6/7 (Babel).
I want to use system.js for module loading.
No unit or e2e tests, no styles, no docs.
As few node and jspm modules as possible.
Please do also explain a little on each step and describe what the necessary Aurelia files is and does.
I would be very thankful for any help :)

Install the jspm command line interface. jspm is a package manager for client-side dependencies. Read up on it... it's great.
npm install jspm -g
Create a folder for the project.
mkdir minimal
cd minimal
Initialize jspm client package management...
Accept all the defaults EXCEPT use the Babel transpiler option (vs Traceur)
jspm init
Enable all the fancy cutting edge babel goodness by adding the following line to the babelOptions in your config.js (jspm init created the config.js file):
System.config({
defaultJSExtensions: true,
transpiler: "babel",
babelOptions: {
"stage": 0, <------ add this to turn on the hotness
"optional": [
"runtime"
]
},
...
Install Aurelia
jspm install aurelia-framework
jspm install aurelia-bootstrapper
Create an index.html that uses the SystemJS loader (jspm's module loader counter-part) to bootstrap Aurelia.
<!doctype html>
<html>
<head>
<title>Aurelia</title>
</head>
<body aurelia-app>
<h1>Loading...</h1>
<script src="jspm_packages/system.js"></script>
<script src="config.js"></script>
<script>
System.import('aurelia-bootstrapper');
</script>
</body>
</html>
When Aurelia bootstraps it's going to look for a default view and view-model... create them:
app.js
export class App {
message = 'hello world';
}
app.html
<template>
${message}
</template>
Install gulp and browser-sync to serve the files:
npm install gulp
npm install --save-dev browser-sync
Add a gulpfile.js
var gulp = require('gulp');
var browserSync = require('browser-sync');
// this task utilizes the browsersync plugin
// to create a dev server instance
// at http://localhost:9000
gulp.task('serve', function(done) {
browserSync({
open: false,
port: 9000,
server: {
baseDir: ['.'],
middleware: function (req, res, next) {
res.setHeader('Access-Control-Allow-Origin', '*');
next();
}
}
}, done);
});
Start the webserver.
gulp serve
Browse to the app:
http://localhost:9000
Done.
Here's what your project structure will look like when you're finished:
Note: this is just a quick and dirty setup. It's not necessarily the recommended folder structure, and the loader is using babel to transpile the js files on the fly. You'll want to fine tune this to your needs. The intent here is to show you how to get up and running in the fewest steps possible.

Check the article https://github.com/aurelia-guides/aurelia-guides.md-articles/blob/master/Building-Skeleton-Navigation-From-Scratch.md written specifically to ease the introduction to Aurelia.

I created a repo (up to date as of April 2017) that includes the absolute barebones necessary items to run Aurelia at https://github.com/nathanchase/super-minimal-aurelia
It's an ES6-based Aurelia implementation (rather than Typescript), it incorporates
code-splitting by routes (using the latest syntax in Aurelia's router to designate chunk creation according to files under a route), and it works in all evergreen browsers AND Internet Explorer 11, 10, and 9 thanks to a few necessary included polyfills.

The Aurelia documentation has a really nice chapter that explains what each part of a simple application does, one step at a time. It is probably a good start that do not overwhelm you with dependencies like Bootstrap and similar.
Also note that there is now a CLI interface to Aurelia that simplifies setting up a project from scratch.

I would definitely use the aurelia-cli for this.
Do the following:
npm install -g aurelia-cli
Then to start a new project do:
au new project-name
to run your project do:
au run --watch
I really feel the aurelia-cli "is the future" for aurelia!

I'm working with a Java Play project and still want to use the scala conversion of HTML file. Thus, I did the following
Download aurelia-core available via the basic aurelia project, which is linked from the quickstart tutorial
Fetch SystemJS using WebJars: "org.webjars.npm" % "systemjs" % "0.19.38"
Since systemjs-plugin-babel is currently unavailable as webjar, I ran npm install systemjs-plugin-babel and copied the fetched files to the assets directroy
The HTML code is like this:
<div aurelia-app="/assets/aurelia/main">
...loading data...
</div>
<script src="#routes.Assets.versioned("lib/systemjs/dist/system.js")" type="text/javascript"></script>
<script src="#routes.Assets.versioned("javascripts/aurelia-core.min.js")" type="text/javascript"></script>
<script>
System.config({
map: {
'plugin-babel': '#routes.Assets.versioned("javascripts/systemjs-plugin-babel/plugin-babel.js")',
'systemjs-babel-build': '#routes.Assets.versioned("javascripts/systemjs-plugin-babel/systemjs-babel-browser.js")'
},
transpiler: 'plugin-babel',
packages: {
'/assets/aurelia': {
defaultExtension: 'js'
}
}
});
System.import('aurelia-bootstrapper');
</script>
Use main.js etc. from the quickstart tutorial

How do I use a NodeJS package with RequireJS?

I would like to use the the node-bencoding package with my current RequireJS project setup, but I have been unable to get it configured.
I have followed these instructions and have ran:
npm install requirejs
npm install node-bencoding
Then in my app.js file I had changed it:
var requirejs = require('requirejs');
// Place third party dependencies in the lib folder
//
// Configure loading modules from the lib directory,
// except 'app' ones,
requirejs.config({
nodeRequire: require,
"baseUrl": "assets/js/lib",
"paths": {
"app": "../app",
"jquery": "https://ajax.googleapis.com/ajax/libs/jquery/2.0.0/jquery.min",
"angularjs": "https://ajax.googleapis.com/ajax/libs/angularjs/1.0.6/angular.min"
},
});
However when I load the page I get the error:
Error: Module name "requirejs" has not been loaded yet for context: _. Use require([]) http://requirejs.org/docs/errors.html#notloaded
I'm not exactly sure where I should have my node_modules directory. My directory structure is as follows: all my JS files are contained within src/assets/js - there is assets/js/app and assets/js/lib as is the RequireJS convention. Currently I have put my node_modules directory in src/.

Looks like you are trying to use it in a browser. And your application is not server side JavaScript, so RequireJS usage sample in a Node does not apply. In this case you would like to use node only to optimize your scripts.
I recently blogged about Understanding AMD & RequireJS, it might be useful.