Fetching all private npm modules from an npmjs user - javascript

I'm working on a task to migrate all of our npm modules from npmjs.com.
Problem is, all of our modules are stored under an npm user as private packages, and I can't find a way to consume all packages the user owns programmatically.
I found many examples that explain how to migrate a single package or a list of packages by name, but I've yet to understand how can I fetch every package a user owns to create that list.
There are a few modules that allow interacting with the NPM registry API, but I couldn't find out how to make them consume private scoped packages. I'm currently working with the npm-registry-fetch to try and get an example query working with no luck.
This is the code I'm trying to run -
var npmFetch = require('npm-registry-fetch')
async function run() {
var response = await npmFetch.json('#<scope redacted>/<package name redacted>',{token: '<token redacted>'})
}
run()
This code results in this error -
HttpErrorGeneral: 404 Not Found - GET https://registry.npmjs.org/#<scope redacted>/<package name redacted> - Not found
How can I successfully query these type of packages?
Thanks.

Updating that this SO answer gave me the tools I've needed, performing the request as a curl command worked, so I'll just use that instead of using fancy registry clients. Thanks again internet stranger!

Related

Failed to resolve module specifier

I`m only starting my JS journey and I will be really grateful if you help me to receive data using the JS. I found that info on the alcor exchange site that is the exchange site for wax (gaming crypto currency).
What is on site:
// Code not tested yet, and provided for explanation reason
import fetch from 'node-fetch'
import { Api, JsonRpc, RpcError } from 'eosjs'
const rpc = new JsonRpc('https://wax.greymass.com', { fetch })
// Get buy orderbook from table
const { rows } = await rpc.get_table_rows({
code: 'alcordexmain',
table: 'buyorder',
limit: 1000,
scope: 29, // Market id from /api/markets
key_type: 'i128', // we are using it for getting order sorted by price.
index_position: 2
})
I faced with some trouble because of JSHint version and updated it to 9. But still "await" is red and JSHint is asking for semicolon after it - which causes huge amount of new errors. However the project is opening in the browser with no info of course. But in the console I see an error.
index.html:1 Uncaught TypeError: Failed to resolve module specifier "node-fetch". Relative references must start with either "/", "./", or "../".
P.S. I checked the posts with such error but actually didn't understand what should I do because all of them are proposing some changes for JSON file and I only have index.html and that js. file.
There is a difference between JavaScript in your browser and JavaScript on a server.
JavaScript in a browser is the code that can be injected into HTML (inlined or linked), which is evaluated by the browser.
JavaScript on a server is not related to JavaScript in a browser. The language is the same, but the environment is not. It's like “draw in Paint” and “draw on a real life canvas”. Colors are the same, but everything else is not.
So, what you are trying to do here is to run server-side JavaScript in a browser. That's the real reason you're getting this error.
There are two ways to fix this error.
Probably the one you should go
You should install Node.js, read about npm, init your npm project, put everything into .js file and eval using Node.
In a nutshell, let's say you've already installed Node.js and node -v outputs something in your terminal. Then everything you need to do is:
$ cd path/to/the/directory/you/like
$ npm init -f
$ npm install --save node-fetch eosjs
$ touch index.js
Then edit index.js using your favorite editor, adding there the code you're trying to run.
You may encounter error due to using await on a “top-level”. In this case, put it into an async function:
import fetch from 'node-fetch'
import { Api, JsonRpc, RpcError } from 'eosjs'
const rpc = new JsonRpc('https://wax.greymass.com', { fetch })
(async () => {
const { rows } = await rpc.get_table_rows({
code: 'alcordexmain',
table: 'buyorder',
limit: 1000,
scope: 29, // Market id from /api/markets
key_type: 'i128', // we are using it for getting order sorted by price.
index_position: 2
});
})();
Aaaand, that's it. You do not need to run browser here at all.
Probably the one you should not go, but can
If you need to run your JavaScript in a browser, then you need to either bundle all the deps using a tool like webpack (which still requires you to use Node.js & npm), or you may replace the deps you're trying to use with client-side alternatives.
E.g. instead of requiring node-fetch you may use Fetch API.
What to use instead of eosjs I do not know, but if you decide to use this dependency in a browser, you will at least need to use import maps to tell the browser how to resolve such dependencies. Because, well, the error you've got tells you exactly this: your browser does not know what you're trying to import. Browsers use URLs as import paths, not ids (or “bare names”).

How do I convert SmartyStreets' jQuery.LiveAddress plugin to its JavaScript SDK?

I have a website where the jQuery.LiveAddress plugin is implemented, but it was deprecated and then totally removed by SmartyStreets in 2014.
https://www.smartystreets.com/archive/jquery-plugin/website/configure
<script src="//ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script src="//d79i1fxsrar4t.cloudfront.net/jquery.liveaddress/5.2/jquery.liveaddress.min.js"></script>
<script>var liveaddress = $.LiveAddress({
key: "YOUR_WEBSITE_KEY_HERE",
debug: true,
target: "US|INTERNATIONAL",
addresses: [{
address1: '#street-address',
locality: '#city',
administrative_area: '#state',
postal_code: '#zip',
country: '#country'
}]
});
</script>
There is now a JavaScript SDK: https://www.smartystreets.com/docs/sdk/javascript
const SmartyStreetsSDK = require("smartystreets-javascript-sdk");
const SmartyStreetsCore = SmartyStreetsSDK.core;
const Lookup = SmartyStreetsSDK.usStreet.Lookup;
// for Server-to-server requests, use this code:
// let authId = process.env.SMARTY_AUTH_ID;
// let authToken = process.env.SMARTY_AUTH_TOKEN;
// const credentials = new SmartyStreetsCore.StaticCredentials(authId, authToken);
// for client-side requests (browser/mobile), use this code:
let key = process.env.SMARTY_WEBSITE_KEY;
const credentials = new SmartyStreetsCore.SharedCredentials(key);
It says "for client-side requests (browser/mobile), use this code", but am I correct in assuming that it is not possible to get this code working as browser JavaScript? Wouldn't I need to be running node as a service on my server to be using this code?
The client-side code example can be used in the browser, not just in node. You will need to store your website key in a different way since the browser doesn't have access to process.env. You can store the website key in plaintext here since it will be tied to your hostname.
You will need to be using some kind of library or JS bundler to process the require statement you see on line 1. You can also import the SDK if that's what works better for your setup, for example
import SmartyStreetsSDK from "smartystreets-javascript-sdk"
Something to keep in mind with using the SDK vs the jQuery plugin is that the SDK does not provide and UI elements on the page. It's less of a drop in solution than the jQuery plugin is. You will need to design and create your own UI elements for the user to interact with.
disclaimer: I work for Smarty (formerly SmartyStreets)
I came across this same issue and unfortunately smarty streets decided to make it more difficult for users to implement their product.
Upon my search I came across this fiddle, it doesn't have all of the features the jquery plugin did but it is a good place to start that doesn't require building your own suggestion UI.
<script async src="//jsfiddle.net/smartystreets/uptgh7c8/embed/"></script>
jsfiddle source
The key file that I was missing was the browserify.js file in the SmartyStreets JavaScript SDK.
Run npm install smartystreets-javascript-sdk --save-dev in your project's root directory
Change directory to node_modules/smartystreets-javascript-sdk
Run npm install in this directory
Run rm -rf dist && node browserify.js (script will fail if the dist folder exists)
Copy dist/smartystreets-sdk-1...min.js to a dependencies folder for your main project.
I tried to write a grunt task to take the SmartyStreets SDK and automatically update it, browserify it, and deploy it along with the rest of our website, but it turned out to be too much of a hassle with the time constraints given. If anyone else writes a grunt task for this please share!

Static context in an NPM module across local PM2 cluster (NodeJS)

I'm dealing and wondering with context issues (and possibilities) in our microservice backend and I'm trying to better understand what exactly is shared across NodeJS forked cluster instances (for example using PM2).
The situation - lets assume I have an NPM package named that exports one instance of itself as in:
class Content {
constructor(){
if(!Content.Registry) Content.Registry = {};
}
registerStatic(key,obj) {
Content.Registry[key] = obj;
}
getStatic(key){
return Content.Registry[key];
}
}
module.exports = new Content();
and my Node application includes this module at several points and registers the instances of several objects using the static context methods.
Now, I know that the require registry from Node caches the instances when you require a package (or to maybe go with a better wording that suits the node documentation - it will 'probably' bring out the same cached instance).
Questions asked:
can I count on the package being cached? I require it as an NPM module and not as a file (require('package-name'))
if even though the instance returned from the require registry won't be the same - will the static context be the same across the same PID? (Content.Registry)
to complicate things a bit more - when running this whole thing over PM2 - will every forked child PID will have a different package instance AND static context? will the static context be preserved? I have already noticed that Node shares some resources (the sharing of sockets via IPC is documented, however we found that even file handles are shared when trying to write to the same file from different PM2 instances - logic says each one will try to acquire a lock handle on the file but only one will succeed. in reality they all write together - this was done using winston)
Thanks for any help!
EDIT: would there even be any way to test for this? could I somehow access within the NPM package some node/javascript functionality that would give me some UUID of the package's instance? memory address?

Method Overriding of a Node Module

I am using keystone node module to develop cms based pages in my application.
I initialize keystone just by adding it to my js file as: var keystone = require('keystone');
But the problem which i am facing currently is that, the the route for every keystone based cms feature is
localhost:3000/keystone/<featue-name>
I want to remove keystone from the url with another name required for the app. Making changes inside the node-module of keystone does the trick!
But if i do an npm update all my changes goes in vain. Normally, in other languages i used to do it it by METHOD OVERRIDING. I don't know about Method Overriding in a node-module.
Is there any other way of doing it?
From what I can see in the source, the admin path prefix is configurable (just not documented):
keystone.init({
...
'admin path' : 'your-own-path',
...
});
(which would make the path localhost:3000/your-own-path/<feature-name>)
If you're not using the latest Keystone, perhaps this might work (although it's a bit of a hack):
keystone.pre('routes', function(req, res, next) {
req.url = req.url.replace(/^\/your-own-path/, '/keystone');
next();
});
What this does is replace a /your-own-path prefix in requested URL's with /keystone, sort of like an "internal redirect".
Making changes to the source code in node_modules is a not a good way of solving it since it's not version controlled so everyone working on the repo will have to change it manually. Instead you can fork the repo to your own Github account, do the changes and then install that version of Keystone:
$ npm install --save keystone#git+https://github.com/your-username/keystone.git
Even better, make a fix that defaults to /keystone but is possible to change with the Keystone options and create a PR back to Keystone! The use after you have fixed it should look something like this:
keystone.set('url', 'custom'); // Changes /keystone to /custom

Getting started with Faker.js with Meteorite

I am trying to user Faker.js with Meteorite. I have added this package:
https://atmosphere.meteor.com/package/Faker.js
I am trying to add fake users on server startup:
Meteor.startup(function () {
// code to run on server at startup
// add 100 fake tips and 10 fake articles and 50 fake users
for (var i=0; i<50; i++) {
// Accounts.createUser(Faker.Internet.userName(), Faker.Internet.email(), "1234");
}
});
but I get an error:
ReferenceError: Faker is not defined
Not sure how to fix this...
I'd recommend installing the npm package, and adding Faker as an NPM module. It's quite simple:
mrt add npm
Create a packages.json file in your project root, and add { "Faker": "0.5.11" }
mrt update
Use Faker in your code by doing Faker = Meteor.require('Faker');
Now you can use it like normal, e.g. Faker.Internet.email()
The author of Faker.js on atmosphere needs to update the package for it to work properly on Meteor 0.6.5.
In meteor 0.6.5 packages need to explicitly expose their variables. This was done so that packages variables don't conflict.
For the 'fakerjs' package the edits look pretty minor because it just interfaces the npm module.
You need to export 'Faker' in the package.on_use method in the package.js with an api.export, if you contact the author or send him a push request you could push it through faster. More details on how to expose the variables can be found in the namespacing section on the meteor docs.

Categories