WebHookIt development -- Y! Pipes extension and replacement - javascript

So Y!Pipes is shutting down, no real alternatives exist, especially none quite similar to Y!Pipes.
WebHookIt, here for the sources, seems like a potentially good alternative, keeping the simplicity of Y!Pipes' interface, and quite a potential to do what Y!Pipes does, and more: open-source, javascript generator (allowing to host "pipes" on google scripts for example).
However, that's a project that was launched a few years ago, and the dev left soon thereafter. There is work to be done on it, and the user documentation is next to non-existent.
I know SX is absolutely not a forum but I'd like to use this very generic question: "anyone to help me update and complete WebHookIt?" to gather all contributions on it. If it takes, we'll move on to a fork on github.
WebHookIt is coded in javascript using node.js. I can code a bit in javascript, but I'm a newbie regarding node.js (though I'll be very invested in the project for the time being, and I'll begin by getting on page with node as soon as tomorrow).
I'll start with the installation, for *buntu systems:
First the dependencies: npm (node.js installer), node.js, and mango-db:
npm:
install: sudo npm install npm -g
check install with npm -v
you will probably run into trouble later when trying to install webhookit (or anything I think) over the ownership of npm, as I did, if you don't claim it. Hence that generic enough command: sudo chown -R $(whoami) $HOME/.npm
node.js
taaem has made a script to download the latest node and satisfy dependencies. Download "node-install.sh".
make it executable: chmod +x node-install.sh
execute it: ./node-install.sh
follow instructions.
mango-db: follow instruction on their install page. Herein reproduced:
add mango-db repository key: sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
add repository to the sources: echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list
update package database: sudo apt-get update
install mango-db packages: sudo apt-get install mongodb-org*
start mango-db service (it should already be started): sudo service mongod start
WebHookIt is installed by first downloading the repository on github. Now, if you try to install right away you'll run into trouble. This is an old project (latest commit is 3 years old), and the developer didn't provide for improvement to npm. Hence the following steps:
download the sources. With git installed: git clone git://github.com/neyric/webhookit.git
go into the webhookit folder, just created, and edit "package.json". Here is the list of the required dependencies, amongst other info about the program.
find the npm line (should be the 27th), and add ">=" before the version number. End result is: "dependencies": { ... "npm": ">=1.1.0-2", ... },
install! When into the "webhookit" folder, execute: npm install .
a few warnings will pop up, from node being more recent than required by the dependencies. As node is still in the same major version, it shouldn't be a problem (if well developed, a software shouldn't have compatibility problems within a given major revision (eg the first number)).
start webhookit server! node server.js
start using it, in your web browser, by default at http://localhost:8124
Next step is to figure out what it means when it says "Open a wiring first."...

Related

Where to install DataTables in slim4 by Daniel Opitz

I purchased the eBooks by Daniel Opitz. I am on page 226 of the books because I want to implement data tables in my project. In the books it reads.
DataTables Setup
DataTables.net is a very flexible table plug-in for jQuery.
You have to setup jQuery for Webpack first.
To install the DataTables Core library, run:
npm install datatables.net
npm install datatables.net-dt
To install the DataTables Bootstrap 5 styles, run:
npm install datatables.net-bs5
npm install datatables.net-responsive-bs5
npm install datatables.net-select-bs5
Add a new webpack entry in webpack.config.js:
In my naivete, I can't figure out on my own where I should be running the npm install. I was hoping this would be the fastest route to get help.
I have enjoyed learning the skeleton that he published. Most things have been straight forward and not to hard to figure out. This one has me stumped.
I did some more searching looking for the package.json location.
https://odan.github.io/slim4-skeleton/frontend.html
https://odan.github.io/slim4-skeleton/directory-structure.html
I looked over the directory structure and still not obvious to me where the npm should be executed.
Most package managers work by using a configuration file found in the root directory for the tooling you're trying to use. Generally, the assumption is these go in the root of the project. For instance:
/composer.json # PHP's Composer
/package.json # Javascript's NPM
/requirements.txt # Python's PIP
These are typically created by the package manager:
composer init
npm init
pip freeze > requirements.txt
Here are a some articles that can help explain this process:
https://getcomposer.org/doc/01-basic-usage.md
https://docs.npmjs.com/creating-a-package-json-file
https://learnpython.com/blog/python-requirements-file/
I'll also note many package managers will use a lock file, which records what was the last resolved and installed packages. The files above, in other words, depict what should be, while the lock file depicts what was (last) installed.
Your lock file should be handled with care and committed to the your project's repo. When you remove it, you are telling that package manager to recalculate the installed packages list and you may have subtly different versions (leading to potential bugs).
/composer.lock
/package-lock.json

Good way to switch between 2 versions of the same dependency in package.json?

Turns out you can't have comments in JSON files, and it's a bit awkward to have people refer to some documentation telling them what line to copy/paste in and where in order to achieve this.
I think I can make a python script to copy/paste in one of two package.json files depending on what flags they pass in, but that feels overcomplicated.
I think I can include both dependencies (under different names) but that would create a requirement for both to be available, which is not good either.
Looking for ideas/thoughts on a good way to accomplish this. I have a release and dev version of the same dependency and I often need to swap between the two. Would like to improve the workflow beyond just having a notepad on the side with the two lines pasted in it...
yarn and npm already do this job, why not use them?
Releases
Tag the dev versions when you release them
yarn publish --tag dev dep
npm publish --tag dev dep
Then reference the tag at install time yarn install dep#dev.
Local
For local dependencies, npm and yarn provide the "link" command.
In your dependency dir run yarn link
In you project dir run yarn link dep
You could document the commands as scripts so easy to run or view.
"scripts" : {
"dep:local": "yarn link dep",
"dep:dev": "yarn install dep#dev",
"dep:latest": "yarn install dep#latest"
}

Best way to audit dependencies in Yarn 2 (berry)?

I'm looking for a way to audit dependencies for vulnerabilities in Yarn 2. In Yarn 1.x it was possible the same was as in npm, by running yarn audit instead of npm audit. But with Yarn 2 there is no such command. And according to this issue on the berry github, it won't be implemented (project maintainers prefer it was done via plugin).
I have tried running npm install --package-lock-only && npm audit but the install chokes on some of my local packages (which I am listing in package.json using the link: url type).
It wouldn't be a complex plugin to build and I'm up for some fun doing so, but it would not be so muchfun as just installing something and then going about my day. I have looked around but always end up at the same couple of vapourware / abandonware repos.
But I still guess that I'm just not finding them. Or there's an undocumented trick for making it easy. Hence my question :)
PS, yeah I can delete the local packages using link: temporarily while I run the npm install and npm audit commands above, but it's not exactly the kind of thing I want to try and automate for CI.
Update (28-Oct-2020):
Yarn 2 just merged long-awaited yarn npm audit enhancement.
PR - https://github.com/yarnpkg/berry/pull/1892
Docs - https://yarnpkg.com/cli/npm/audit
I was experimenting with Yarn 2 recently, and I saw that you can do this with #efrem/auditdeps utility:
yarn dlx #efrem/auditdeps [--level=(low|moderate|high|critical)] [--production]
The output is not as pretty as from npm audit, but you get more details in JSON format and you can pipe it to other tools or to any custom reformatting script to get exactly what you want.
I'd try https://snyk.io/ not free for huge team commercial use, but it could get you started on daily runs etc.
( I'm not affiliated in any way with Snyk.io )

NPM post-install, CA certificates, and Windows

TLDR;
Is it possible, in a Windows environment, to set NODE_EXTRA_CA_CERTS in a way that works with NPM packages' post-install scripts, without making system changes, configuration file changes, or changes that require admin-level permissions?
Details
This has been driving me nuts.
I added image optimization to our Webpack build process via imagemin, imagemin-webpack, and the various imagemin format-specific plugins.
All of the imagemin plugins have one thing in common -- during post-install, they:
a. Attempt to download a pre-built EXE.
b. If (a) fails, they attempt to build the EXE from source.
I.T. snoops on our traffic, so (a) fails due to the "self-signed certificate in chain" error when attempting to fetch the remote EXE. (b) fails because our studio is Windows-based, and we don't have all the various build tools installed to make that happen. It's also not reasonable to have them installed on every machine where npm install might be run.
I did some digging (thanks S.O.), found our company's CA certificate, added it to the repo, and was able to get (a) working with the following commands:
> SET NODE_EXTRA_CA_CERTS=%cd%\ca.cert
> npm install
I thought I was home free at that point -- all I'd have to do is add this npm script to our package.json:
{
"preinstall": "SET NODE_EXTRA_CA_CERTS=%cd%\\ca.cert"
}
But that doesn't work. I'm guessing it's because there's a separate process involved, and the environment variable doesn't carry over to the other process.
Note that this does work, but is absolutely awful:
{
"preinstall": "SET NODE_EXTRA_CA_CERTS=%cd%\\ca.cert&& npm install imagemin-gifsicle imagemin-mozjpeg imagemin-optipng imagemin-svgo"
}
Is there a way to set this environment variable automatically in a way that works with NPM packages' post-install scripts?
I'd like this to be transparent to other team members so that they can just continue to npm install without any additional steps, system changes, or configuration file changes if at all possible. Some team members are not developers, so while they're used to running npm install, I don't want to introduce any additional complications. I super appreciate any help in advance!
Have you tried
npm config set cafile "<path to your certificate file>"

meteor on armv6l (raspberry pi)

I like the look of meteor, never tried it before, thought it would make a great platform for a web dashboard on my raspberry pi. Tried the recommended installer and got this:
$ curl install.meteor.com | /bin/sh
Unable to install Meteor on unsupported architecture: armv6l
Installation failed.
Well, I'm not gonna take that lying down! Am going to try and manually install it, but has anyone tried/failed/succeeded already? Any tips?
Apparently, there are prebuilt packages for raspi now. Alas, I lost interest long ago - details of prebuilt packages here: http://meteor-universal.tumblr.com/
(Thanks to #Archonic - see the comments below - and if you'd like to make that a real answer, I'd be happy to change the accepted answer for this question.)
I'll keep the text below for reference.
(Updating as I go along...)
Note: i'm doing this on the stock debian image, albeit after having installed ruby, passenger and padrino (and all their dependencies). Archlinux may be easier, I don't know.
Install node.js
Does not work by default, followed instructions here
Checkout meteor
git clone git://github.com/meteor/meteor.git
modify admin/generate-dev-bundle.sh
Essentially, remove all the instructions regarding building node.js, and all references to a tempdir, so you end up just installing node packages. I will post mine as a gist once it's all figured out...
Then, run it in an empty folder
install mongodb. From source.
cd to extracted directory, and install using scons
sudo apt-get install scons libboost-all-dev
scons libboost-dev libpcre++-dev xulrunner-1.9.1-dev
(Note: I will be finishing this soon. I think the trick might be to use meteor's bundle functionality - especially the big red box http://docs.meteor.com/#deploying - but unfortunately my pi has crashed and I'm away from home. Please post if you found that worked, and I'll update this answer accordingly or accept your answer)
You can find a fork of meteor at github called Meteor universal fork. That has a ready to go installer for yet unsupported architectures like ARM and BSD.
For the installation just run:
cd $HOME
git clone --depth 1 https://github.com/4commerce-technologies-AG/meteor.git
and then get the binaries and finish the installation by:
$HOME/meteor/meteor --version
Running the above command for the first time will try to download a compatible pre-built dev_bundle and finish the installation. Depending on your hardware this may take some time.

Categories