We Keep Coding

How can I get a value from index.html and use it to render a second template after parsing JSON file?

I'm trying to wrap my head around gulp and modern JavaScript bundling.
The application is pretty simple:
A html page is displayed to the user, where he must check a few radio buttons.
User then submits his selections clicking on a button.
Upon choice submit, a JSON file must be parsed, in order to display one amongst 20 possible html pages
I have the following (simplified) structure:
dist/
index.html
src/
scripts/
main.js
modules/
dispatchForm.js
styles/
index.scss
templates/
index.pug
partials/
default.pug
selectors.pug
gulpfile.js
data.json
In my gulpfile.js, I have:
const bundler = () => {
return rollup({
input: './src/scripts/main.js',
plugins: [
babel(pkg.babel),
nodeResolve(),
commonJS(),
],
}).then((bundle) => bundle.write({
file: '.tmp/bundle.js',
format: 'umd',
sourceMap: 'inline',
}));
};
const uglify = () => {
return src('.tmp/bundle.js', {sourcemaps: true})
.pipe(plugins.uglify())
.pipe(dest('.tmp'))
.pipe(plugins.size({title: 'bundler minified'}));
};
const styles = () => {
const AUTOPREFIXER_BROWSERS = [
'ie >= 10',
'ie_mob >= 10',
'ff >= 30',
'chrome >= 34',
'safari >= 7',
'opera >= 23',
'ios >= 7',
'android >= 4.4',
'bb >= 10',
];
return src([
'src/styles/main.scss',
'src/styles/**/*.css',
])
.pipe(plugins.sassGlob())
.pipe(plugins.sass({
precision: 10,
}).on('error', plugins.sass.logError))
.pipe(plugins.autoprefixer(AUTOPREFIXER_BROWSERS))
.pipe(dest('.tmp'))
.pipe(plugins.if('*.css', plugins.cssnano()))
.pipe(dest('.tmp'))
.pipe(plugins.size({title: 'styles'}));
};
// Uses PUG as template
const templates = (env) => () => {
return src('./src/templates/*.pug')
.pipe(plugins.pug({locals: {
title: pkg.title,
description: pkg.description,
env,
}}))
.pipe(dest('dist'))
.pipe(plugins.size({title: 'templates'}));
};
const reload = (done) => {
server.reload();
return done();
};
const images = (env) => () => {
const destination = env === 'deploy' ? 'dist' : '.tmp';
return src('./src/images/**/*.{gif,jpg,png,svg}')
.pipe(dest(destination))
.pipe(plugins.size({title: 'size'}))
};
const serve = () => {
server.init({
notify: false,
logPrefix: 'WSK',
scrollElementMapping: ['main', '.mdl-layout'],
server: ['.tmp', 'dist'],
port: 3000,
});
watch(
['src/**/*.pug'],
series(templates('development'), reload)
);
watch(
['src/styles/**/*.{scss,css}'],
series(styles, templates('development'), reload)
);
watch(
['src/scripts/main.js', 'src/scripts/**/*.js'],
series(bundler, templates('development'), reload)
);
watch(
['src/images/**/*.{gif,jpg,png,svg}'],
series(images('development'), templates('development'), reload)
);
};
const clean = () => del(['.tmp', 'dist/*', '!dist/.git'], {dot: true});
exports.default = series(
clean,
bundler,
uglify,
styles,
templates('deploy'),
images('deploy')
);
exports.serve = series(
bundler,
styles,
templates('development'),
images('development'),
serve
);
The way I understand it, after cleaning the files, the bundler will:
Serve a html page in dist folder after compiling from my sources main.js, dispatchForm.js, scss and pug templates.
Main.js
import dispatchForm from './modules/dispatchForm';
const domContentLoad = (fn) => {
if (document.readyState !== 'loading') fn();
else document.addEventListener('DOMContentLoaded', fn);
};
domContentLoad(() => {
dispatchForm();
});
dispatchForm.js
const button = document.querySelector('[data-calculator-button]');
function updateValue() {
const gain = document.querySelector('[data-calculator-form][name="gain"]:checked');
const cost = document.querySelector('[data-calculator-form][name="cost"]:checked');
if (gain && cost) {
button.removeAttribute('disabled');
button.classList.remove('selectors__button--inactive');
} else {
button.setAttribute('disabled', '');
button.classList.add('selectors__button--inactive');
}
}
function dispatchForm() {
const radioInput = document.querySelectorAll('[data-calculator-form]');
radioInput.forEach(element => element.addEventListener('input', updateValue));
}
export default dispatchForm;
selectors.pug
...
.selectors__form
.selectors__radio
input.selectors__input(type='radio' id='gain-points' name='gain' value='points' data-calculator-form)
label.selectors__label(for='gain-points')
.selectors__radio
input.selectors__input(type='radio' id='gain-money' name='gain' value='money' data-calculator-form)
label.selectors__label(for='gain-money')
.selectors__form
for value in [70, 80, 90, 100, 110, 120, 130]
.selectors__radio
input.selectors__input(type='radio' id=`have-${value}` name='cost' value=value data-calculator-form)
label.selectors__label(for=`have-${value}`)
| Até
b= ` C$${value}`
button.selectors__button.selectors__button--calculate.selectors__button--inactive(disabled=true data-calculator-button)
...
The above creates some selectors for 'cost' or 'gain' from selectors.pug, main.js and dispatchForm.js, via gulps 'bundler', and renders it as html.
But now I would like to:
Use one of the two button submitted values (${value}) and pass it as an argument to a function that will parse a JSON file.
Finally, the parsed JSON result will be passed to another pug file
Questions
How do I get this JSON (from dispatchForm.js? from gulpfile.js? from pug natively?) and pass it to another pug template?
Should JSON fetching be dealt with on separate JS module, since displayed JSON will be rendered on a separate html page, using another pug template partial? How so?
Should all possible second page html files be generated at build time and JS be used to display only the one based on the submitted value? Or should this second html page be rendered dinamically?
gulp-data
I also learned that there are packages like gulp-data, used to handle json data, and I don't know if they are the way to go here.
Also, this SO question hints as how to pass pug JSON objects to client side JavaScript.

The way you've phrased this question makes me think your main issue is conflating the build step with the application "runtime" (as in, when users are using your application), like when dispatchForm.js would ever be run. Gulp is a tool to generate out your project, but this is meant to happen way before any users are interacting with your site.
The SO question you linked is having express render the pug pages at "runtime", which, architecturally, is pretty different from having it happen at the build step with gulp.
If I'm understanding what you want correctly, off the top of my head there's 3 main ways to do it. The first would be by having the client-side JS manipulate the dom and change the page appropriately. You could use pug for this, rendered into a JS library via something like rollup-plugin-pug (found via this SO answer).
The second would be having this be an api call to a server, which then renders a pug template (which is what the SO question you linked is doing).
Or third, you could do something like rendering out all possible pages you'd want your users to go to at build time, and just make the dispatchForm.js just be sending them to the appropriate pages. In this case I'd recommend defining out these options in one place (in say, a json file) and having that be the source of a gulp pipeline. It gets a little zany having gulp generate multiple files from a single file, but you can find a variety of ways people have done something similar like this github thread, this Stack Overflow answer, this gist, etc.
Edit
If you want stuff to happen "upon choice submit", that's way 1. So using rollup-plugin-pug, it would look something like (completely untested and off the top of my head):
//- template.pug
.content
p= gain
p= cost
// modules/calculateButton.js
import templateFn from '../../templates/partials/template.pug';
const button = document.querySelector('[data-calculator-button]');
button.addEventListener('click', (e) => {
if (e.target.getAttribute('disabled')) return;
const gain = document.querySelector('[data-calculator-form][name="gain"]:checked').value;
const cost = document.querySelector('[data-calculator-form][name="cost"]:checked').value;
document.getElementById("result").innerHTML = templateFn({
gain, cost
});
});
Otherwise nothing's parsed on submit. For examples of the 3rd way, I'd recommend checking out the links I sent above. A lot of build scripts is just finding your own way to do things that has the right amount of power/complexity balanced with ease of maintenance.

Interpolate env vars client side in gatsby react app

I am using Gatsby as a Static Site Generator and using Netlify to deploy.
Netlify lets you set Environment Variables in its UI backend.
I've set a few env vars in the Netlify backend to be able to post subscribers to a mailing list.
DATA_NO = 'XXXX'
LIST_ID = '123456'
API_KEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXX'
In my src files, I've got a component that responds to a onSubmit event and constructs a URL to post a new subscriber.
(axios is used as a package for sending HTTP requests, etc)
import React, { useState } from "react"
import axios from 'axios'
const Form = () => {
const [userEmail, setState] = useState({'email_address': ''})
const creds = 'anystring:'+ process.env.API_KEY
let URL = 'https://'+ process.env.DATA_NO +'.api.example.com/3.0'
URL += '/lists/'+ process.env.LIST_ID +'/members'
const submitSubscribe = async e => {
e.preventDefault()
const payload = {
'email_address': userEmail.email_address,
'status': 'subscribed'
}
try {
const response = await axios.post( URL , payload, {
headers: {
'Authorization': 'Basic ' + Buffer.from(creds ).toString('base64')
}
})
console.log('r', response)
console.log('r data', response.data)
} catch(err) {
console.log(err);
}
}
return (
<form name="newsletter-signup" method="post" onSubmit={submitSubscribe}>
{/*<input type="hidden" name="form-name" value="newsletter-signup" />*/}
<input type="email" placeholder="Email required" onChange={handleChange} value={userEmail.email_address} required />
<button type="submit" className="button primary-button-inverted">Send'</button>
</form>
)
}
So, what's happening is that on RUN time, my env vars are coming out as undefined.
I've been on the Netlify docs and they keep saying you need to interpolate the values to the client to be able to use them. I understand the logic here. These env vars need to be printed and bundled during build time, not invoked at run time.
The question I'm struggling with is HOW do I do this?
I have set up a .env.development file in the root of my project. I have tried prefixing my env vars with GATSBY_ but I still have the same trouble.
I tried using require('dotenv').config() but I'm not sure where exactly to put that (in my gatsby-node.js, gatsby-config.js) or do I need to include on the page with my component that is using these env vars.
I'd like to be able to set these vars up in one place (maybe two if testing in development) but I don't want to much tweaking involved to be able to use them in both development and production builds.
I also understand that Netlify or Gatsby can process these vars into a functions/ folder in my source code that I can somehow make use of but that seems like more than I need to just post a simple form.
Please help!
Update
Current code:
In my project root, I created two .env files, one for development and one for production. They each share the following format (remember, I am developing in GatsbyJS):
GATSBY_MC_API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxx-xxxx"
GATSBY_MC_DATA_NO="xxxx"
GATSBY_MC_AUDIENCE_ID="xxxxxxxxxxx"
I've set up a separate config.js file in src/config/config.js to organize and validate my env vars (thanks #Baboo_). It looks like:
export const MC_API_KEY = process.env.GATSBY_MC_API_KEY;
export const MC_DATA_NO = process.env.GATSBY_MC_DATA_NO;
export const MC_AUDIENCE_ID = process.env.GATSBY_MC_AUDIENCE_ID;
const envVars = [
{name: "MC_API_KEY", value: MC_API_KEY},
{name: "MC_DATA_NO", value: MC_DATA_NO},
{name: "MC_AUDIENCE_ID", value: MC_AUDIENCE_ID}
]
export const checkEnvVars = () => {
const envVarsNotLoaded = envVars.filter((envVar) => envVar.value !== undefined);
if (envVarsNotLoaded.length > 0) {
throw new Error(`Could not load env vars ${envVarsNotLoaded.join(",")}`);
}
}
checkEnvVars()
However, when I run gatsby develop, the "Could not load env vars" error gets thrown.

You are doing it the right way.
What you have to do is indeed prefix your environment variables with GATSBY_, Gatsby will automatically load them. Then call them in your code:
const creds = 'anystring:'+ process.env.GATSBY_API_KEY
let URL = 'https://'+ process.env.GATSBY_DATA_NO +'.api.example.com/3.0'
tURL += '/lists/'+ process.env.GATSBY_LIST_ID +'/members'
Make sure to use the whole string process.env.GATSBY_LIST_ID instead of process.env[GATSBY_LIST_ID] because the object process.env is undefined.
Locally
Make sure to create to .env files, .env.development and .env.production. The former is used when you run gatsby develop and the latter when you run gatsby build.
You may already know that you shouldn't commit these files.
Netlify
Add the same environment variables in your deployment pipeline on Netlify. Here is the related doc. This way Netlify can build your webiste when being deployed.
Improvements
Instead of refering environment variables directly, create a file where they are loaded and if one of them cannot be retrieved, throw an error. This way you will be noticed when the loading fails and save debugging time.
Example:
// config.js
export const API_KEY = process.env.GATSBY_API_KEY;
export const DATA_NO = process.env.GATSBY_DATA_NO ;
const envVars = [
{name: "API_KEY", value: API_KEY},
{name: "DATA_NO", value: DATA_NO},
]
const checkEnvVars = () => {
const envVarsNotLoaded = envVars.filter(isUndefined);
if (envVarsNotLoaded.length > 0) {
throw new Error(`Could not load env vars ${envVarsNotLoaded.join(",")}`);
}
}
const isUndefined = (envVar) => typeof envVar.value === "undefined";
// component.js
import React, { useState } from "react"
import axios from 'axios'
// Import environment variables
import { API_KEY, DATA_NO } from "./config"
const Form = () => {
// ...
const [userEmail, setState] = useState({'email_address': ''})
const creds = 'anystring:'+ API_KEY
let URL = 'https://'+ DATA_NO +'.api.example.com/3.0'

You need to add a different env file for the two environments to make this work.
Meaning .env.development and .env.production.

Read contents of file into array inside vue component

Inside my media.txt file, I have:
"https://www.dropbox.com/s/******/687.jpg?dl=0",
"https://www.dropbox.com/s/******/0688.jpg?dl=0
I have the following Vue carousel component:
<template>
<section>
<v-card
class="mx-auto"
color="#26c6da"
dark
max-width="1200"
>
<v-carousel>
<v-carousel-item v-for="(item,i) in items" :key="i" :src="item.src"></v-carousel-item>
</v-carousel>
</v-card>
</section>
</template>
<script>
var cache = {};
// const images = require.context('../static/', false, /\.png$|\.jpg/);
// const images = require.context('../static/', false, /\.png$|\.jpg|\.mp4/); // WORKING
const images = require.context('../static/media.txt', false, /\.png$|\.jpg|\.mp4/);
var imagesArray = Array.from(images.keys());
// const images = ["./52lv.PNG", "./Capture1.PNG", "./maps.PNG"]
console.log(images.keys());
console.log('images');
console.log(images);
var constructed = [];
function constructItems(fileNames, constructed) {
fileNames.forEach(fileName => {
constructed.push({
'src': fileName.substr(1)
})
});
return constructed;
}
console.log('items ');
console.log(imagesArray);
// At build-time cache will be populated with all required modules.
var res = constructItems(imagesArray, constructed);
console.log(res);
export default {
data: function() {
return {
items: res
};
}
};
</script>
I want to read the images from the media.txt file into an array, then populate the carousel src with these. I've been trying with Webpack's require.context function, but I'm getting a module not found error.
How can I get this working?

It looks like you're trying to import a string array (JSON) into a variable. That string array should be delimited by square brackets like this:
[
"foo",
"bar"
]
require.context(dirPath, useSubDirs, filenameRegex) is not the appropriate API to use here, as that method imports multiple files from a specified directory. For instance, the code below tells Webpack to load *.png, *.jpg, and *.mp4 files from the directory named ../static/media.txt (which presumably is actually a file).
require.context('../static/media.txt', false, /\.png$|\.jpg|\.mp4/) // DON'T DO THIS
Instead, you could simply use require('path/to/file.json') to import the specified file as a JSON object/array. For example, to import src/assets/media.json (containing the array mentioned at the top), the syntax would be:
// e.g., in src/components/Foo.vue
const media = require('../assets/media.json')
console.log(media) // => [ "foo", "bar" ]
demo

You probably should install https://github.com/webpack-contrib/raw-loader#getting-started (a loader for webpack read txt files), configure it in your vue.config.js and you should be able to import like this: import txt from 'raw-loader!./file.txt'; instead using require.

Find ES6-module in which entity is declared

TL;DR There is some imported entity on ES6-module. Should find original ES module and line number, where this entity has been initially declared - not re-imported / re-exported.
There is some hierarchy of nested imported ES6-modules, some of which are not controlled by library code and had been imported from well-known place - it's some user-side configuration file.
Imagine following files structure:
config.js (not controlled)
import nested from './nested';
export default {
param1: { aaa: 10, bbb: 20 },
param2: nested
}
nested.js (not controlled)
export default {
ccc: 30,
ddd: 40
}
index.js (controlled)
import config from './config'
const ddd = config.ddd;
if(notValid(ddd)) {
const decl = %FIND_DECLARATION%(ddd);
console.log('Config variable "config.ddd" is not valid');
console.log('You declared it here: ', decl);
}
Output should be like following
You declared it here:
File webpack://./nested.js , line 2
ddd: 40
^^^^^^^
Should write function %FIND_DECLARATION%, which can find original ES-module and line number for some object declaration, and retrieve it's source code
Environment is webpack. Any dirty hacks are welcome if can't be solve without it.

Importing data object from file-to-file

I have a .JSON (named dataOut.json) file that looks similar to the following:
[
{
"scan_type": "nexpose",
"date_time": "2011-07-18 11:11:11",
"source_address": "1108",
"source_lat": "40.581160",
"source_lng": "-105.184110",
"dest_address": "11.166.181.0",
"dest_lat": "30.003880",
"dest_lng": "-604.800360"
},
...
]
I have a script that seperates the source/destination data points, and places the output in separate files, here is my code for doing so.
So here is my problem, I need to export the object variable "scan_type" from the code above, to another .JS file, named jsontonbh.js .
I tried using module exports:
//exporting 'scan_type'
obj.forEach(block => {
...
module.exports = {scan_type:block.scan_type};
...
});
and then importing it in my jsontobh.js file by requiring it, as shown here:
let sourceFile = require('./jsonParents.js');
obj.forEach(block => {
console.log(sourceFile.scan_type)
});
this code returns the error "Unexpected end of JSON input"
How can I export this variable scan_type from my first script, to my second script?