I am using Gatsby as a Static Site Generator and using Netlify to deploy.
Netlify lets you set Environment Variables in its UI backend.
I've set a few env vars in the Netlify backend to be able to post subscribers to a mailing list.
DATA_NO = 'XXXX'
LIST_ID = '123456'
API_KEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXX'
In my src files, I've got a component that responds to a onSubmit event and constructs a URL to post a new subscriber.
(axios is used as a package for sending HTTP requests, etc)
import React, { useState } from "react"
import axios from 'axios'
const Form = () => {
const [userEmail, setState] = useState({'email_address': ''})
const creds = 'anystring:'+ process.env.API_KEY
let URL = 'https://'+ process.env.DATA_NO +'.api.example.com/3.0'
URL += '/lists/'+ process.env.LIST_ID +'/members'
const submitSubscribe = async e => {
e.preventDefault()
const payload = {
'email_address': userEmail.email_address,
'status': 'subscribed'
}
try {
const response = await axios.post( URL , payload, {
headers: {
'Authorization': 'Basic ' + Buffer.from(creds ).toString('base64')
}
})
console.log('r', response)
console.log('r data', response.data)
} catch(err) {
console.log(err);
}
}
return (
<form name="newsletter-signup" method="post" onSubmit={submitSubscribe}>
{/*<input type="hidden" name="form-name" value="newsletter-signup" />*/}
<input type="email" placeholder="Email required" onChange={handleChange} value={userEmail.email_address} required />
<button type="submit" className="button primary-button-inverted">Send'</button>
</form>
)
}
So, what's happening is that on RUN time, my env vars are coming out as undefined.
I've been on the Netlify docs and they keep saying you need to interpolate the values to the client to be able to use them. I understand the logic here. These env vars need to be printed and bundled during build time, not invoked at run time.
The question I'm struggling with is HOW do I do this?
I have set up a .env.development file in the root of my project. I have tried prefixing my env vars with GATSBY_ but I still have the same trouble.
I tried using require('dotenv').config() but I'm not sure where exactly to put that (in my gatsby-node.js, gatsby-config.js) or do I need to include on the page with my component that is using these env vars.
I'd like to be able to set these vars up in one place (maybe two if testing in development) but I don't want to much tweaking involved to be able to use them in both development and production builds.
I also understand that Netlify or Gatsby can process these vars into a functions/ folder in my source code that I can somehow make use of but that seems like more than I need to just post a simple form.
Please help!
Update
Current code:
In my project root, I created two .env files, one for development and one for production. They each share the following format (remember, I am developing in GatsbyJS):
GATSBY_MC_API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxx-xxxx"
GATSBY_MC_DATA_NO="xxxx"
GATSBY_MC_AUDIENCE_ID="xxxxxxxxxxx"
I've set up a separate config.js file in src/config/config.js to organize and validate my env vars (thanks #Baboo_). It looks like:
export const MC_API_KEY = process.env.GATSBY_MC_API_KEY;
export const MC_DATA_NO = process.env.GATSBY_MC_DATA_NO;
export const MC_AUDIENCE_ID = process.env.GATSBY_MC_AUDIENCE_ID;
const envVars = [
{name: "MC_API_KEY", value: MC_API_KEY},
{name: "MC_DATA_NO", value: MC_DATA_NO},
{name: "MC_AUDIENCE_ID", value: MC_AUDIENCE_ID}
]
export const checkEnvVars = () => {
const envVarsNotLoaded = envVars.filter((envVar) => envVar.value !== undefined);
if (envVarsNotLoaded.length > 0) {
throw new Error(`Could not load env vars ${envVarsNotLoaded.join(",")}`);
}
}
checkEnvVars()
However, when I run gatsby develop, the "Could not load env vars" error gets thrown.
You are doing it the right way.
What you have to do is indeed prefix your environment variables with GATSBY_, Gatsby will automatically load them. Then call them in your code:
const creds = 'anystring:'+ process.env.GATSBY_API_KEY
let URL = 'https://'+ process.env.GATSBY_DATA_NO +'.api.example.com/3.0'
tURL += '/lists/'+ process.env.GATSBY_LIST_ID +'/members'
Make sure to use the whole string process.env.GATSBY_LIST_ID instead of process.env[GATSBY_LIST_ID] because the object process.env is undefined.
Locally
Make sure to create to .env files, .env.development and .env.production. The former is used when you run gatsby develop and the latter when you run gatsby build.
You may already know that you shouldn't commit these files.
Netlify
Add the same environment variables in your deployment pipeline on Netlify. Here is the related doc. This way Netlify can build your webiste when being deployed.
Improvements
Instead of refering environment variables directly, create a file where they are loaded and if one of them cannot be retrieved, throw an error. This way you will be noticed when the loading fails and save debugging time.
Example:
// config.js
export const API_KEY = process.env.GATSBY_API_KEY;
export const DATA_NO = process.env.GATSBY_DATA_NO ;
const envVars = [
{name: "API_KEY", value: API_KEY},
{name: "DATA_NO", value: DATA_NO},
]
const checkEnvVars = () => {
const envVarsNotLoaded = envVars.filter(isUndefined);
if (envVarsNotLoaded.length > 0) {
throw new Error(`Could not load env vars ${envVarsNotLoaded.join(",")}`);
}
}
const isUndefined = (envVar) => typeof envVar.value === "undefined";
// component.js
import React, { useState } from "react"
import axios from 'axios'
// Import environment variables
import { API_KEY, DATA_NO } from "./config"
const Form = () => {
// ...
const [userEmail, setState] = useState({'email_address': ''})
const creds = 'anystring:'+ API_KEY
let URL = 'https://'+ DATA_NO +'.api.example.com/3.0'
You need to add a different env file for the two environments to make this work.
Meaning .env.development and .env.production.
Related
Based on this question I posted earlier today, I have added some logging and narrow in that the problem is with the s3.GetObject.
NOTE: I've seen about 6 different ways to do the S3 get object, and have tried a few of them. My mind is mush now... The current one I'm using is based on this: https://blog.tomasztarnowski.com/how-to-fetch-and-update-s3-files-with-aws-lambda-serverless-framework-and-typescript
But I added the try/catch around it.
I think now this better describes my problem:
s3.getObject().promise() never returns anything
I'm exploring all the options there.
But since I'm running with Serverless and React - any ideas specific to that would be greatly appreciated.
This is my current code. I basically took the demo from Serverless/React (running on AWS Lambda) and am I'm just trying to load some HTML from S3 and show in the browser web page.
I'm trying to use React state to set the value on the web page to the value from S3 (based on dangerouslySetInnerHTML getting data from S3 bucket object in Serverless React).
I have defined this in my serverless.yml file:
plugins:
- serverless-webpack
- serverless-plugin-scripts
- serverless-offline
- serverless-s3-deploy
- serverless-iam-roles-per-function
functions:
serve:
handler: handler.serve
events:
- http:
path: /
method: any
cors: true
- http:
path: /{any+}
method: any
cors: true
iamRoleStatementsName: LambdaGetS3BucketCognitivePsychologyCmsDemo
iamRoleStatements:
- Effect: Allow
Action:
- s3:GetObject
Resource: arn:aws:s3:::mybucket/*
Based on what I see in the log (see in yellow in picture below), I think await promise is the problem. It doesn't seem to be waiting.
If I'm getting security error, I want to log that.
How do you call an async routine from a sync routine?
I tried an approach from this page, to wrap my call in a nameless async function.
import "./App.css";
import * as React from "react";
import useConfig from "./components/useConfig";
import logo from "./logo.svg";
import logoPsych from "./psych_mind_logo.jpg"
/**
* Our Web Application
*/
export default function App() {
console.log("Start: 'function App()'" );
const [html, setHtml] =
React.useState<{ __html: string}>({ __html: '<h1>Blog goes here</h1>'})
const config = useConfig();
async function createPageBody() {
console.log("function createPageBody");
const AWS = require('aws-sdk');
const params = {
Bucket: 'mybucket',
Key: 'myObjectFileName.html'
}
// console.log(params);
let s3 = new AWS.S3();
// getting the object with aws-sdk, ".promise()" is required to use async-await
console.log("before await s3.getObject.promise");
try {
const output = await s3
.getObject(params)
.promise();
console.log("after await s3.getObject.promise");
const fileContents = output.Body?.toString() || "<h1>Failed to get S3 File/Body</h1>";
console.log("S3 fileContents=");
console.log(fileContents);
setHtml({ __html: fileContents})
} catch (e) {
console.log("s3.getObject: error=" + e.message);
setHtml({ __html: e.message})
}
// output.Body contains the content of the S3 JSON file
// we expect the file to not be empty, script doesn't fail if it is empty though
return;
}
console.log("About to call createPageBody");
(async() => {
await createPageBody();
})
console.log("Back from call createPageBody");
return (
<div className="App">
<header className="App-header">
<img src={logoPsych} className="App-logo" alt="logo" />
<h1 className="App-title">CognitivePsychology.com</h1>
</header>
<p className="App-intro">
<div dangerouslySetInnerHTML={html} />
</p>
</div>
);
}
This is what the CloudWatch log shows from the execution of the Lambda service. None of the falling logs show up there:
console.log("before await s3.getObject.promise");
console.log("after await s3.getObject.promise");
console.log("s3.getObject: error=" + e.message);
Some of my previous tests (before I put the nameless async around the call to createPageBody) have been showing "h1>Blog goes here" in the browser, the default value for the state. But the above code shows nothing in the browser.
I am trying to send some text on basic of hosted url (where my build is deployed).but i am getting this error
ReferenceError: location is not defined
here is my code
https://codesandbox.io/s/laughing-mendel-pf54l?file=/pages/index.js
export const getStaticProps = async ({ preview = false, previewData = {} }) => {
return {
revalidate: 200,
props: {
//req.host
name: location.hostname == "www.google.com" ? "Hello" : "ccccc"
}
};
};
Can you show your imports, because it could be that you are importing router from 'next/client'
Assuming that you are using functional-based component
You need to import router as follows:
import {useRouter} from "next/router";
in your function body:
const router = useRouter();
getStaticProps() is executed at build time in Node.js, which has no location global object – Location is part of the browser API. Additionally, because the code is executed at build time, the URL is not yet known.
Change getStaticProps to getServerSideProps (see documentation). This will mean the function is called at runtime, separately for each request.
From the context object passed to getServerSideProps, pull out the Node.js http.IncomingMessage object.
On this object, look for the Host header.
export const getServerSideProps = async ({ req }) => {
return {
props: {
name: req.headers.host === "www.google.com" ? "Hello" : "ccccc"
}
};
};
Note:
I also changed == to ===, as it's generally advised to use the latter. The former can produce some unexpected results because of silent type conversions.
I also removed revalidate, as this is not applicable to getServerSideProps().
I am building a NextJS application, currently I am using getStaticPaths and getStaticProps to build the static pages, doing the necessary requests for them.
So I want to build all the pages following this url: challenge/[slug]/ and for each slug that corresponds to an id I want to have a applications page like this: challenge/[slug]/applications to archive this I builded a file [...slug] inside /pages/challenge
Inside that file I have the following code to handle the static generation:
export async function getStaticPaths() {
const response: any = await getPrograms()
const paths = response.results.map(result => {
return { params: { slug: [result.id.toString()] } }
})
return { paths, fallback: true }
}
export async function getStaticProps({ params }) {
const res = await getProgram(params.slug[0])
const stages = await getStages(params.slug[0])
return { props: { program: res, stages: stages }, revalidate: 1 }
}
this solution works for /challenge/[slug], but the /challenge/[slug]/applications receives a 404, how can I render a specific application page for the slug?
I tried to add a second position to the slug array, but if I do it I can just render /challenge/[slug]/applications and not /challenge/[slug]
Any advice?
Thanks!
Firstly, You need to create a FOLDER named [slug]. Then, Create a FILE named applications.js. Lastly, copy and paste that code into this page.
__ challenge
|__ [slug]
|__ applications
In this page you can get or set slug as your desired parameter.
I have the following object:
const basePoints = {}
which I need to fill with json files. Currently I do:
import WH11 from 'assets/WH11';
const basePoints = { WH11}
I have like a dozen of such Json files but only 2-3 can be used at a given time. INstead of importing and loading all the JSON files i don't need, I want to require/import based on a config file as shown below:
and my config.js:
const config = {
basePoints: {
"WH11": "West Gate",
"WH12": "West Gate Back Drop"
}
}
WH11, WH12 etc basically exist in json format in my assets directory:
assets/basepoints/WH11.json
{
"startingID" : 198
}
etc. Now there can a dozen or more of such json files. The user just adds the ones to be used for the month in config.js.
Is there a way to require/import the json file based on the config file. The app can't compile if I do:
Object.keys(config.basePoints).forEach(key => {
basePoints[key] = require('../assets/basepoints/' + key + '.json');
});
the error is unexpected require().
You can use the latest ES2020 feature - Dynamic Import
Syntax -
import('/modules/<module_name>')
.then(module => {
//
})
.catch(err => {
//
});
You can learn more about it in this MDN document (scroll down to the dynamic import section) -
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
I'm trying to make a web worker to prevent stalling the React main thread. The worker is supposed to read an image and do various things.
The app was created using create-react-app.
Currently I have
WebWorker.js
export default class WebWorker {
constructor(worker) {
const code = worker.toString();
const blob = new Blob(['('+code+')()'], {type: "text/javascript"});
return new Worker(URL.createObjectURL(blob), {type: 'module'});
}
}
readimage.worker.js
import Jimp from "jimp";
export default () => {
self.addEventListener('message', e => { // eslint-disable-line no-restricted-globals
if (!e) return;
console.log('Worker reading pixels for url', e.data);
let data = {};
Jimp.read(e.data).then(image => {
// jimp does stuff
console.log('Worker Finished processing image');
})
postMessage(data);
})
};
And then in my React component AppContent.js I have
import WebWorker from "./workers/WebWorker";
import readImageWorker from './workers/readimage.worker.js';
export default function AppContent() {
const readWorker = new ReadImageWorker(readImageWorker);
readWorker.addEventListener('message', event => {
console.log('returned data', event.data);
setState(data);
});
// callback that is executed onClick from a button component
const readImageContents = (url) => {
readWorker.postMessage(url);
console.log('finished reading pixels');
};
}
But when I run it, I get the error
Uncaught ReferenceError: jimp__WEBPACK_IMPORTED_MODULE_0___default is not defined
How can I properly import a module into a web worker?
EDIT:
As per suggestions from Kaiido, I have tried installing worker-loader, and edited my webpack.config.js to the following:
module.exports = {
module: {
rules: [
{
test: /\.worker\.js$/,
use: { loader: 'worker-loader' }
}
]
}
};
But when I run it, I still get the error
Uncaught ReferenceError: jimp__WEBPACK_IMPORTED_MODULE_0__ is not defined
I'm not too much into React, so I can't tell if the module-Worker is the best way to go (maybe worker-loader would be a better solution), but regarding the last error you got, it's because you didn't set the type of your Blob when you built it.
In this case, it does matter, because it will determine the Content-Type the browser sets when serving it to the APIs that fetch it.
Here Firefox is a bit more lenient and somehow allows it, but Chrome is picky and requires you set this type option to one of the many javascript MIME-types.
const script_content = `postMessage('running');`;
// this one will fail in Chrome
const blob1 = new Blob([script_content]); // no type option
const worker1 = new Worker(URL.createObjectURL(blob1), { type: 'module'});
worker1.onerror = (evt) => console.log( 'worker-1 failed' );
worker1.onmessage = (evt) => console.log( 'worker-1', evt.data );
// this one works in Chrome
const blob2 = new Blob([script_content], { type: "text/javascript" });
const worker2 = new Worker(URL.createObjectURL(blob2), { type: 'module'});
worker2.onerror = (evt) => console.log( 'worker-2 failed' );
worker2.onmessage = (evt) => console.log( 'worker-2', evt.data );
But now that this error is fixed, you'll face an other error, because the format import lib from "libraryname" is still not supported in browsers, so you'd have to change "libraryname" to the path to your actual script file, keeping in mind that it will be relative to your Worker's base URI, i.e probably your main-page's origin.
I experienced the same problem. Firefox could not show me where exactly the error was (in fact it was plain misleading...) but Chrome did.
I fixed my problem by not relying on an import statement (importing one of my other files) which would only have worked within a React context. When you load a Worker script (via the blob()
/ URL() hack), it has no React context (as it is loaded at runtime and not at transpile time). So all the React paraphernalia __WEBPACK__blah_blah is not going to exist / be visible.
So... within react... import statements in worker files will not work.
I haven't thought of a workaround yet.