How to dynamically create zip archives from gatsby Directory nodes - javascript

I have a bunch of Directories that I am using as gatsby nodes and generating pages from these Directories like:
exports.createPages = async ({ graphql, actions }) => {
const { createPage } = actions
const result = await graphql(`
query DosboxQuery {
allDirectory(filter: {sourceInstanceName: {eq: "dosbox"} name: {ne: "dosbox"}}) {
edges {
node {
name
dir
modifiedTime
children {
__typename ... on File{
name
relativePath
}
}
}
}
}
`)
result.data.allDirectory.edges.forEach(({ node }) => {
console.log("directory node", node);
createPage({
path: `/${node.name}/`,
component: path.resolve(`./src/templates/game-page.js`),
context: {
slug: `/${node.name}/`,
},
})
})
console.log(JSON.stringify(result, null, 4))
}
I would like to generate zip archives of these Directories that users could download from the pages. I don't want to store or commit the zip archives because the content of the directories can change so I want to create the zip archives at gatsby build time. I can probably use a npm module like archiver to create the zip archives.
I'm not sure how to integrate the zip archives into the gatsby infrastructure. I could update onCreateNode or createPages to create the zips. should I write the zips to the public folder? can I store the zips as buffers on the gatsby node?

Related

How to delete a file in NX generator

According to https://nx.dev/recipes/generators/creating-files
Generators provide an API for managing files within your workspace. You can use generators to do things such as create, update, move, and delete files.
it's possible to delete files in an NX generator.
I have the following code that generates a node application but I want to delete some of the generated files.
import { readProjectConfiguration, Tree } from '#nrwl/devkit';
import { applicationGenerator } from '#nrwl/node'
export interface Schema {
name: string
}
export default async function (tree: Tree, schema: Schema) {
// create node application with name `schema.name`
await applicationGenerator(tree, {
name: schema.name
})
const projectRoot = readProjectConfiguration(tree, schema.name).sourceRoot
if (!projectRoot) throw new Error(`${schema.name} is not a project found in project configuration`)
// here I want to delete generated files:
// apps/myapp/src/app/.gitkeep
// apps/myapp/src/assets/.gitkeep
// apps/myapp/src/environments/environment.prod.ts
// apps/myapp/src/environments/environment.ts
}
Output
CREATE apps/myapp/src/app/.gitkeep
CREATE apps/myapp/src/assets/.gitkeep
CREATE apps/myapp/src/environments/environment.prod.ts
CREATE apps/myapp/src/environments/environment.ts
CREATE apps/myapp/src/main.ts
CREATE apps/myapp/tsconfig.app.json
CREATE apps/myapp/tsconfig.json
CREATE apps/myapp/project.json
CREATE apps/myapp/.eslintrc.json
CREATE apps/myapp/jest.config.ts
CREATE apps/myapp/tsconfig.spec.json
What API is used for deleting files?
Turned out to be very simple.
tree.delete(filePath)
In my case,
...
// Delete generated files:
// apps/myapp/src/app/.gitkeep
// apps/myapp/src/assets/.gitkeep
// apps/myapp/src/environments/environment.prod.ts
// apps/myapp/src/environments/environment.ts
tree.delete(joinPathFragments(projectRoot, 'src/app'))
tree.delete(joinPathFragments(projectRoot, 'src/assets'))
tree.delete(joinPathFragments(projectRoot, 'src/environments'))

How to get a list of main folders inside root directory in the S3 bucket? Amazon S3

I am having a S3 bucket with folders and sub-folders containing files, I am trying to list all the folder names inside root directory. I dont want to list the sub-folders and files.
ex:
Test/Test1
Test/Test2
Test/Test3
Test/Test4
etc...
I have tried the following code which lists all the keys inside root directory
getListAllKeys(rootDirectory) {
const containedEntities = {
files: [],
directories: []
};
const params = {
Bucket: this.bucket,
Prefix: Test/,
Delimiter: Test/
};
return new Promise((resolve, reject) => this._listAllKeys(
resolve,
reject,
params,
containedEntities
));
}
_listAllKeys(resolve, reject, params, containedEntities) {
this.s3.listObjectsV2(params).promise()
.then(({
Contents, IsTruncated, NextContinuationToken,
}) => {
// filter files and directories
Contents.forEach((item) => {
containedEntities[item.Key.endsWith('/') ? 'directories' : 'files'].push(item);
});
// fetch further info if response is truncated
if (IsTruncated) {
this._listAllKeys(
resolve,
reject,
Object.assign(params, { ContinuationToken: NextContinuationToken }),
containedEntities
);
} else {
resolve(containedEntities);
}
})
.catch(reject);
}
I am using AWS-SDK package. Please let me know how to list only the main folders
To obtain a list of directories, specify Delimiter='/' (with no Prefix specified).
A list of directories at the top (root) level will be returned in a list called CommonPrefixes.
It also works for sub-folders by specifying a Prefix.

Generate new page after slug

I am building a NextJS application, currently I am using getStaticPaths and getStaticProps to build the static pages, doing the necessary requests for them.
So I want to build all the pages following this url: challenge/[slug]/ and for each slug that corresponds to an id I want to have a applications page like this: challenge/[slug]/applications to archive this I builded a file [...slug] inside /pages/challenge
Inside that file I have the following code to handle the static generation:
export async function getStaticPaths() {
const response: any = await getPrograms()
const paths = response.results.map(result => {
return { params: { slug: [result.id.toString()] } }
})
return { paths, fallback: true }
}
export async function getStaticProps({ params }) {
const res = await getProgram(params.slug[0])
const stages = await getStages(params.slug[0])
return { props: { program: res, stages: stages }, revalidate: 1 }
}
this solution works for /challenge/[slug], but the /challenge/[slug]/applications receives a 404, how can I render a specific application page for the slug?
I tried to add a second position to the slug array, but if I do it I can just render /challenge/[slug]/applications and not /challenge/[slug]
Any advice?
Thanks!
Firstly, You need to create a FOLDER named [slug]. Then, Create a FILE named applications.js. Lastly, copy and paste that code into this page.
__ challenge
|__ [slug]
|__ applications
In this page you can get or set slug as your desired parameter.

Yeoman generator add a new file generated exsiting project

I've yeoman generator which generate a simple sproject successfully.
I want that after the project generation, in latter time that the use will have the ability to generate a new file deployment.yaml under the app folder, however it needs to read some data from the main generator
for example appName as the sub-generator needs to generate a new file
inside the generated application.
e.g. yo tdk
This command generates a new project
And when I run yo tdk:event (or something similar) it will generate a new file inside the project app folder
For illustration I've created this very simple generator
const Generator = require("yeoman-generator");
module.exports = class extends Generator {
prompting() {
this.props = {
appName: "my-app",
srvName: "my-service"
};
const prompts = [
{
name: "appName",
message: "Project name: ",
type: "input",
default: this.props.appName
},
{
name: "srvName",
message: "Service name: ",
type: "input",
default: this.props.srvName
}
];
return this.prompt(prompts).then(props => {
this.props = props;
});
}
writing() {
this.fs.copyTpl(
this.templatePath("app"),
this.destinationPath(this.props.appName),
this.props
);
}
};
This generator have two simple question
app name
service name
And it will generate a project like
myapp /root
-app /folder
- service.yaml /single file at the project generation
The generated service.yaml looks like following:
apiVersion: v1
kind: Service
metadata:
name: <%= appName %>
spec:
selector:
app: <%= srvName %>
ports:
- protocol: TCP
port: 80
Now after the generation of the project with this service.yaml file
I want in latter time (after the project generation)to add new file deployment.yaml under the app folder
deployment.yaml
apiVersion: v1
kind: Deployment
metadata:
name: <%= appName %> //this is the appname from the project generation
spec:
replicas: <%= replica %>
selector:
app: <%= srvName %>
The appName & srvName are coming from the main generator,
(I saw that there is option to share data between sub generator
https://yeoman.io/authoring/storage.html , not sure how to share this between generators )
and the replica should come from the new/sub generator
This is the project structure after the generation
myapp /root
-app /folder
- service.yaml /single file at the project generation
- deployment.yaml / new file added to the project under app folder
Like user start another generator/sub and have a new question e.g. how much replicas do you want? and then generates the file.
How can I do it ?
update
This is my project strucutre
myapp
- node_modules
- package.json //here I declare the main-generator command -> tdk
- generators
-- app
---index.ts
--deployment
---index.ts
---package.json //here I declare the sub-generator command -> deploy
- node_modules
- package.json
-.yo-rc.json //here I see the data that I keep via config.set api
Update
When I call to the sub generator via program like
const yeoman = require('yeoman-environment');
const env = yeoman.createEnv();
env.lookup(function () {
env.run("tdk:deploy", {
replicas: 100
}, (err) => {
console.log("done", err);
});
});
I got error:
out from config undefined : undefined //the undefind is from the console in the sub-generator
done TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
at validateString (internal/validators.js:125:11)
at Object.join (path.js:1037:7)
I put a console.log in the subgenerator code like
initializing() {
this.srvName = this.config.get("srvName");
this.appName = this.config.get("appName");
console.log("out from config", this.srvName, ":", this.appName);
}
And when I run the subgenerator I got empty config ( from the .yo-rc.json)
while checking the .yo-rc.json . I was able to see the entry from the main generator, the data was stored but when I run it from the program it doesnt find it...any idea ?
This is the link for both project (very basic yeoman generator which demonstrate the point) just need to run npm install for both projects
and for the generator run also npm link.
At the end: a project should be generated with two files
1. service.yaml // generated from the main generator
2. deployment.yaml - // generated from sub generator with the properties from the main & sub generator
currently, the deployment.yaml file is not generated
https://drive.google.com/drive/folders/1kBnZxpVcRR9qhGZagVtod7W4wFmt73C6
1 . generator-tdk - Generator and sub-generator
2. yeomanEnv - The code which is running the sub-generator to create the file inside the generated project
What am I doing wrong ? :(
if there is a way from the sub-generator to read the .yo-rc.json , it can help
You can set the values to config inside configuring of the main generator like this:
configuring() {
this.config.set('appName', this.props.appName);
this.config.set('srvName', this.props.srvName);
}
and read the values inside the sub-generators:
initializing() {
this.srvName = this.config.get("srvName");
this.appName = this.config.get("appName");
}
So you'll have access to these values via this.srvName and this.appName upon writing.
Example code:
app/index.js:
const Generator = require("yeoman-generator");
module.exports = class extends Generator {
prompting() {
this.props = {
appName: "my-app",
srvName: "my-service",
};
const prompts = [
{
name: "appName",
message: "Project name: ",
type: "input",
default: this.props.appName,
},
{
name: "srvName",
message: "Service name: ",
type: "input",
default: this.props.srvName,
},
];
return this.prompt(prompts).then((props) => {
this.props = props;
});
}
configuring() {
this.config.set('appName', this.props.appName);
this.config.set('srvName', this.props.srvName);
}
writing() {
this.fs.copyTpl(
this.templatePath("app"),
this.destinationPath(this.props.appName),
this.props
);
}
};
deploy/index.js:
const Generator = require("yeoman-generator");
module.exports = class extends Generator {
initializing() {
this.srvName = this.config.get("srvName");
this.appName = this.config.get("appName");
}
prompting() {
this.props = {
replicas: 0,
};
const prompts = [
{
name: "replica",
message: "how much replicas do you want?",
type: "input",
default: this.props.replicas,
},
];
return this.prompt(prompts).then((props) => {
this.props = props;
});
}
writing() {
this.fs.copyTpl(
this.templatePath("deploy"),
this.destinationPath(this.appName),
{
srvName: this.srvName,
appName: this.appName,
...this.props,
}
);
}
};
and commands:
yo <name for the main project generation
yo <name>:deploy to ask for replicas and create deployment.yaml
To execute the sub-generator without the use of yo:
var yeoman = require("yeoman-environment");
var env = yeoman.createEnv();
env.lookup(function () {
env.run("<name>:deploy", {
replicas: 100
}, (err) => {
console.log("done", err);
});
});
and a sample sub-generator that skips question if values are passed via options (deploy/index.js):
const Generator = require("yeoman-generator");
module.exports = class extends Generator {
initializing() {
this.srvName = this.config.get("srvName");
this.appName = this.config.get("appName");
}
prompting() {
this.props = {
replicas: 0,
};
const prompts = [
{
name: "replicas",
message: "which app to generate?",
type: "input",
default: this.props.replicas,
when: !this.options.replicas, // disable the question if it's found in options
},
];
return this.prompt(prompts).then((props) => {
this.props = props;
// set values from options (if found)
this.props.replicas = this.options.replicas || this.props.replicas;
});
}
writing() {
this.fs.copyTpl(
this.templatePath("deploy"),
this.destinationPath(this.appName),
{
srvName: this.srvName,
appName: this.appName,
...this.props,
}
);
}
};

Gatsby dynamic content and schema stitching

I have the following graphql query that is executing at build time from my gatsby-node.js file that I'm using to bring in my article data. Is there a way to use Apollo/Axios to retrieve new articles without having to rebuild the site essentially rehydrating my site in between builds? Any help is greatly appreciated!!
I'm using Gatsby v2, Drupal as my CMS and GraphQL.
exports.createPages = ({ graphql, actions }) => {
const { createPage } = actions;
const blogTemplate = path.resolve('src/templates/blog-post.js');
return graphql(`
{
blog: allNodeArticle{
edges {
node {
id
path {
alias
}
}
}
}
`
).then(result => {
if (result.errors) {
Promise.reject(result.errors);
}
// Create blog pages
result.data.blog.edges.forEach(({ node }) => {
createPage({
path: node.path.alias,
component: blogTemplate,
context: {
alias: node.path.alias,
},
});
});
});
}
I would like to merge in the new data as it becomes available while keeping older data completely static (massive benefit of Gatsby)

Categories