How to validate multipart file on AdonisJS? - javascript

I am using Adonis.js in the latest version but cannot validate...
Already tried
request.multipart.file('avatar', {
types: ['jpeg', 'jpg', 'png'], // I already tried -> type: ['image'] types: ['image'],
size: "4mb"
}, async file => {
await Drive.put(key, file.stream)
})
.../Validators/changeAvatar.js
'use strict'
class UserChangeAvatar {
get rules() {
return {
avatar: 'required|file|file_ext:png,jpg,jpeg,svg'
}
}
}
module.exports = UserChangeAvatar
Nothing works, the code lets you upload any type of file, like .pdf or .mp4
There's nothing in the Adonis.js documentation talking about it either.
Package version
Version 4.1
"#adonisjs/framework": "^5.0.9"
Node.js and npm version
NODE - v10.15.0
NPM - 6.10.1

The validation rules do not work for the multipart file upload on adonis. You need to do manual validation. For example:
// Helper function
function fileStreamValidation(file, validationRules) {
const validationErrors = []
if (!RegExp(/^[0-9a-zA-Z_\-.]+$/).test(file._clientName)) {
validationErrors.push(
`${file._clientName}'s name should only contain alphanumeric, underscore, dot, hypen`
)
}
if (validationRules.extnames && validationRules.extnames.length) {
const [_, fileExtension] = file._clientName.split(/\.(?=[^.]+$)/)
if (!validationRules.extnames.includes(fileExtension)) {
validationErrors.push(`${file._clientName}'s extension is not acceptable`)
}
}
if (validationRules.maxFileSizeInMb) {
if (file.stream.byteCount > validationRules.maxFileSizeInMb * 1000000) {
validationErrors.push(`${file._clientName}'s size exceeded limit`)
}
}
return validationErrors
}
/* validation in controller */
const validationOptions = {
extnames: ['in', 'out'],
maxFileSizeInMb: parseInt(Env.get('MAX_FILE_SIZE_IN_MB'))
}
request.multipart.file('datasets[]', {}, async file => {
const errors = fileStreamValidation(file, validationOptions)
})

Related

How to change file reference strings to use generated hashes from copy file webpack plugin

I'm copying all my assets to my build directory using copy webpack plugin. It generates all the files with the hashes properly but is there a simple way of changing any of these file references in code with the new hashed filename?
new CopyWebpackPlugin({
patterns: [
{ from: 'src/assets', to: '[path][name].[contenthash][ext]' },
],
}),
Im currently using strings like below to load assets into phaser, using a query param to break the cache but i want the url to be updated with the new hash filename so I don't need to use the query param to bust the server cache and can take advantage of server caching.
{
key: 'atlas',
url: `assets/img/atlas.json?t=${new Date().getTime().toString()}`,
path: './assets/img/'
},
{
key: 'atlas',
url: `assets/img/atlas.json`,
path: './assets/img/'
},
so Im hoping the above will look like this in the output after webpack has run
{
key: 'atlas',
url: `assets/img/atlas.{generatedHash}.json`,
path: './assets/img/'
},
Edit:
Okay so I accomplished the above by using webpack-asset-manifest
new ManifestPlugin({
// publicPath: 'assets/',
}),
and then having a function for getting my urls from the generated json after it is loaded
protected getAssetFromKey(key: string | Array<string>): string | Array<string> {
if (Array.isArray(key)) {
let urls = new Array<string>();
key.forEach((urlKey: string) => {
urls.push(this.assetKeys[urlKey]);
});
return urls;
} else {
return this.assetKeys[key];
}
}
But now I have hit an issue where the json atlas files are pointing to the old images and there seems to be no easy way to edit these. Im thinking of something like string replace loader but Im wondering if there is a better way and I am unsure of how to replace the string with a value from the manifest.json that is exported by webpack.
Okay So I figured this out by adding the manifest plugin to my webpack config which generates a json file with all the original file names as keys and all the new hashed filenames as values.
new ManifestPlugin({
}),
and then I added in a compiler hook in the plugins area that does reads all the atlas files and replaces the strings in them, as well as any references to the assets in the compiled code.
https://webpack.js.org/api/compiler-hooks/
{
apply: (compiler) => {
compiler.hooks.done.tap("update links", (stats) => {
Fs.readJson('dist/assets-manifest.json').then( (value) => {
let keys = Object.keys(value)
let files = Object.values(value);
files.forEach( (file, index) => {
if( file.includes('json')) {
let splitString = keys[index].split('/');
let findFile = splitString[splitString.length-1].replace('.json', '');
console.log(`find file- ${findFile}`);
let replaceWithString = '';
let replaceString = ''
for( let i =0 ; i < keys.length; i++) {
if( keys[i].includes(`${findFile}`) && keys[i].includes('.webp') ) {
console.log(keys[i]);
let splitFiles = files[i].split('/');
let splitKeys = keys[i].split('/');
replaceWithString = splitFiles[splitFiles.length-1];
replaceString = splitKeys[splitKeys.length-1];
break;
}
}
console.log( `REPLACE WITH STRING = ${replaceWithString}`)
console.log(`reading file-${file}`);
Fs.readJson(`dist/${file}`).then( (val) => {
let stringJson = JSON.stringify(val);
console.log(`replacing ${replaceString} with ${replaceWithString}`);
let result = stringJson.replace(replaceString, replaceWithString);
let outputJson = JSON.parse(result);
Fs.writeJson(`dist/${file}`, outputJson, 'utf8').then( () => {
console.log( `!!!!! SUCCESS !!!!!`);
});
});
}
});
files.forEach( (file) => {
if( file.includes('.js') && !file.includes('json') ) {
console.log('FILE: ' + file)
Fs.content(`dist/${file}`).then( ( val) => {
keys.forEach( (key,index) => {
if( key.includes('assets/')) {
val = val.replaceAll(key, files[index]);
console.log(`REPLACING: ${key} with ${files[index]} in ${file}`)
}
});
Fs.writeFile(`dist/${file}`, val).then( () => {
console.log("--SUCCESS---")
});
});
}
})
}).catch( (err) => {
console.log(`error ${err}`);
})
});
}
},

How do I obfuscate Javascript in Ember?

Does anyone know how to use the javascript-obfuscator (or similar) in Ember ?
I guess it needs to be called inside ember-cli-build.js but I don't know where and how.
Thank you in advance for any help :)
I don't think there's a really straightforward answer to this. If you're running on embroider, then all your Javascript will be handled by webpack, so you could try using webpack-obfuscator -- in your ember-cli-build.js something like
return require('#embroider/compat').compatBuild(app, Webpack, {
plugins: [
new WebpackObfuscator(/*whatever args*/)
],
rules: [
{
test: /\.js$/,
enforce: 'post',
use: {
loader: WebpackObfuscator.loader,
}
}
]
});
The other options I know of would be to write a broccoli plugin. What you're doing is analogous to what ember-cli-terser does -- post-process Javascript files before they get concatenated together -- so you could use that as reference.
You would need to write a broccoli plugin that actually performs the transformations (the analog is broccoli-terser-sourcemap) and an Ember addon to hook it into ember-cli's build pipeline (the analog is ember-cli-terser).
Broccoli plugin
Looking at broccoli-terser-sourcemap's index.js, which is only 114 lines of code, I would think you could adapt it to something like this:
module.exports = class TerserWriter extends Plugin {
constructor(_inputNodes, options = {}) {
let inputNodes = Array.isArray(_inputNodes) ? _inputNodes : [_inputNodes];
super(inputNodes, {
name: options.name,
annotation: options.annotation,
needsCache: false,
});
this.options = defaults(options, {
obfuscator: {},
});
let exclude = this.options.exclude;
if (Array.isArray(exclude)) {
this.excludes = new MatcherCollection(exclude);
} else {
this.excludes = MatchNothing;
}
}
async build() {
let pendingWork = [];
this.inputPaths.forEach(inputPath => {
walkSync(inputPath).forEach(relativePath => {
if (relativePath.slice(-1) === '/') {
return;
}
let inFile = path.join(inputPath, relativePath);
let outFile = path.join(this.outputPath, relativePath);
fs.mkdirSync(path.dirname(outFile), { recursive: true });
if (this._isJSExt(relativePath) && !this.excludes.match(relativePath)) {
pendingWork.push(() => this.processFile(inFile, outFile, relativePath, this.outputPath));
} else {
symlinkOrCopy.sync(inFile, outFile);
}
});
});
for (let fn of pendingWork) {
await fn();
}
}
_isJSExt(relativePath) {
return relativePath.slice(-3) === '.js' || relativePath.slice(-4) === '.mjs';
}
async processFile(inFile, outFile, relativePath, outDir) {
let input = await readFile(inFile).toString();
let result = obfuscate(input, this.options.obfuscator);
await writeFile(outFile, result.getObfuscatedCode());
}
};
You could also do the worker pooling this that broccoli-terser-sourcemaps does, and if you care about source maps you'd need to handle them as well, but broccoli-terser-sourcemaps does just that, so you could use it as reference.
ember-cli addon
ember-cli-terser has even less code -- looking at its index.js, you could adapt it to something like
'use strict';
module.exports = {
name: require('./package').name,
included(app) {
this._super.included.apply(this, arguments);
let defaultOptions = {
enabled: app.env === 'production',
obfuscator: {
// default `javascript-obfuscator` options
},
};
let addonOptions = app.options['ember-cli-obfuscator'];
this._obfuscatorOptions = Object.assign({}, defaultOptions, addonOptions);
},
postprocessTree(type, tree) {
if (this._obfuscatorOptions.enabled === true && type === 'all') {
// Import the plugin code above
const Obfuscator = require('./broccoli-obfuscator');
return new Obfuscator(tree, this._obfuscatorOptions);
} else {
return tree;
}
}
};
Then you'd have to install the above addon in your app (it could be an in-repo addon), and it should do its thing!
This would definitely take some doing, but what you're doing is so similar to what ember-cli-terser is doing, just using the obfuscator API instead of the terser API, that you have a really good starting point.
BUT, if embroider is an option for you, I'd definitely try that route first because it might just be a matter of configuration, rather than writing a bunch of code.

How to test file inputs with Cypress?

How can I write an e2e test of flow that requires interaction with the file Input DOM element?
If it's a text input I can interact with it (check value, set value) etc as its a DOM component. But If I have a File Input element, I am guessing that the interaction is limited till I can open the dialog to select a File. I can't move forward and select the file I want to upload as the dialog would be native and not some browser element.
So how would I test that a user can correctly upload a file from my site? I am using Cypress to write my e2e tests.
it('Testing picture uploading', () => {
cy.fixture('testPicture.png').then(fileContent => {
cy.get('input[type="file"]').attachFile({
fileContent: fileContent.toString(),
fileName: 'testPicture.png',
mimeType: 'image/png'
});
});
});
Use cypress file upload package: https://www.npmjs.com/package/cypress-file-upload
Note: testPicture.png must be in fixture folder of cypress
For me the easier way to do this is using this cypress file upload package
Install it:
npm install --save-dev cypress-file-upload
Then add this line to your project's cypress/support/commands.js:
import 'cypress-file-upload';
Now you can do:
const fixtureFile = 'photo.png';
cy.get('[data-cy="file-input"]').attachFile(fixtureFile);
photo.png must be in cypress/fixtures/
For more examples checkout the Usage section on README of the package.
Since 9.3.0 you can use selectFile.
cy.get('input[type=file]').selectFile('cypress/fixtures/file.json')
See:
https://github.com/cypress-io/cypress/issues/170
https://cypress.io/blog/2022/01/18/uploading-files-with-selectfile/
https://docs.cypress.io/guides/references/migration-guide#Migrating-from-cypress-file-upload-to-selectFile
With this approach/hack you can actually make it:
https://github.com/javieraviles/cypress-upload-file-post-form
It is based on different answers from the aformentioned thread https://github.com/cypress-io/cypress/issues/170
First scenario (upload_file_to_form_spec.js):
I want to test a UI where a file has to be selected/uploaded before
submitting the form.
Include the following code in your "commands.js" file within the cypress
support folder, so the command cy.upload_file() can be used from any test:
Cypress.Commands.add('upload_file', (fileName, fileType, selector) => {
cy.get(selector).then(subject => {
cy.fixture(fileName, 'hex').then((fileHex) => {
const fileBytes = hexStringToByte(fileHex);
const testFile = new File([fileBytes], fileName, {
type: fileType
});
const dataTransfer = new DataTransfer()
const el = subject[0]
dataTransfer.items.add(testFile)
el.files = dataTransfer.files
})
})
})
// UTILS
function hexStringToByte(str) {
if (!str) {
return new Uint8Array();
}
var a = [];
for (var i = 0, len = str.length; i < len; i += 2) {
a.push(parseInt(str.substr(i, 2), 16));
}
return new Uint8Array(a);
}
Then, in case you want to upload an excel file, fill in other inputs and submit the form, the test would be something like this:
describe('Testing the excel form', function () {
it ('Uploading the right file imports data from the excel successfully', function() {
const testUrl = 'http://localhost:3000/excel_form';
const fileName = 'your_file_name.xlsx';
const fileType = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet';
const fileInput = 'input[type=file]';
cy.visit(testUrl);
cy.upload_file(fileName, fileType, fileInput);
cy.get('#other_form_input2').type('input_content2');
.
.
.
cy.get('button').contains('Submit').click();
cy.get('.result-dialog').should('contain', 'X elements from the excel where successfully imported');
})
})
Testing File Input elements is not yet supported in Cypress. The only way to test File Inputs is to:
Issue native events (which Cypress has on their Roadmap).
Understand how your application handles file uploads with File API and then stub it out. It's possible but not generic enough to give any specific advice on.
See this open issue for more detail.
In my case I had client & server side file validation to check if the file is JPEG or PDF. So I had to create a upload command which would read the file in binary from Fixtures and prepare a blob with the file extension.
Cypress.Commands.add('uploadFile', { prevSubject: true }, (subject, fileName, fileType = '') => {
cy.fixture(fileName,'binary').then(content => {
return Cypress.Blob.binaryStringToBlob(content, fileType).then(blob => {
const el = subject[0];
const testFile = new File([blob], fileName, {type: fileType});
const dataTransfer = new DataTransfer();
dataTransfer.items.add(testFile);
el.files = dataTransfer.files;
cy.wrap(subject).trigger('change', { force: true });
});
});
});
then use it as
cy.get('input[type=file]').uploadFile('smiling_pic.jpg', 'image/jpeg');
smiling_pic.jpg will be in fixtures folder
The following function works for me,
cy.getTestElement('testUploadFront').should('exist');
const fixturePath = 'test.png';
const mimeType = 'application/png';
const filename = 'test.png';
cy.getTestElement('testUploadFrontID')
.get('input[type=file')
.eq(0)
.then(subject => {
cy.fixture(fixturePath, 'base64').then(front => {
Cypress.Blob.base64StringToBlob(front, mimeType).then(function(blob) {
var testfile = new File([blob], filename, { type: mimeType });
var dataTransfer = new DataTransfer();
var fileInput = subject[0];
dataTransfer.items.add(testfile);
fileInput.files = dataTransfer.files;
cy.wrap(subject).trigger('change', { force: true });
});
});
});
// Cypress.Commands.add(`getTestElement`, selector =>
// cy.get(`[data-testid="${selector}"]`)
// );
Also based on previously mentioned github issue, so big thanks to the folks there.
The upvoted answer worked initially for me, but I ran into string decoding issues trying to handle JSON files. It also felt like extra work having to deal with hex.
The code below handles JSON files slightly differently to prevent encode/decode issues, and uses Cypress's built in Cypress.Blob.base64StringToBlob:
/**
* Converts Cypress fixtures, including JSON, to a Blob. All file types are
* converted to base64 then converted to a Blob using Cypress
* expect application/json. Json files are just stringified then converted to
* a blob (prevents issues with invalid string decoding).
* #param {String} fileUrl - The file url to upload
* #param {String} type - content type of the uploaded file
* #return {Promise} Resolves with blob containing fixture contents
*/
function getFixtureBlob(fileUrl, type) {
return type === 'application/json'
? cy
.fixture(fileUrl)
.then(JSON.stringify)
.then(jsonStr => new Blob([jsonStr], { type: 'application/json' }))
: cy.fixture(fileUrl, 'base64').then(Cypress.Blob.base64StringToBlob)
}
/**
* Uploads a file to an input
* #memberOf Cypress.Chainable#
* #name uploadFile
* #function
* #param {String} selector - element to target
* #param {String} fileUrl - The file url to upload
* #param {String} type - content type of the uploaded file
*/
Cypress.Commands.add('uploadFile', (selector, fileUrl, type = '') => {
return cy.get(selector).then(subject => {
return getFixtureBlob(fileUrl, type).then(blob => {
return cy.window().then(win => {
const el = subject[0]
const nameSegments = fileUrl.split('/')
const name = nameSegments[nameSegments.length - 1]
const testFile = new win.File([blob], name, { type })
const dataTransfer = new win.DataTransfer()
dataTransfer.items.add(testFile)
el.files = dataTransfer.files
return subject
})
})
})
})
You can do it with new Cypress command:
cy.get('input[type=file]').selectFile('file.json')
This is now available within Cypress library itself from version 9.3 and above. Follow the migration guide on how to move from cypress-file-upload plugin to Cypress .selectFile() command:
Migrating-from-cypress-file-upload-to-selectFile
in your commands.ts file within your test folder add:
//this is for typescript intellisense to recognize new command
declare namespace Cypress {
interface Chainable<Subject> {
attach_file(value: string, fileType: string): Chainable<Subject>;
}
}
//new command
Cypress.Commands.add(
'attach_file',
{
prevSubject: 'element',
},
(input, fileName, fileType) => {
cy.fixture(fileName)
.then((content) => Cypress.Blob.base64StringToBlob(content, fileType))
.then((blob) => {
const testFile = new File([blob], fileName);
const dataTransfer = new DataTransfer();
dataTransfer.items.add(testFile);
input[0].files = dataTransfer.files;
return input;
});
},
);
Usage:
cy.get('[data-cy=upload_button_input]')
.attach_file('./food.jpg', 'image/jpg')
.trigger('change', { force: true });
another option is to use cypress-file-upload, which is buggy in version 4.0.7 (uploads files twice)
cy.fixture("image.jpg").then((fileContent) => {
cy.get("#fsp-fileUpload").attachFile({
fileContent,
fileName: "image",
encoding: "base64",
mimeType: "image/jpg",
});
});
Here is the multiple file upload version:
Cypress.Commands.add('uploadMultiFiles',(args) => {
const { dataJson, dirName, inputTag, mineType} = args
const arr = []
dataJson.files.forEach((file, i) => {
cy.fixture(`${ dirName + file }`).as(`file${i}`)
})
cy.get(`${inputTag}`).then(function (el) {
for(const prop in this) {
if (prop.includes("file")) {
arr.push(this[prop])
}
}
const list = new DataTransfer()
dataJson.files.forEach((item, i) => {
// convert the logo base64 string to a blob
const blob = Cypress.Blob.base64StringToBlob(arr[i], mineType)
const file = new FileCopy([blob], `${item}`, { type: mineType }, `${ dirName + item }`)
const pathName = dirName.slice(1)
file.webkitRelativePath = `${ pathName + item}`
console.log(file)
list.items.add(file)
})
const myFileList = list.files
el[0].files = myFileList
el[0].dispatchEvent(new Event('change', { bubbles: true }))
})
})
The usage:
First, prepare a data.json file inside the fixtures folder, example:
data.json
{
"files":[
"1_TEST-JOHN-01.jpeg",
"2_TEST-JOHN-01.jpeg",
"3_TEST-JOHN-01.jpeg",
"4_TEST-JOHN-01.jpeg",
"5_TEST-JOHN-01.jpeg",
"6_TEST-JOHN-01.jpeg",
"7_TEST-JOHN-01.jpeg",
"8_TEST-JOHN-01.jpeg",
"9_TEST-JOHN-01.jpeg",
"10_TEST-JOHN-01.jpeg"
]
}
Second, import the json data into your spec.js
import data from '../fixtures/data.json'
Third, Write a class to extend the File web API object with the functions to set and get webkitRelativePath value
class FileCopy extends File {
constructor(bits, filename, options) {
super(bits, filename, options)
let webkitRelativePath
Object.defineProperties(this, {
webkitRelativePath : {
enumerable : true,
set : function(value){
webkitRelativePath = value;
},
get : function(){
return webkitRelativePath;
}
},
});
}
}
Finally, call the cmd in the spec.js
cy.uploadMultiFiles(
{
dataJson:data, // the data.json you imported.
dirName:"/your/dirname/",
inputTag:"input#upload",
mineType:"image/jpeg"
}
)
if your file input display: none; use
cy.get('[data-type=inputFile]').selectFile('cypress/fixtures/avatar.jpg', { force: true })
else
cy.get('[data-type=inputFile]').selectFile('cypress/fixtures/avatar.jpg')

How to get mimeType from Cordova File Transfer Plugin?

I am developing hybrid mobile application.
In one of the scenario we need to fetch mimeType from a file when we select or upload a file.
I am using apache FileTransfer.
window.resolveLocalFileSystemURL(fileURI , resolveOnSuccess, resolveOnFail)
you can get it from cordova File plugin.
$cordovaFile.checkFile(uri, '')
.then(function(entry) {
// success
var name = entry.name;
entry.file(function(data) {
// get mime type
var mime = data.type;
alert(mime);
})
}, function(error) {
// error
// show toast
});
I got it working like this in TypeScript and Angular 2:
this._File.resolveLocalFilesystemUrl(somefileUri).then((entry: Entry) => {
if (entry) {
var fileEntry = entry as FileEntry;
fileEntry.file(success => {
var mimeType = success.type;
}, error => {
// no mime type found;
});
}
});
file-transfer does not expose mimeType and other FileUploadOptions params.
Mimetype autodetection is only supported for uploads in Windows plugin code.
And here is a Jira ticket for this feature CB-5946 - it also has some suggestions on Android implementation.
In Angular 2 I use this:
export class Plugins {
albums = {
open () : Promise<any> {
return ImagePicker.getPictures({
quality: 100,
maximumImagesCount: 1,
}).then((imgUrls) => {
return imgUrls;
}, (err) => {
if(err.error == "cordova_not_available") {
alert("Cordova is not available, please make sure you have your app deployed on a simulator or device");
} else {
console.log("Failed to open albums: " + err.error);
}
});
},
}
...
#Component({
templateUrl: 'build/pages/home/home.html',
directives: [UploadButton]
})
export class HomePage implements OnInit {
openAlbums = (): void => {
var $self = this;
this._plugins.albums.open().then((imgUrls) => {
imgUrls.forEach((imageUrl: string): void => {
if (imageUrl) {
window.resolveLocalFileSystemURL(imageUrl, function (entry: FileEntry) {
entry.file(file=> {
console.log('mimeType', file.type);
}, ((error:FileError) => console.log(error)));
});
}
});
}); };
resolveLocalFileSystemURL gives back through the success callback an Entry which I had to cast to FileEntry to get access to the file method which gives back a File which extends Blob that has the mime type property.

How can I read all JSON files from a folder in Meteor?

I want to read all json files inside a folder in my Meteor application.
I have the following structure:
/server
-- /methods
-- -- file1.json
-- -- file2.json
I try to read all JSON files using the following code:
var fs = Npm.require('fs');
var path = Npm.require('path');
var base = path.resolve('.');
try {
var files = fs.readdirSync(base + '/methods/*.json');
console.log(files);
} catch (e) {
console.dir(e)
}
But this doesn't work and it shows me an error saying that the directory or file doesn't exist.
Am I making some mistake? Or is there another way to do that?
First of all, please be careful with finding Meteor's project root, because the outcome of path.resolve('.') or even process.env.PWD may change in different deployment setups.
Secondly, the argument path of fs.readdirSync(path) needs a directory. As a result, the proper call would be var files = fs.readdirSync(base + '/server/methods/');.
However, I recommend to use Assets. Just move your JSON files to your private directory and access them on the server via Assets.getText(assetPath, [asyncCallback]) or Assets.getBinary(assetPath, [asyncCallback]).
For example:
if (Meteor.isServer) {
Meteor.startup(function() {
var example1 = JSON.parse(Assets.getText('methods/example1.json'));
var example2 = JSON.parse(Assets.getText('methods/example2.json'));
console.log(example1);
console.log(example2);
});
}
If you want to read all JSON files, you may need the following workaround:
if (Meteor.isServer) {
Meteor.startup(function() {
var exec = Npm.require('child_process').exec;
var files = [],
fileNames = [];
exec('ls -m assets/app/methods | tr -d \' \n\' ', Meteor.bindEnvironment(function(error, stdout, stderr) {
if (error !== null) {
console.log('exec error: ' + error);
}
fileNames = stdout.split(',');
/* Print all file names. */
console.log("File names:");
console.log(fileNames);
_.each(fileNames, function(fileName) {
/* Check if file has proper extension. */
if (fileName.split('.').pop() == 'json') files.push(JSON.parse(Assets.getText('methods/' + fileName)));
});
/* Print all JSON files. */
_.each(files, function(file) {
console.log(file);
});
}));
});
}
If you want to make the exec call synchronously, you may need to use Meteor.wrapAsync(func, [context]):
if (Meteor.isServer) {
var exec = Npm.require('child_process').exec;
var files = [], fileNames = [];
var execAsync = function (options, callback) {
console.log("execAsync()");
exec('ls -m assets/app/methods | tr -d \' \n\' ', Meteor.bindEnvironment(function (error, stdout, stderr) {
if (error !== null) {
console.log('exec error: ' + error);
}
fileNames = stdout.split(',');
/* Print all file names. */
console.log("File names:");
console.log(fileNames);
_.each(fileNames, function (fileName) {
/* Check if file has proper extension. */
if (fileName.split('.').pop() == 'json') files.push(JSON.parse(Assets.getText('methods/' + fileName)));
});
callback(null, options.callback);
}));
};
function postProcessing(callback) {
console.log("postProcessing()");
/* Print all JSON files. */
_.each(files, function (file) {
console.log(file);
});
callback();
}
Meteor.startup(function () {
/* Wrap asynchronous exec function, in order to call it in a synchronous style. */
var execSync = Meteor.wrapAsync(execAsync);
var refToPostProcessing = execSync({callback: postProcessing});
var postProcessingSync = Meteor.wrapAsync(refToPostProcessing);
postProcessingSync();
});
}
Here is my server output:
I20150919-09:27:09.189(2)? execAsync()
I20150919-09:27:09.210(2)? File names:
I20150919-09:27:09.213(2)? [ 'example1.json', 'example2.json' ]
I20150919-09:27:09.215(2)? postProcessing()
I20150919-09:27:09.217(2)? { name: 'Product',
I20150919-09:27:09.217(2)? properties:
I20150919-09:27:09.218(2)? { id:
I20150919-09:27:09.218(2)? { type: 'number',
I20150919-09:27:09.218(2)? description: 'Product identifier',
I20150919-09:27:09.218(2)? required: true },
I20150919-09:27:09.218(2)? name:
I20150919-09:27:09.218(2)? { description: 'Name of the product',
I20150919-09:27:09.219(2)? type: 'string',
I20150919-09:27:09.219(2)? required: true },
I20150919-09:27:09.219(2)? price: { type: 'number', minimum: 0, required: true },
I20150919-09:27:09.219(2)? tags: { type: 'array', items: [Object] } } }
I20150919-09:27:09.220(2)? { red: '#f00',
I20150919-09:27:09.221(2)? green: '#0f0',
I20150919-09:27:09.221(2)? blue: '#00f',
I20150919-09:27:09.221(2)? cyan: '#0ff',
I20150919-09:27:09.221(2)? magenta: '#f0f',
I20150919-09:27:09.221(2)? yellow: '#ff0',
I20150919-09:27:09.221(2)? black: '#000' }
Assuming you have the following structure:
your-meteor-project
├── .meteor
├── server
├── private
│ └── methods
│ └── example1.json
│ └── example2.json
└── …
Based on Matthias Eckhart's answer above, I wrote a method to load all csv files in a folder, convert the contents to JSON and return data to the client as an object with a child object per csv file. I post this here in case it helps somebody else: there are a couple of wrinkles using Assets and methods, and when converting csv to JSON.
import CSVToJSON from 'csvtojson';
if (Meteor.isServer) {
const getParameterFilenames = (options, callback) => {
// read the contents of the 'private/parameters' folder
const { exec } = Npm.require('child_process');
// "ls -m" will return directories as well as folders, so make sure to filter the results before loading files
exec('ls -m assets/app/parameters | tr -d \' \n\' ', Meteor.bindEnvironment((error, stdout, stderr) => {
if (error !== null) {
console.log(`Error in getParameterFilenames: ${error}`);
}
const filenames = stdout.split(',');
callback(null, filenames);
}));
};
Meteor.methods({
'loadParameters'() {
const syncFunc = Meteor.wrapAsync(getParameterFilenames);
const filenames = syncFunc({});
// load parameters from csv files in 'private/parameters'
// this will be assets/app/parameters in the built app
// csv file contains key / value pairs
// first row is key, second row is value
// first key must be propertyName which will be the key for this sheet's child object in parameters, e.g.:
/*
"propertyName","value1","value2"
"map",10,20
*/
const promises = [];
const filesData = [];
// although Assets.getText is used synchronously, the files must be retrieved before creating the promises
filenames.forEach((filename) => {
if (filename.split('.').pop() === 'csv') {
filesData.push(Assets.getText(`parameters/${filename}`));
}
});
filesData.forEach((data) => {
promises.push(CSVToJSON().fromString(data));
});
// Meteor will wait for Promise.all to resolve before returning the result to the client
return Promise.all(promises)
.then((results) => {
// aggregate results into an object
const parameters = {};
results.forEach((result) => {
const data = result[0];
const parameter = { ...data };
delete parameter.propertyName;
parameters[data.propertyName] = parameter;
});
return parameters;
});
},
});
}

Categories