Gulp get a ls of a directory - javascript

I have a directory assets with a number of images inside. Inside my JavaScript code, I have an array, with the same list of images, that I use to preload everything. It means I have twice the same information, the list of images, both in my JavaScript and in my filesystem. I would like to have a gulp task to list all those images, and populate the array in JavaScript, with something like:
var listFiles = require("list-files-in-folder");
var images = listFiles("./assets/");
Do you know of any plugin who does that? Or a simple way to implement it maybe?

Assuming you're talking about nodejs, take a look at this
var glob = require("glob")
// options is optional
glob("**/*.js", options, function (er, files) {
// files is an array of filenames.
// If the `nonull` option is set, and nothing
// was found, then files is ["**/*.js"]
// er is an error object or null.
})

Related

Deletion with API sometimes works and sometimes it does not

I am trying to delete files from folder with API call.
It seems to work randomly, some files get deleted, some don't.
The data structure is as follow:
Each folder object has files property.
In files_to_delete I have files to delete and index of folder in folders array.In API call I put whole folder objectI remove file with lodash remove function.
Any idea why it does not work?
angular.forEach(files_to_delete, function(file_to_delete) {
_.remove(folders[file_to_delete.index].files, function(file) {
if (file === file_to_delete) {
$http.put('/save_folders_file', folders[file_to_delete.index])
return (file === file_to_delete)
});
Your example isn't entirely clear, but it looks like you save the container the files are in repeatedly, each time you find a file to delete. That seems chaotic. It also assumes that Lodash modifies the container as it's doing the loop, but the documentation doesn't say it does, and a simple (perhaps naive) test suggests it doesn't:
var array = [1, 2, 3, 4];
var evens = _.remove(array, function(n) {
  console.log(n, JSON.stringify(array));
return n % 2 == 0;
});
console.log("array", JSON.stringify(array));
console.log("evens", JSON.stringify(evens));
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.11/lodash.min.js"></script>
When I run that, I see:
1 [1,2,3,4]
2 [1,2,3,4]
3 [1,2,3,4]
4 [1,2,3,4]
array [1,3]
evens [2,4]
...which suggests that it does the callbacks first, then goes back and removes the relevant entries.
Even if it did remove them as it went, though, those overlapping HTTP requests would bother me a lot.
Instead, I'd save the container once, when done, after all the files are removed:
angular.forEach(files_to_delete, function(file_to_delete) {
const removed = _.remove(folders[file_to_delete.index].files, function(file) {
return file === file_to_delete;
});
if (removed.length) {
$http.put('/save_folders_file', folders[file_to_delete.index])
}
});

Pass parameter to Gulp task in gulpfile.js (not command-line)

EDIT: As I wrote IN THE TITLE AND FROM THE BEGINNING, this is not about command-line parameters and is thus NOT A DUPLICATE. //EDIT
I have a Sass setup with an indefinite number of uniquely-designed pages (page_1, page_2, etc), each having their own sass/pages/page_1/page_1.scss file.
The pages all belong to the same website, and each page's sass file #imports the same set of files from a sass/includes folder.
With a basic gulp task watching sass/**/*, every page's styles get compiled anytime I make a change to any page's styles. Obviously this doesn't scale well.
I tried using gulp-watch, but it doesn't catch if changes are made to one of the included .scss files. It only catches changes made to the files that actually get compiled into an equivalent .css.
For the purposes of having my gulpfile be as DRY as possible, the best solution I could come up with was to maintain a basic array of folder names in gulpfile.js, and to loop through and watch each of them separately, using the same sass-compiling task for each folder.
var pageFolderNames = [
'page_1',
'page_2'
// etc
];
Then for the gulp task, I have:
gulp.task('watch_pages', function()
{
// Get array length
var numPages = pageFolderNames.length;
// Add a new watch task for each individual page
for (var i = 0; i < numPages; i++)
{
gulp.watch('sass/pages/' + pageFolderNames[i] + '/**/*.scss', ['sass_page']);
}
});
The (simplified) task that compiles sass:
// Task: Compile page-specific Sass
gulp.task('sass_page', function()
{
return gulp.src('sass/pages/' + pageFolderNames[i] +'/**/*.scss')
.pipe(plumber(plumberErrorHandler))
.pipe(sass(...))
.pipe(gulp.dest('css/pages/' + pageFolderNames[i]));
});
This approach (I know my JS-fu is weaksauce) results in an error:
'sass_page' errored after 71 μs
ReferenceError: i is not defined
Is there any way to pass parameters, such as i, to gulp tasks to get this working? Alternately, is there a better way to accomplish what I'm trying to do? I have a sneaking suspicion there is. :-/
I found out there is an on change event for gulp watch. So this might be what you're looking for:
var pagesDir = 'sass/pages/';
gulp.task('watch_pages', function() {
gulp.watch(pagesDir + '**/*')
.on("change", function(file) {
// absolute path to folder that needs watching
var changedDest = path.join(__dirname, pagesDir);
// relative path to changed file
var changedFile = path.relative(changedDest, file.path);
// split the relative path, get the specific folder with changes
var pageFolder = changedFile.split('\\')[0];
gulp.src(path.join(pagesDir, pageFolder) +'/**/*.scss')
.pipe(plumber(plumberErrorHandler))
.pipe(sass(...))
.pipe(gulp.dest('css/pages/' + pageFolder));
console.log(changedDest);
console.log(changedFile);
console.log(pageFolder);
});
});
Also, this way you don't need to declare the folder variables. If you add directories within the path being watched, it should pick it up and name the destination folder accordingly.
Theoretically the gulp task to compile sass should work within the watch task. I played around with the paths, and it seems to spitting them out. Let me know what happens, I can modify if necessary.
The required packages:
var gulp = require("gulp"),
path = require("path"),
rimraf = require("rimraf");
BTW, since you already have access to the file path, you can perhaps target the specific scss file instead of the whole directory.
As Brian answered, the best approach is to have one watcher. In the same way as the principle of delegation with dom event listeners. it's better, even in our case it can not really matter. We need our thing done. if the consumed resources and performance doesn't bother us. It's ok. But stay as Brian answered. one watcher is the best approach. then you've got to get the file that was changed. and from there you get your page folder. So for that i will not add a thing. Except that you don't necessarily need to use the .on("change". You can directly set the same parameter for your function as this example show:
watch('./app/tempGulp/json/**/*.json', function (evt) {
jsonCommentWatchEvt = evt
gulp.start('jsonComment')
})
evt here is what Brian set as file.
What i want to add, is about how to pass a parameter from your watcher to your task. For example about the i in your initial work. A way of doing that, that i see and use, is to set a global variable that hold the data wanted to be passed. So it's accessible in both blocks. You set the value in watcher just before starting the task. And then you use it in the task. A good practice, is to save it in a task local variable just at start. That way you avoid the problem that it can change by another watch handling triggering.
For a lively example. check my answer out here : https://stackoverflow.com/a/49733123/7668448

What are the purposes of vinyl-buffer and gulp-streamify in gulp?

As the documentation says, they both deal with transforming non-stream plugins to stream.
What I try to understand is, if I can use the .pipe() method on something, doesn't it mean it's a stream?
If so, what do I convert to what here?
vinyl-source-stream example:
(from: https://www.npmjs.com/package/vinyl-buffer)
var browserify = require('browserify')
var source = require('vinyl-source-stream')
var buffer = require('vinyl-buffer')
var uglify = require('gulp-uglify')
var size = require('gulp-size')
var gulp = require('gulp')
gulp.task('build', function() {
var bundler = browserify('./index.js')
return bundler.pipe()
.pipe(source('index.js'))
.pipe(buffer()) // <---------------------- why?
.pipe(uglify())
.pipe(size())
.pipe(gulp.dest('dist/'))
})
gulp-streamify example:
(from: https://www.npmjs.com/package/vinyl-source-stream)
var source = require('vinyl-source-stream')
var streamify = require('gulp-streamify')
var browserify = require('browserify')
var uglify = require('gulp-uglify')
var gulp = require('gulp')
gulp.task('browserify', function() {
var bundleStream = browserify('index.js').bundle()
bundleStream
.pipe(source('index.js'))
.pipe(streamify(uglify())) // <----------- why?
.pipe(gulp.dest('./bundle.js'))
})
One semi-useful example is to think about putting out a campfire with a bucket of water. To put out the fire you would want to completely fill up the bucket before dumping it on the fire rather putting a few drops in the bucket and then dumping lots of little drops over time on the fire. This metaphor doesn't capture everything but the big idea is this: you need a FULL bucket of water before you can put out the fire.
That "uglify" plugin works the same way. Imagine some enormous JS file you'd want to compress/uglify.
It will take a little bit of time to load the whole codebase & you definitely wouldn't want to try minifying each line as it comes in, right? Imagine you load a single line, minify it, load another line, minify it, etc etc-- it'd be a mess. You can't stream it (you need a full "bucket" of code before you can uglify it.) To uglify that file properly you'd need to load all that code first before attempting to uglify it.
Since Gulp is a "streaming" build system, you can't use uglify unless you have some mechanism to turn the stream into a buffer (& when it's done emit a stream.) Both tools you mention make this possible.
Here's the flow:
STREAM > (BUFFER) > {perform some work on the whole "buffered" file} > STREAM > {other gulp work, etc }
To your specific question, you can use .pipe() because vinyl-buffer/gulp-streamify help "convert" streams to buffers then buffers to streams. They're different approaches to accomplish essentially the same thing.
As said, most plugins work with buffers (although some of them also support streams). Examples include gulp-uglify and gulp-traceur. You can do the conversion to buffers using gulp-buffer.
via https://medium.com/#webprolific/getting-gulpy-a2010c13d3d5
gulp-uglify dosen't support stream, so you should convert stream to buffer (example uses vinyl-buffer)
gulp-streamify can wrap old plugins to support streams(example uses gulp-uglify)
Different approaches but equally satisfactory results.
What I try to understand is if I can use the .pipe() method on
something, doesn't it mean that it's a stream?
No, .pipe() can also pass buffers. this blog post explains it well:
https://medium.com/#sogko/gulp-browserify-the-gulp-y-way-bb359b3f9623
Some gulp-* plugins works by taking in buffered vinyl files objects as
input.
But vinyl-source-stream emits a streaming vinyl file object.
That’s where vinyl-buffer comes in. So we simply need to convert that
to a buffered vinyl by using vinyl-buffer, like so
What I try to understand is if I can use the .pipe() method on
something, doesn't it mean that it's a stream?
Yes! It is a stream. But it's a an object stream!
Instead of streaming a series characters, it streams a series of objects, which are the files that you sourced.
Each 'data' event in a gulp stream emits a Vinyl file object, which looks something like this:
{
cwd: '/', //<string>
base: '/test/', //<string>
path: '/test/file.js', //<string>
contents: contents //<string> | <Buffer> | <stream.Readable>
}
So gulp-buffer plugin is a Transform stream that converts the file contents from stream.Readable to Buffer.
You can see this in the source, where it saves the original content stream on line 24 and assigns a Buffer as the new file contents on line 35.
Streamify does the same thing, on line 35 and line 48.
It's ok to leave the file contents as a Buffer after Uglify is done processing it. It's always ok for the contents to be a Buffer, gulp just doesn't do this when sourcing because it's too costly.

Loading one JS file before another

Ok so I have a .js file with about 10k lines of code. This code can be split up in
sub-object definitions
container object definitions
initialization code (after the objects have been defined)
program functionality
I would like to split this one file into 4 separate files, because it provides a better oversight. How do I go about doing this, given that they absolutely have to be declared in that order? What should I wrap up in a $(document).ready() and what not?
If I just separate the files and link them to the html in the correct order, I get undefined object errors. I was also thinking of something like this; but I don't know if that's any good...
Second JS File
function initializeContainers() {
var containerObj1 = {
bla: 'bla',
bla2: 'bla2'
},
var containerObj2 = {
bla: 'bla',
bla2: 'bla2'
};
};
First JS File
$(document).ready(function() {
function initializeSubObjects(callback) {
var subObj1 = {
somekey: 'somevalue',
someke2: 'someothervalue'
};
callback();
};
initializeSubObjects(initializeContainers);
});
I have no clue whether this is the correct way to do it?
PS: I also know you can add the script tags dynamically; but is that good practice?
In your example, you should swap the contents of your first and second file. You should only call the initializeContainers method when you know for sure the file has been loaded.
The easiest way to think about this is to load all files with definitions first (helpers, functions, classes, ...). Once all these are loaded, put the rest in the last file and start executing the code only in the last file
On a side note: If you deploy this into a production environment, you should consider bundling these files. Downloading 4 files will impact your load time, so it's better to just bundle them together and send them over as a single file. While you're at it, you probably also want to minify it.

How do I change the order in which Meteor loads Javascript files?

When you make a project with the Meteor framework, it packages all the files together, but there doesn't seem to be a way to explicitly say "I want this file to be loaded before that one".
Let's say, for example, I have 2 javascript files: foo.js and bar.js.
The file bar.js is actually containing code depending one the one inside foo.js but Meteor is loading bar.js before foo.js, breaking the project.
In node.js I would simply use require('./bar') inside foo.js
In the browser, I would put a <script> tag pointing to foo.js and another, after, pointing to bar.js, in order to load the files in the correct order.
How can we do that in Meteor?
According to the Meteor documentation, files are currently loaded in this order:
Files in [project_root]/lib are loaded first
Files are sorted by directory depth. Deeper files are loaded first.
Files are sorted in alphabetical order.
main.* files are loaded last.
Source:
http://docs.meteor.com/#structuringyourapp
Not a solution for all scenarios, but I think ideally anything that is dependent on other code would be placed in a Meteor.startup function, to ensure everything is already loaded.
You can always us a JS loader like yepnope.js and add it to the client.js file. This works for me.
I have a set of utility functions that I structured under common namespace (js global).
I.e.
// utils/utils.js
Utils = {};
and then in subfolders:
// utils/validation/validation.js
Utils.Validation = {};
// utils/validation/creditCard.js
Utils.Validation.creditCard = ... // validation logic etc
also I have bunch of code that uses Utils and it's subobjects.
Obviously, this structure doesn't work as Meteor load subfolders first.
To make it work as expected, I had to create /subfolder/subfolder/subfolder with meaningless names, and then shove root object in most deep subfolder, and branch objects in subfolders not so deep.
It is extremely counterintuitive for my taste and error-prone (suppose you have component that is even deeper in folder structure).
To address this issue, I used Q library with defers and promises. Solution still isn't clean as it makes you routine code repeating and checks but it gives you full control over the load order without messing with directory structure (hello to people who says you can organise meteor code as you want).
Example:
//utils.js
UtilsDefer = UtilsDefer || Q.defer();
UtilsDefer.resolve({
// here some root utils stuff
});
//cards.js
// here we'll depend on Utils but don't want to care about directory structure
UtilsDefer = UtilsDefer || Q.defer(); // it will be a) already
// resolved defer from utils.js, or b) new defer that will
// be resolved later in utils.js
UtilsDefer.then(function(Utils) {
// do something with utils usage, or for instance add some fields here
Utils.CreditCardDefer = Utils.CreditCardDefer || Q.defer();
Utils.CreditCardDefer.resolve({
// Credit card utils here
})
});
//someOtherFile.js
// it will be pain to use sub-objects with this method though:
UtilsDefer = UtilsDefer || Q.defer();
UtilsDefer.then(function(Utils) {
Utils.CreditCardDefer = Utils.CreditCardDefer || Q.defer();
Utils.CreditCardDefer.then(function(CreditCard) {
// do stuff with CreditCard _if_ you need to do it on startup stage
})
});
This is the example of rather narrow use case, as mostly you will be happy with handling these globals inside some user interaction callbacks or Meteor.startup where everything already initialised. Otherwise, if you want fine-grained control over initialisation order on very early stage, that could be a solution.

Categories