Gulp - Async Completion with a Loop - javascript

I'm working with a Gulp file. In this file, I have two functions that perform two separate tasks. These functions are called using the following:
gulp.task('build', gulp.parallel(buildJs, buildCss));
When the build task is executed, I receive the following error:
The following tasks did not complete: default, build, buildCss
Did you forget to signal async completion?
My buildCss function is defined like this:
const files= [
{ input:'core.scss', output:'core.css' },
{ input:'theme.scss', output:'theme.css' },
{ input:'controls.scss', output:'controls.css'}
];
function buildCss() {
files.forEach(function(f) {
return gulp.src(`./scss/${f.input}`)
.pipe(sass())
.pipe(concat(f.output))
.pipe(gulp.dest('./'))
;
});
}
I suspect that each iteration of the loop is spinning up it's own thread. So, Gulp never knows when buildCss is finished. Which means, somehow, I need to know when all of the .css files are generated and call something. I'm unsure how to do the last piece if my understanding is correct though.
How do I address async completion of items in a loop in Gulp?
Thank you!

You can use .on('end') and create a Promise for each task, and then check that all promises went ok, sth like this should work:
function buildCss() {
return Promise.all(files.map(task => {
return new Promise((resolve,reject) => {
return gulp.src(`./scss/${task.input}`)
.pipe(sass())
.pipe(concat(task.output))
.pipe(gulp.dest('./'))
.on('end', () => resolve())
;
});
}));
}

Related

Batching using RxJS?

I'm guessing this should be somewhat easy to achieve but I've having trouble (conceptually, I guess) figuring out how to tackle it.
What I have is an API that returns an array of JSON objects. I need to step through these objects, and, for each object, make another AJAX call. The issue is the system that handles each AJAX call can only handle two active calls at a time (as it's quite a CPU-intensive task that hooks out into a desktop application).
I was wondering how I could achieve this using RxJS (either using version 5 or 4)?
EDIT: In addition, is it possible to have a chain of steps running concurrently. i.e.
Downloading File: 1
Processing File: 1
Converting File: 1
Uploading File: 1
Downloading File: 2
Processing File: 2
Converting File: 2
Uploading File: 2
Downloading File: 3
Processing File: 3
Converting File: 3
Uploading File: 3
I've tried doing something like:
Rx.Observable.fromPromise(start())
.concatMap(arr => Rx.Observable.from(arr))
.concatMap(x => downloadFile(x))
.concatMap((entry) => processFile(entry))
.concatMap((entry) => convertFile(entry))
.concatMap((entry) => UploadFile(entry))
.subscribe(
data => console.log('data', new Date().getTime(), data),
error => logger.warn('err', error),
complete => logger.info('complete')
);
However that doesn't seem to work. The downloadFile, for example doesn't wait for processFile, convertFile and uploadFile to all complete, rather, the next one will run again as soon as the previous one completes.
Here are 2 approaches, if you want the sequence of requests exactly like this
Downloading File: 1
Processing File: 1
Converting File: 1
Uploading File: 1
Downloading File: 2
Processing File: 2
...
You need to resolve all promises inside single concatMap method, like this
Rx.Observable.fromPromise(getJSONOfAjaxRequests())
.flatMap(function(x) { return x;})
.concatMap(function(item) {
return downloadFile(item)
.then(processFile)
.then(convertFile);
})
.subscribe(function(data) {
console.log(data);
});
see the working plunkr here: https://plnkr.co/edit/iugdlC2PpW3NeNF2yLzS?p=preview
This way, the new ajax call will be sent only when the previous is finished.
Another approach is that allow the files to send requests in parallel but the operations 'downloading,processing,converting,uploading' will be in sequence. For this you can get it working by
Rx.Observable.fromPromise(getJSONOfAjaxRequests())
.flatMap(function(x) { return x;})
.merge(2) // in case maximum concurrency required is 2
.concatMap(function(item) {
return downloadFile(item);
})
.concatMap(function(item) {
return processFile(item);
})
.concatMap(function(item) {
return convertFile(item)
})
.subscribe(function(data) {
//console.log(data);
});
see plunkr here: https://plnkr.co/edit/mkDj6Q7lt72jZKQk8r0p?p=preview
You could use merge operator with the maxConcurrency overload (Rxjs v4), so something like :
Rx.Observable.fromArray(aJSONs)
.map (function (JSONObject) {
return ajaxRequest(JSONObject) // that would be an observable (or a promise)
})
.merge(2)
You can have a look to see other examples of use at :
Limit number of requests at a time with RxJS,
or How to limit the concurrency of flatMap?
Official documentation :
merge(maxConcurrency)
How about something like this? You could use from to break the array into bite sized chunks and process them one by one using concatMap.
function getArr() {
return Rx.Observable.of([1, 2, 3, 4, 5, 6, 7, 8]);
}
function processElement(element) {
return Rx.Observable.of(element)
.delay(500);
}
getArr()
.concatMap(arr => {
return Rx.Observable.from(arr);
})
.concatMap(element => {
return processElement(element);
})
.subscribe(res => {
console.log(res);
});
Old post but I believe this could work, for console log we could use tap. Note editor would through intellisense error since from expects an array, but the code should work.
from(start()).pipe(
switchMap(files => from(files).pipe(
switchMap(file => from(downloadFile(file)).pipe(
map(_ => ({file: file, downloaded: true}))
)),
switchMap(attr => from(processFile(attr.file)).pipe(
map(_ => ({...attr, downloaded: true}))
)),
switchMap(attr => from(convertFile(attr.file)).pipe(
map(_ => ({...attr, converted: true}))
)),
switchMap(attr => from(uploadFile(attr.file)).pipe(
map(_ => ({...attr, uploaded: true}))
))
))
).subscribe(_ => {})

I need to pass a parameter to a gulp task from another task or replace a task with a function called by runSequence

I have gulp file that I use to create the Html and Js for multiple kinds of exams, Ex1, Ex2 etc
Here's the task I use to create these for Ex1. It has hardcoded a call to the makeEx1Html task, three other tasks and followed by a function call where I can pass a parameter:
gulp.task('make_prod_ex1', function () {
runSequence(
'makeEx1Html',
'makeTemplate',
'rename_bundle_css',
'rename_bundle_js',
function () {
make_prod_index('ex1');
});
});
Here's the task that's hardcoded for Ex1:
gulp.task('makeEx1Html', function () {
return gulp.src(config.srcEx1Html, { base: process.cwd() })
.pipe(print(function (file) {
return "Found file " + file;
}))
.pipe(rename({ basename: 'base' }))
.pipe(gulp.dest('./'));
});
Here's the function where I can pass a parameter:
function make_prod_index(name) {
return function () {
gulp.src('index.html')
.pipe(htmlreplace({
'css': 'content/bundles/css.min.css',
'js': 'content/bundles/js.min.js'
}))
.pipe(eol())
.pipe(lec({ eolc: 'CRLF' }))
.pipe(replace('content/bundles/css.min.css', 'content/bundles/css-' + md5File('content/bundles/css.min.css') + '.min.css.gz'))
.pipe(replace('content/bundles/js.min.js', 'content/bundles/js-' + md5File('content/bundles/js.min.js') + '.min.js.gz'))
.pipe(rename('index-' + name + '.html'))
.pipe(gulp.dest('./'));
}
}
I would like to avoid having a specific task like 'makeEx1Html' and 'makeEx2Html' etc but I am not sure how to do this.
Note all these task needs to run in order which is why I'm using runSequence.
I would appreciate any suggestions. Ideally I would like the task that makes the Html to be a function that I can pass a parameter to but I am not sure how to fit this into my requirements.
Ideally I would like the task that makes the Html to be a function that I can pass a parameter to
You can do just that, except you don't just pass your parameters to the function, but also a callback cb that will be called when your function is done:
function makeExHtml(files, cb) {
return gulp.src(files, { base: process.cwd() })
.pipe(print(function (file) {
return "Found file " + file;
}))
.pipe(rename({ basename: 'base' }))
.pipe(gulp.dest('./'))
.on('end', cb);
}
In your gulp tasks you can then use the above makeExHtml() function and pass a callback that will execute the rest of your runSequence():
gulp.task('make_prod_ex1', function () {
makeExHtml(config.srcEx1Html, function() {
runSequence(
'makeTemplate',
'rename_bundle_css',
'rename_bundle_js',
function () {
make_prod_index('ex1');
});
});
});

gulp-watch doesn't track new or deleted files

I've got a task:
gulp.task('assetsInject', function () {
gulp.src(paths.index)
.pipe(plugins.inject(
gulp.src(paths.scripts, {read: false}),
{
addRootSlash: false
}
))
.pipe(gulp.dest(paths.static))
});
and want it to run on new *.js files created or deleted:
gulp.task('watch', function () {
plugins.watch({glob: path.scripts}, function () {
gulp.start('assetsInject')
});
});
but gulp-watch starts task only when existing files were modified, not when new files were created.
Is it possible to start task after creation and deletion only, not when files were modified?
Look at this similar question, it can be that path.scripts contains wrong globs.

Why is Jasmine not resetting a spy when spying on $.ajax?

I am trying to spy on $.ajax in Jasmine 2.0 tests. Here is a simplified example (TypeScript) showing my scenario:
describe("first test", () => {
var deferred = jQuery.Deferred();
spyOn($, "ajax").and.callFake((uri: string, settings: JQueryAjaxSettings) => {
return deferred.resolve("ThisIsADummyResult");
});
it("should return dummy result", done => {
$.ajax("http://somedummyserver.net").then(result => {
expect(result).toBe("ThisIsADummyResult");
done();
});
});
});
describe("second test", () => {
var deferred = jQuery.Deferred();
spyOn($, "ajax").and.callFake((uri: string, settings: JQueryAjaxSettings) => {
return deferred.resolve("ThisIsAnotherResult");
});
it("should return another result", done => {
$.ajax("http://somedummyserver.net").then(result => {
expect(result).toBe("ThisIsAnotherResult");
done();
});
});
});
firstTest as well as second test work if I run them alone. However, if I run both tests as shown above, I get the following error message: ajax has already been spied upon.
So my questions are:
Shouldn't the spies be reset by Jasmine after each test automatically? Why doesn't that work in my case?
Is there another way of using spyOn which makes Jasmine reset the spies?
How can I manually reset the spies?
Update: I continued experimenting and found a possible solution myself. If I set up the spies inside of the it spec, both tests run fine. Here is the code for first test showing what I mean:
describe("first test", () => {
it("should return dummy result", done => {
var deferred = jQuery.Deferred();
spyOn($, "ajax").and.callFake((uri: string, settings: JQueryAjaxSettings) => {
return deferred.resolve("ThisIsADummyResult");
});
$.ajax("http://somedummyserver.net").then(result => {
expect(result).toBe("ThisIsADummyResult");
done();
});
});
});
Still, it would be very interesting why the first version does not work. Why does Jasmine not reset the spies in the first version whereas it does in the second one?
For stuff that is used across all tests but you need it reset for each test use 'beforeEach' : http://jasmine.github.io/2.0/introduction.html#section-Setup_and_Teardown
Jasmine does not magically know which lines of your describe body you want reevaluated for each 'it' block.

How can I analyse my JS program to ensure a particular method is always called with 2 arguments?

We're using promises in an AngularJS project and want to ensure that the then method is always called with 2 arguments, the 2nd being an error handler, like so:
$http.get(url).then(function () {
console.log('hooray!');
}, function (error) {
console.log('boo! error');
});
We're using jshint on the project. Can that perform this analysis?
Some calls to then do not require an error handler, i.e. in a chain of handlers:
$http.get(url1).then(function () {
console.log('hooray!');
return $http.get(url2);
}).then(function () {
console.log('hooray again! all our data loaded');
}, function (error) {
console.log('boo! error in one of our 2 requests');
});
We could mark these up using jshint's /* jshint ignore:start */ comments or similar.

Categories