Deletion with API sometimes works and sometimes it does not - javascript

I am trying to delete files from folder with API call.
It seems to work randomly, some files get deleted, some don't.
The data structure is as follow:
Each folder object has files property.
In files_to_delete I have files to delete and index of folder in folders array.In API call I put whole folder objectI remove file with lodash remove function.
Any idea why it does not work?
angular.forEach(files_to_delete, function(file_to_delete) {
_.remove(folders[file_to_delete.index].files, function(file) {
if (file === file_to_delete) {
$http.put('/save_folders_file', folders[file_to_delete.index])
return (file === file_to_delete)
});

Your example isn't entirely clear, but it looks like you save the container the files are in repeatedly, each time you find a file to delete. That seems chaotic. It also assumes that Lodash modifies the container as it's doing the loop, but the documentation doesn't say it does, and a simple (perhaps naive) test suggests it doesn't:
var array = [1, 2, 3, 4];
var evens = _.remove(array, function(n) {
  console.log(n, JSON.stringify(array));
return n % 2 == 0;
});
console.log("array", JSON.stringify(array));
console.log("evens", JSON.stringify(evens));
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.11/lodash.min.js"></script>
When I run that, I see:
1 [1,2,3,4]
2 [1,2,3,4]
3 [1,2,3,4]
4 [1,2,3,4]
array [1,3]
evens [2,4]
...which suggests that it does the callbacks first, then goes back and removes the relevant entries.
Even if it did remove them as it went, though, those overlapping HTTP requests would bother me a lot.
Instead, I'd save the container once, when done, after all the files are removed:
angular.forEach(files_to_delete, function(file_to_delete) {
const removed = _.remove(folders[file_to_delete.index].files, function(file) {
return file === file_to_delete;
});
if (removed.length) {
$http.put('/save_folders_file', folders[file_to_delete.index])
}
});

Related

Efficient way to get the actual filename in a case-insensitive filesystem in node

This is NOT a dupe of this question. This question is NOT about Windows. It's a general question across OSes.
Is there an efficient way to get the correct case of a filename in node.js other than getting the directory and finding the matching name?
Example: Assume I have a folder with 3 files
+-someFolder
+-fooBar.txt
+-Moo.txt
+-ReadMe.txt
I want a function that passed somefolder/readme.txt returns someFolder/ReadMe.txt.
AFAICT the only way to do that is to call fs.readDir or fs.readDirSync and see if there is a matching file, something like
const fs = require('fs');
const path = require('path');
function getActualFilename(filename) {
if(!fs.existsSync(filename)) {
throw new Error(`${filename} does not exist`);
}
return getActualFilenameImpl(filename);
}
function getActualFilenameImpl(filename) {
const lcFilename = path.basename(filename).toLowerCase();
// handles passing in `c:\\`
if (!lcFilename) {
return filename.toUpperCase();
}
const dirname = path.dirname(filename);
let filenames;
try {
filenames = fs.readdirSync(dirname);
} catch (e) {
// we already verified the path exists above so if this
// happens it means the OS won't let use get a listing (UNC root on windows)
// so it's the best we can do
return filename;
}
const matches = filenames.filter(name => lcFilename === name.toLowerCase());
if (!matches.length) {
throw new Error(`${filename} does not exist`);
}
const realname = matches[0];
if (dirname !== '.') {
if (dirname.endsWith('/') || dirname.endsWith('\\')) {
return path.join(dirname, realname);
} else {
return path.join(getActualFilenameImpl(dirname), realname);
}
} else {
return realname;
}
}
The code above is pretty hacky. Trying in on different things has made it clear there's lots of edge cases. On Windows in particular UNC paths fail since you can't call fs.readdirSync once you get to the network path root. I have no idea what functions to call to figure out where that path separates and then how to get the correct case path for that which is probably an entirely separate set of Windows API calls (like calling whatever functions net use uses to show shares) etc...
I did notice path.dirname stops removing the trailing slash when it gets to a UNC path so using that to try to figure out when stop trying.
Notes:
I get that for example on Linux (and optionally on Mac) the file system may be case sensitive and I'd have to check for that but I'm mostly concerned with Windows and standard macOS and will deal with case-sensitive issues later.
I also get that JavaScript's toLowerCase might not match the OSes concept of case insensitivity so if there is a solution that takes that into account that would also be great!
I get that I could cache results or directory listing for a speed up but was wondering if there is some other function to use that doesn't read the entire directory listing.
I'm actually trying to solve several problems and am open to other suggestions
Problem 1: What filename to store in an app specific database. It seems best to store the actual filename. See #3
Problem 2: Figuring out if 2 filenames reference the same file/folder. So if the user specifies SomeFolder/foobar.txt and somefolder/FOOBAR.txt I don't want that to appear as 2 separate files if they are actually the same file. I need my app to know they reference the same file. I think I can call fs.stat for this and check if the ino field matches?
Problem 3: Related to problem 1, reloading metadata related to the file. If the user specifies SomeFolder/foobar.txt at some point and my app generates metadata related to the file, then at some other point in time they specify somefolder/FOOBAR.txt I need to find the matching metadata. My current thinking is by looking up the actual filename and using that to match with this problem would be solved. Although I suppose if they rename the file from FooBar.txt to foobar.txt it would lose the metadata. Not sure I care about that situation though since if they rename from FooBar.txt to SomethingElse.txt I definitely do not care if I lose the metadata.
That said, maybe I should store the ino as the key in my DB? Not sure I'm comfortable with that idea yet but it's a possibility and would love to know if others do that. Some checking reveals that at least on macOS the ino stays the same across moves and rename on the same drive which would be a good thing for my use case. On the other hand I'd assume ino is only valid per file system so if I have 2 different drives mounted I could get clashing inos. I could use dev and ino as a key as in
const stat = fs.statSync(filename);
const key = `${stat.dev}:${stat.ino}`;
Though I have no idea if stat.dev is always the same with removable storage. I assume it's not. So it seems like filename as key is probably better?
As long as the filesystem doesn't keep a connection between files with the same names in different cases (and I don't know any such filesystem) there can't be a solution other than scanning the directory because there is simply no API provided for this at all at any level.
So you have to either scan manually as you already suggested or by using libraries like glob to find files while ignoring case.
But you say you also have the filenames in a database. So if you can make sure that the filenames in the DB are exactly matching the filenames in the filesystem then you should be able to find the files in different cases by doing case-insensitive DB queries. If it is an SQL database then it should already provide this functionality. If it is a more primitive data store you may add another filename property which is always lower-case so you can match against this to find the real file.

Pass parameter to Gulp task in gulpfile.js (not command-line)

EDIT: As I wrote IN THE TITLE AND FROM THE BEGINNING, this is not about command-line parameters and is thus NOT A DUPLICATE. //EDIT
I have a Sass setup with an indefinite number of uniquely-designed pages (page_1, page_2, etc), each having their own sass/pages/page_1/page_1.scss file.
The pages all belong to the same website, and each page's sass file #imports the same set of files from a sass/includes folder.
With a basic gulp task watching sass/**/*, every page's styles get compiled anytime I make a change to any page's styles. Obviously this doesn't scale well.
I tried using gulp-watch, but it doesn't catch if changes are made to one of the included .scss files. It only catches changes made to the files that actually get compiled into an equivalent .css.
For the purposes of having my gulpfile be as DRY as possible, the best solution I could come up with was to maintain a basic array of folder names in gulpfile.js, and to loop through and watch each of them separately, using the same sass-compiling task for each folder.
var pageFolderNames = [
'page_1',
'page_2'
// etc
];
Then for the gulp task, I have:
gulp.task('watch_pages', function()
{
// Get array length
var numPages = pageFolderNames.length;
// Add a new watch task for each individual page
for (var i = 0; i < numPages; i++)
{
gulp.watch('sass/pages/' + pageFolderNames[i] + '/**/*.scss', ['sass_page']);
}
});
The (simplified) task that compiles sass:
// Task: Compile page-specific Sass
gulp.task('sass_page', function()
{
return gulp.src('sass/pages/' + pageFolderNames[i] +'/**/*.scss')
.pipe(plumber(plumberErrorHandler))
.pipe(sass(...))
.pipe(gulp.dest('css/pages/' + pageFolderNames[i]));
});
This approach (I know my JS-fu is weaksauce) results in an error:
'sass_page' errored after 71 μs
ReferenceError: i is not defined
Is there any way to pass parameters, such as i, to gulp tasks to get this working? Alternately, is there a better way to accomplish what I'm trying to do? I have a sneaking suspicion there is. :-/
I found out there is an on change event for gulp watch. So this might be what you're looking for:
var pagesDir = 'sass/pages/';
gulp.task('watch_pages', function() {
gulp.watch(pagesDir + '**/*')
.on("change", function(file) {
// absolute path to folder that needs watching
var changedDest = path.join(__dirname, pagesDir);
// relative path to changed file
var changedFile = path.relative(changedDest, file.path);
// split the relative path, get the specific folder with changes
var pageFolder = changedFile.split('\\')[0];
gulp.src(path.join(pagesDir, pageFolder) +'/**/*.scss')
.pipe(plumber(plumberErrorHandler))
.pipe(sass(...))
.pipe(gulp.dest('css/pages/' + pageFolder));
console.log(changedDest);
console.log(changedFile);
console.log(pageFolder);
});
});
Also, this way you don't need to declare the folder variables. If you add directories within the path being watched, it should pick it up and name the destination folder accordingly.
Theoretically the gulp task to compile sass should work within the watch task. I played around with the paths, and it seems to spitting them out. Let me know what happens, I can modify if necessary.
The required packages:
var gulp = require("gulp"),
path = require("path"),
rimraf = require("rimraf");
BTW, since you already have access to the file path, you can perhaps target the specific scss file instead of the whole directory.
As Brian answered, the best approach is to have one watcher. In the same way as the principle of delegation with dom event listeners. it's better, even in our case it can not really matter. We need our thing done. if the consumed resources and performance doesn't bother us. It's ok. But stay as Brian answered. one watcher is the best approach. then you've got to get the file that was changed. and from there you get your page folder. So for that i will not add a thing. Except that you don't necessarily need to use the .on("change". You can directly set the same parameter for your function as this example show:
watch('./app/tempGulp/json/**/*.json', function (evt) {
jsonCommentWatchEvt = evt
gulp.start('jsonComment')
})
evt here is what Brian set as file.
What i want to add, is about how to pass a parameter from your watcher to your task. For example about the i in your initial work. A way of doing that, that i see and use, is to set a global variable that hold the data wanted to be passed. So it's accessible in both blocks. You set the value in watcher just before starting the task. And then you use it in the task. A good practice, is to save it in a task local variable just at start. That way you avoid the problem that it can change by another watch handling triggering.
For a lively example. check my answer out here : https://stackoverflow.com/a/49733123/7668448

Gulp get a ls of a directory

I have a directory assets with a number of images inside. Inside my JavaScript code, I have an array, with the same list of images, that I use to preload everything. It means I have twice the same information, the list of images, both in my JavaScript and in my filesystem. I would like to have a gulp task to list all those images, and populate the array in JavaScript, with something like:
var listFiles = require("list-files-in-folder");
var images = listFiles("./assets/");
Do you know of any plugin who does that? Or a simple way to implement it maybe?
Assuming you're talking about nodejs, take a look at this
var glob = require("glob")
// options is optional
glob("**/*.js", options, function (er, files) {
// files is an array of filenames.
// If the `nonull` option is set, and nothing
// was found, then files is ["**/*.js"]
// er is an error object or null.
})

Loading one JS file before another

Ok so I have a .js file with about 10k lines of code. This code can be split up in
sub-object definitions
container object definitions
initialization code (after the objects have been defined)
program functionality
I would like to split this one file into 4 separate files, because it provides a better oversight. How do I go about doing this, given that they absolutely have to be declared in that order? What should I wrap up in a $(document).ready() and what not?
If I just separate the files and link them to the html in the correct order, I get undefined object errors. I was also thinking of something like this; but I don't know if that's any good...
Second JS File
function initializeContainers() {
var containerObj1 = {
bla: 'bla',
bla2: 'bla2'
},
var containerObj2 = {
bla: 'bla',
bla2: 'bla2'
};
};
First JS File
$(document).ready(function() {
function initializeSubObjects(callback) {
var subObj1 = {
somekey: 'somevalue',
someke2: 'someothervalue'
};
callback();
};
initializeSubObjects(initializeContainers);
});
I have no clue whether this is the correct way to do it?
PS: I also know you can add the script tags dynamically; but is that good practice?
In your example, you should swap the contents of your first and second file. You should only call the initializeContainers method when you know for sure the file has been loaded.
The easiest way to think about this is to load all files with definitions first (helpers, functions, classes, ...). Once all these are loaded, put the rest in the last file and start executing the code only in the last file
On a side note: If you deploy this into a production environment, you should consider bundling these files. Downloading 4 files will impact your load time, so it's better to just bundle them together and send them over as a single file. While you're at it, you probably also want to minify it.

How do I change the order in which Meteor loads Javascript files?

When you make a project with the Meteor framework, it packages all the files together, but there doesn't seem to be a way to explicitly say "I want this file to be loaded before that one".
Let's say, for example, I have 2 javascript files: foo.js and bar.js.
The file bar.js is actually containing code depending one the one inside foo.js but Meteor is loading bar.js before foo.js, breaking the project.
In node.js I would simply use require('./bar') inside foo.js
In the browser, I would put a <script> tag pointing to foo.js and another, after, pointing to bar.js, in order to load the files in the correct order.
How can we do that in Meteor?
According to the Meteor documentation, files are currently loaded in this order:
Files in [project_root]/lib are loaded first
Files are sorted by directory depth. Deeper files are loaded first.
Files are sorted in alphabetical order.
main.* files are loaded last.
Source:
http://docs.meteor.com/#structuringyourapp
Not a solution for all scenarios, but I think ideally anything that is dependent on other code would be placed in a Meteor.startup function, to ensure everything is already loaded.
You can always us a JS loader like yepnope.js and add it to the client.js file. This works for me.
I have a set of utility functions that I structured under common namespace (js global).
I.e.
// utils/utils.js
Utils = {};
and then in subfolders:
// utils/validation/validation.js
Utils.Validation = {};
// utils/validation/creditCard.js
Utils.Validation.creditCard = ... // validation logic etc
also I have bunch of code that uses Utils and it's subobjects.
Obviously, this structure doesn't work as Meteor load subfolders first.
To make it work as expected, I had to create /subfolder/subfolder/subfolder with meaningless names, and then shove root object in most deep subfolder, and branch objects in subfolders not so deep.
It is extremely counterintuitive for my taste and error-prone (suppose you have component that is even deeper in folder structure).
To address this issue, I used Q library with defers and promises. Solution still isn't clean as it makes you routine code repeating and checks but it gives you full control over the load order without messing with directory structure (hello to people who says you can organise meteor code as you want).
Example:
//utils.js
UtilsDefer = UtilsDefer || Q.defer();
UtilsDefer.resolve({
// here some root utils stuff
});
//cards.js
// here we'll depend on Utils but don't want to care about directory structure
UtilsDefer = UtilsDefer || Q.defer(); // it will be a) already
// resolved defer from utils.js, or b) new defer that will
// be resolved later in utils.js
UtilsDefer.then(function(Utils) {
// do something with utils usage, or for instance add some fields here
Utils.CreditCardDefer = Utils.CreditCardDefer || Q.defer();
Utils.CreditCardDefer.resolve({
// Credit card utils here
})
});
//someOtherFile.js
// it will be pain to use sub-objects with this method though:
UtilsDefer = UtilsDefer || Q.defer();
UtilsDefer.then(function(Utils) {
Utils.CreditCardDefer = Utils.CreditCardDefer || Q.defer();
Utils.CreditCardDefer.then(function(CreditCard) {
// do stuff with CreditCard _if_ you need to do it on startup stage
})
});
This is the example of rather narrow use case, as mostly you will be happy with handling these globals inside some user interaction callbacks or Meteor.startup where everything already initialised. Otherwise, if you want fine-grained control over initialisation order on very early stage, that could be a solution.

Categories