We Keep Coding

copy a file/object and change the name

I'm doing an API for a gallery; so, I create a method that let copy an image from the database.
Now, I want to add a number at the end of the copy-image name. For example:
-original image name: image
-copy image name: image(1)
-2nd copy image name: image(2)
How can I add the number to the name of copied name automatically?
'use strict'
let imageObject= require('../models/image-object');
let fs=require('fs');
let path= require('path');
let gallery_controllers={
copyImage:function(req,res){
//get the id param of the image to copy
let imageId=req.params.id;
if(imageId==null) return res.status(404).send({message:"no ID defined"});
//I search the requiere image on the database
imageObject.findById(imageId,(err,image)=>{
if(err) return res.status(500).send({message:'err to response data'});
if(!image) return res.status(404).send({message:'image not found'});
if(image){
//set a new model-object
let imageCopied= new imageObject();
imageCopied.name= image.name;
imageCopied.image=image.image;
//save image copied on the database
imageCopied.save((err,image_copied)=>{
if(err) return res.status(500).send({message:"error 500"});
if(!image_copied) return res.status(404).send({message:"error 404"});
return res.status(200).send({
image:image_copied
})
})
}
})
},
}

Here's a function that looks in the directory passed to it for files of the name file(nnn) where nnn is some sequence of digits and returns back to you the full path of the next one in sequence (the one after the highest number that already exists).
This function pre-creates a placeholder file by that name to avoid concurrency issues with multiple asynchronous operations calling this function and potentially conflicting (if it only returned the filename without creating the file). To further handle conflicts, it creates the placeholder file in a mode that fails if it already exists (so only one invocation of this function will ever create that particular file) and it automatically retries to find a new number if it gets a conflict (e.g. someone else created the next available file before we got to it). All of this logic is to avoid the subtleties of possible race conditions in creating the next filename in the sequence.
Once the caller has a unique filename that this resolves to, then it is expected that they will overwrite the placeholder contents with their own contents.
// pass directory to look in
// pass base file name so it will look for next in sequence as in "file(3)"
// returns the full path of the unique placeholder file it has created
// the caller is then responsible for that file
// calling multiple times will create a new placeholder file each time
async function findNextName(dir, base) {
let cntr = 0;
const cntr_max = 5;
const regex = new RegExp(`^${base}\\((\\d+)\\)$`);
async function run() {
const files = await fs.promises.readdir(dir);
let highest = 0;
for (let f of files) {
let matches = f.match(regex);
if (matches) {
let num = parseInt(matches[1]);
if (num > highest) {
highest = num;
}
}
}
let name = `${base}(${highest + 1})`;
// create placeholder file with this name to avoid concurrency issues
// of another request also trying to use the same next file
try {
// write to file, fail if the file exists due to a concurrency issue
const fullPath = path.resolve(path.join(dir, name));
await fs.promises.writeFile(fullPath, "placeholder", { flag: "wx" });
return fullPath;
} catch (e) {
// if this fails because of a potential concurrency issue, then try again
// up to cntr_max times to avoid looping forever on a persistent error
if (++cntr < cntr_max) {
return run();
} else {
throw e;
}
}
}
return run();
}
You could call it like this:
findNextName(".", "file").then(filename=> {
console.log(filename);
}).catch(err => {
console.log(err);
});

Firebase web: upload multiple files to Storage an then download their URLs

I am creating a blogging website, and am writing code that does the following in order:
1. stores multiple photos that the user uploads
2. download their URLs
3. save them to the realtime database.
I wrote the function below to do #1 and #2. Basically the idea is to store all the urls to an array, in this case 'urlarray'.
function url_array_get(){
return new Promise(function(resolve,reject){
let filez=review_photo.files;
let urlarray=[];
let user=firebase.auth().currentUser;
let files=Array.from(filez);
files.forEach(function(file) {
let storageRef=firebase.storage().ref('data/'+user.uid+'/posts/'+file.name);
storageRef.put(file).then(function(snapshot){
snapshot.ref.getDownloadURL().then(function(url) {
urlarray.push(url);
})
})
});
if (!urlarray){
reject("oops");
}
else {
resolve(urlarray);
}
});
}
Here is the part of the upload function code that would upload all the relevant data to the database, including the array of URLs returned by the promise in the function above. (I omitted the rest of the code to make the case concise)
let userpostRef=firebase.database().ref('posts/');
let newpostRef=userpostRef.push();
newpostRef.set({
userid: user.uid,
post_target: rtarget,
post_content:rtext,
time: firebase.database.ServerValue.TIMESTAMP
}).then(function(){
url_array_get().then(function(result){
newpostRef.once('value', function(snapshot) {
newpostRef.update({
postnum:snapshot.key,
photolink:result
})
})})}).
then(function(){
alert("Upload successful!");
window.location.href='/'+username;
})
.catch(function(error){
alert("Error!");
});
}
Here is the issue: The code would write to database everything except the 'photolink', which should be the array of URLs.
Here is what I found out doing debugging:
-Photos are stored without any problem.
-urls are downloaded for each file, and urlarray is returned successfully in the execution code as expected.
What might have gone wrong? I am lost here. Any advice would be very much welcome.
Thanks a lot!

Each time you call storageRef.put(...) it starts an asynchronous operation. Right now your `` doesn't wait for these asynchronous operations to complete, and instead returns the list of URLS before it's been populated.
The easiest way to see this is by adding some simple logging to your code:
function url_array_get(){
return new Promise(function(resolve,reject){
let filez=review_photo.files;
let urlarray=[];
let user=firebase.auth().currentUser;
let files=Array.from(filez);
files.forEach(function(file) {
console.log("Starting to put file...");
let storageRef=firebase.storage().ref('data/'+user.uid+'/posts/'+file.name);
storageRef.put(file).then(function(snapshot){
console.log("Upload done, getting download URL...");
snapshot.ref.getDownloadURL().then(function(url) {
console.log("Download URL gotten, adding to array...");
urlarray.push(url);
})
})
});
if (!urlarray){
reject("oops");
}
else {
console.log("Resolving with "+urlarray.length+" download URLs");
resolve(urlarray);
}
});
}
When you run this code, the output will look like:
Starting to put file...
Starting to put file...
Starting to put file...
Resolving with 0 download URLs
Upload done, getting download URL...
Download URL gotten, adding to array...
Upload done, getting download URL...
Download URL gotten, adding to array...
Upload done, getting download URL...
Download URL gotten, adding to array...
That is not the order you want of course, as you're returning the array before you added any download URL to it, and even before any of the uploads complete.
The solution is (as always when it comes to asynchronous operations) to wait until all operations have finished before resolving/returning. You can most easily do this with Promise.all() with something like this:
function url_array_get(){
let promises = [];
let filez=review_photo.files;
let user=firebase.auth().currentUser;
let files=Array.from(filez);
files.forEach(function(file) {
let storageRef=firebase.storage().ref('data/'+user.uid+'/posts/'+file.name);
promises.push(
storageRef.put(file).then(function(snapshot){
return snapshot.ref.getDownloadURL()
})
});
});
return Promise.all(promises);
}
Or slightly shorter:
function url_array_get(){
let user=firebase.auth().currentUser;
let ref = firebase.storage().ref('data/'+user.uid+'/posts/');
let files=Array.from(review_photo.files);
let promises = files.map(function(file) {
return ref.child(file.name).put(file).then(function(snapshot){
return snapshot.ref.getDownloadURL()
})
});
return Promise.all(promises);
}

How to do first n times async loops then one time function in callback way?

It is not a question about how to do post multipart form in nodejs.
But how to do such logic(first do a n times loops(async) then one time function(async)) in callback way?
for example, client will post multipart form with normal form fields:
req.files[n]: contains n images, needs to save to server's local filesystem
req.body: contains post.title, post.content, post.user
In normal way(php, java...), sample code would be
array savedPath = [];
// save images to local filesystem
foreach image in files
savedPath.push(saveImageToLocal(image))
// append saved images path to post
var post = req.body;
post.images = savedPath;
Posts.insert(post)
But in nodejs, callback way, how can i write it?
var savedPath = [];
saveImageToLocal(files[0], function(path) {
savedPath.push(path);
saveImageToLocal(files[1], function(path) {
savedPath.push(path);
//.... its n elements, how can I write it??
var post = req.body;
post.images = savedPath;
Posts.insert(postfunction(err, result) {
res.send(err, result)
});
});
});
Or
var savedPath = [];
for(i=0;i<n;i++) {
savesaveImageToLocalTo(files[i], function(path) {
savedPath.push(path);
});
}
waitSaveToFinished() ??
var post = req.body;
post.images = savedPath;
Posts.insert(postfunction(err, result) {
res.send(err, result)
});
How to do these kind of things in the way of nodejs/callback?

The best way to coordinate multiple asynchronous operation is to use promises. So, if this were my code, I would change or wrap saveImageToLocalTo() and Posts.insert() to return promises and then use promise features for coordinating them. If you're going to be writing much node.js code, I'd suggest you immediately invest in learning how promises work and start using them for all async behavior.
To solve your issue without promises, you'd have to implement a counter and keep track of when all the async operations are done:
var savedPath = [];
var doneCnt = 0;
for(i=0;i<n;i++) {
savesaveImageToLocalTo(files[i], function(path) {
++doneCnt;
savedPath.push(path);
// if all the requests have finished now, then do the next steps
if (doneCnt === n) {
var post = req.body;
post.images = savedPath;
Posts.insert(postfunction(err, result) {
res.send(err, result)
});
}
});
}
This code looks like it missing error handling since most async operations have a possibility of errors and can return an error code.

Inserting into Collection after Promises in a Meteor Method

I'm using this Gumroad-API npm package in order to fetch data from an external service (Gumroad). Unfortunately, it seems to use a .then() construct which can get a little unwieldy as you will find out below:
This is my meteor method:
Meteor.methods({
fetchGumroadData: () => {
const Gumroad = Meteor.npmRequire('gumroad-api');
let gumroad = new Gumroad({ token: Meteor.settings.gumroadAccessKey });
let before = "2099-12-04";
let after = "2014-12-04";
let page = 1;
let sales = [];
// Recursively defined to continue fetching the next page if it exists
let doThisAfterResponse = (response) => {
sales.push(response.sales);
if (response.next_page_url) {
page = page + 1;
gumroad.listSales(after, before, page).then(doThisAfterResponse);
} else {
let finalArray = R.unnest(sales);
console.log('result array length: ' + finalArray.length);
Meteor.call('insertSales', finalArray);
console.log('FINISHED');
}
}
gumroad.listSales(after, before, page).then(doThisAfterResponse); // run
}
});
Since the NPM package exposes the Gumorad API using something like this:
gumroad.listSales(after, before, page).then(callback)
I decided to do it recursively in order to grab all pages of data.
Let me try to re-cap what is happening here:
The journey starts on the last line of the code shown above.
The initial page is fetched, and doThisAfterResponse() is run for the first time.
We first dump the returned data into our sales array, and then we check if the response has given us a link to the next page (as an indication as to whether or not we're on the final page).
If so, we increment our page count and we make the API call again with the same function to handle the response again.
If not, this means we're at our final page. Now it's time to format the data using R.unnest and finally insert the finalArray of data into our database.
But a funny thing happens here. The entire execution halts at the Meteor.call() and I don't even get an error output to the server logs.
I even tried switching out the Meteor.call() for a simple: Sales.insert({text: 'testing'}) but the exact same behaviour is observed.
What I really need to do is to fetch the information and then store it into the database on the server. How can I make that happen?
EDIT: Please also see this other (much more simplified) SO question I made:
Calling a Meteor Method inside a Promise Callback [Halting w/o Error]

I ended up ditching the NPM package and writing my own API call. I could never figure out how to make my call inside the .then(). Here's the code:
fetchGumroadData: () => {
let sales = [];
const fetchData = (page = 1) => {
let options = {
data: {
access_token: Meteor.settings.gumroadAccessKey,
before: '2099-12-04',
after: '2014-12-04',
page: page,
}
};
HTTP.call('GET', 'https://api.gumroad.com/v2/sales', options, (err,res) => {
if (err) { // API call failed
console.log(err);
throw err;
} else { // API call successful
sales.push(...res.data.sales);
res.data.next_page_url ? fetchData(page + 1) : Meteor.call('addSalesFromAPI', sales);
}
});
};
fetchData(); // run the function to fetch data recursively
}

FirefoxOS: return array using device storage api

I am just getting started with coding for FirefoxOS and am trying to get a list of files in a directory.
The idea is to find the name of each file and add it to the array (which works), but I want to return the populated array and this is where I come unstuck. It seems that the array gets populated during the function (as I can get it to spit out file names from it) but when I want to return it to another function it appears to be empty?
Here is the function in question:
function getImageFromDevice (){
var imageHolder = new Array();
var pics = navigator.getDeviceStorage('pictures');
// Let's browse all the images available
var cursor = pics.enumerate();
var imageList = new Array();
var count = 0;
cursor.onsuccess = function () {
var file = this.result;
console.log("File found: " + file.name);
count = count +1;
// Once we found a file we check if there are other results
if (!this.done) {
imageHolder[count] = file.name;
// Then we move to the next result, which call the cursor
// success with the next file as result.
this.continue();
}
console.log("file in array: "+ imageHolder[count]);
// this shows the filename
}
cursor.onerror = function () {
console.warn("No file found: " + this.error);
}
return imageHolder;
}
Thanks for your help!

Enumerating over pictures is an asynchronous call. Essentially what is happening in your code is this:
You are initiating an empty array
You are are telling firefox os to look for pictures on the device
Then in cursor.onsuccess you are telling firefox os to append to the array you have created WHEN it gets back the file. The important thing here is that this does not happen right away, it happens at some point in the future.
Then you are returning the empty array you have created. It's empty because the onsuccess function hasn't actually happened.
After some point in time the onsuccess function will be called. One way to wait until the array is full populated would be to add in a check after:
if (!this.done) {
imageHolder[count] = file.name;
this.continue();
}
else {
//do something with the fully populated array
}
But then of course your code has to go inside the getImageFromDevice function. You can also pass a callback function into the getImageFromDevice function.
See Getting a better understanding of callback functions in JavaScript

The problem is with the aSynchronous nature of the calls you are using.
You are returning (and probably using) the value of imageHolder when it's still empty - as calls to the "onsuccess" function are deferred calls, they happen later in time, whereas your function returns immediately, with the (yet empty) imageHolder value.
You should be doing in this case something along those lines:
function getImageFromDevice (callback){
...
cursor.onsuccess = function () {
...
if (!this.done) {
// next picture
imageHolder[count] = file.name;
this.continue();
} else {
// no more pictures, return with the results
console.log("operation finished:");
callback(imageHolder);
}
}
}
Or use Promises in your code to accomplish the same.
Use the above by e.g.:
getImageFromDevice(function(result) {
console.log(result.length+" pictures found!");
});