Here is a functon that requests images files. How to return a promise when all files are loaded?
function request() {
for (const [src, nodes] of this.icons.entries()) {
fetch(`${this.baseHref}assets/images/${src}`)
.then((res) => res.text())
.then((content: string) => {
nodes.forEach((node: LayerNode) => {
const { icon } = node;
const { color } = icon;
const replaceIcon = getReplaceIcon(this.defPointIcon, color);
const defIcon = getDefIcon(content, color);
this.cachedIcons.set(node, { defIcon, replaceIcon });
});
});
}}
I have tried this:
const promises = [];
for (const [src, nodes] of this.icons.entries()) {
promises.push(fetch(`${this.baseHref}assets/images/${src}`));
}
Then
Promise.all(promises)
.then((res) => res.text())
.then((content: string) => {
nodes.forEach((node: LayerNode) => {
const { icon } = node;
const { color } = icon;
const replaceIcon = getReplaceIcon(this.defPointIcon, color);
const defIcon = getDefIcon(content, color);
this.cachedIcons.set(node, { defIcon, replaceIcon });
});
});
Problem is I lost the nodes.
If your problem is that you lost the reference to nodes, you can make your promise to include it in the result through a closure:
const promises = [];
for (const [src, nodes] of this.icons.entries()) {
promises.push(fetch(`${this.baseHref}assets/images/${src}`).then(result => [result, nodes]));
}
Then you should change the way you read the result:
.then(([res, nodes]) => res.text().then(content => [content, nodes]))
.then(([content, nodes]: [string, any[]]) => {
Related
I'm writing a little proof of concept thing, which downloads all my HTML assets via fetch(). Currently, I query all the tags with a compatible asset and run it through a for loop for each asset type. What I need to do is run a callback function after all the requests in the loops have finished. I tried await but it downloads each asset one by one. How can I do this?
const scripts = document.querySelectorAll("script");
const links = document.querySelectorAll("link");
const images = document.querySelectorAll("img");
for (const script of scripts) {
(async (s) => {
const doIgnore = s.getAttribute("data-ignoreload");
if (doIgnore) return;
const src = s.getAttribute("data-src");
if (!src) {
console.error("Script does not have a data-src prop.");
return;
}
fetch(src)
.then(x => x.text())
.then(r => {
const newScript = document.createElement("script");
newScript.innerHTML = r;
document.body.appendChild(newScript);
})
.catch(err => {
console.error(`Error loading script with src ${src}: ${err}`);
});
})(script);
}
for (const link of links) {
(async (l) => {
const doIgnore = l.getAttribute("data-ignoreload");
if (doIgnore) return;
const rel = l.getAttribute("rel");
if (!(rel == "stylesheet")) {
return;
}
const href = l.getAttribute("data-href");
if (!href) {
console.error("Stylesheet does not have a data-href prop.");
return;
}
fetch(href)
.then(x => x.text())
.then(r => {
const newStyle = document.createElement("style");
newStyle.innerHTML = r;
document.head.append(newStyle);
})
.catch(err => {
console.error(`Error loading stylesheet with href ${href}: ${err}`);
});
})(link);
}
for (const image of images) {
(async (i) => {
const doIgnore = i.getAttribute("data-ignoreload");
if (doIgnore) return;
const src = i.getAttribute("data-src");
if (!src) {
console.error("Image does not have a data-src prop.");
return;
}
fetch(src)
.then(x => x.blob())
.then(r => {
const url = URL.createObjectURL(r);
i.setAttribute("src", url);
})
.catch(err => {
console.error(`Error loading image ${src}: ${err}`);
});
})(image);
}
Push all of your promises into an array, and then use Promise.allSettled(promises).then((results) => {})
Documentation: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
Example:
const promises = images.map(async (image) => {
// do some async work
return fetch(whatever); // don't .catch this
})
Promise.allSettled(promises).then((results) => {
// results is a result of errors or successes
})
I'm new to javascript, and I'm trying to make some changes.
I have a search code in searchBar and with it a call to an API.json to fetch the information and put it on the site. Everything works fine, however I would like to modify it to add a Promise.all and be able to call more than one API and be able to display it on the site. But all my attempts have failed.
what do i have in js:
const localList = document.getElementById('localList');
const searchBar = document.getElementById('searchBar');
let hpWords = [];
searchBar.addEventListener('keyup', (e) => {
const searchString = e.target.value.toLowerCase();
const filteredWords = hpWords.filter((words) => {
return (
words.name.toLowerCase().includes(searchString)
);
});
displayCharacters(filteredWords);
});
const loadCharacters = async () => {
try {
const res = await fetch('https://duhnunes.github.io/api/local.json');
hpWords = await res.json();
displayCharacters(hpWords);
} catch (err) {
console.error(err);
}
};
const displayCharacters = (words) => {
const htmlString = words
.map((character) => {
return `
<div class="voc-box-content">
<h5>${character.name} - ${character.trans} [${character.type} - ${character.type2}]</h5>
<p>${character.description}</p>
</div>
`;
})
.join('');
localList.innerHTML = htmlString;
};
loadCharacters();
To display on the page I am using <div id="localList"></div>. In the example I am displaying local.json and I would also like to display item.json with <div id ="itemList"></div>.
Both have the same display structure on the <div class="voc-box-content">...</div> site.
Cheers,
DuH
To fetch words from multiple apis you can use this:
const searchBar = document.getElementById('searchBar');
const lists = {
localList: {
api: 'api1'
},
miscList: {
api: 'api2'
},
...
};
Object.entries(lists).forEach(([id, list]) => {
list.element = document.getElementById(id);
});
searchBar.addEventListener('keyup', (e) => {
const searchString = e.target.value.toLowerCase();
Object.values(lists).forEach((list) => {
list.filteredWords = list.words.filter((word) => {
return word.name.toLowerCase().includes(searchString);
});
});
displayCharacters();
});
const loadCharacters = async () => {
try {
await Promise.all(
Object.values(lists).map(async (list) => {
const response = await fetch(list.api);
list.words = await response.json();
})
);
displayCharacters();
} catch (err) {
console.error(err);
}
};
const displayCharacters = () => {
Object.values(lists).forEach((list) => {
list.element.innerHTML = (list.filteredWords ?? list.words)
.map((word) => `
<div class="voc-box-content">
<h5>${word.name} - ${word.trans} [${word.type} - ${word.type2}]</h5>
<p>${word.description}</p>
</div>`)
.join('');
});
};
loadCharacters();
I am trying to execute allCountryData and return a promise its working fine but after allCountryData is done executing I want to perform a operation on that returned data / or allCountryDataArray and store the highest values in arrayOfHighestCases
Note I can't chain the other login in allCountryData.
Please help let me know if you need any more details
export const allCountryDataArray = [];
export const arrayOfHighestCases = [];
const allCountryData = async () => {
sendHTTP()
.then((res) => {
return res.response;
})
.then((res) => {
allCountryDataArray.push(...res);
return allCountryDataArray;
});
return await allCountryDataArray;
// Highest Cases
};
The code is below is not working
const highestCasesData = async () => {
// const allCountryDataArrayy = await allCountryData();
// allCountryData()
// .then((data) => {
// console.log(arrayOfHighestCases[0]);
// })
// .then((res) => {
const np = new Promise((res, rej) => {
res(allCountryData());
});
return np.then((res) => {
console.log(res);
const arrayofHigh = allCountryDataArray.sort((a, b) => {
if (a.cases.total < b.cases.total) {
return 1;
} else if (a.cases.total > b.cases.total) {
return -1;
} else {
return 0;
}
});
console.log(arrayofHigh);
const slicedArray = arrayofHigh.slice(0, 6);
for (const eachHighCase of slicedArray) {
arrayOfHighestCases.push(eachHighCase);
}
console.log(arrayOfHighestCases);
return arrayOfHighestCases;
});
// });
};
highestCasesData();
Filling global arrays with async data is a way into timing conflicts. Bugs where the data ain't there, except when you look it is there and yet another question here on my SO about "Why can't my code access data? When I check in the console everything looks fine, but my code ain't working."
If you want to store something, store Promises of these arrays or memoize the functions.
const allCountryData = async () => {
const res = await sendHTTP();
return res.response;
};
const highestCasesData = async () => {
const allCountryDataArray = await allCountryData();
return allCountryDataArray
.slice() // make a copy, don't mutate the original array
.sort((a, b) => b.cases.total - a.cases.total) // sort it by total cases DESC
.slice(0, 6); // take the first 6 items with the highest total cases
}
This is working please let me know if I can make some more improvements
const allCountryData = async () => {
return sendHTTP()
.then((res) => {
return res.response;
})
.then((res) => {
allCountryDataArray.push(...res);
return allCountryDataArray;
});
// Highest Cases
};
const highestCasesData = async () => {
return allCountryData().then((res) => {
console.log(res);
const arrayofHigh = allCountryDataArray.sort((a, b) => {
if (a.cases.total < b.cases.total) {
return 1;
} else if (a.cases.total > b.cases.total) {
return -1;
} else {
return 0;
}
});
console.log(arrayofHigh);
const slicedArray = arrayofHigh.slice(0, 6);
for (const eachHighCase of slicedArray) {
arrayOfHighestCases.push(eachHighCase);
}
console.log(arrayOfHighestCases);
return arrayOfHighestCases;
});
};
highestCasesData();
I have a script which calls API from React and then triggers email notification function.
I was changing one part of it to call whole array of parameters instead of calling one parameter after another.
Here is part before change(working one). Console log shows correct response and I receive email notification as well.
const getApiData = () => {
const apiCall = (symbol) => {
return `https://min-api.cryptocompare.com/data/pricemulti?fsyms=${symbol}&tsyms=USD&api_key=API-KEY-HERE`
}
const MAX_CHARACKTERS = 300
let bucketArray = ['']
for (let i=0; i < assets.length - 1; i += 1) {
const symbol = `${bucketArray[bucketArray.length - 1]},${assets[i]}`
if (i === 0) {
bucketArray[0] = assets[i]
continue
}
if (symbol.length < MAX_CHARACKTERS) {
bucketArray[bucketArray.length - 1] = symbol
} else {
bucketArray[bucketArray.length] = assets[i]
}
}
const getData = () => {
Promise.all(
bucketArray.map(req => {
return axios(apiCall(req))
.then(({ data }) => data)
})
).then((data) => setDataApi(data))
}
getData()
};
Here is problematic one.
const getApiData = () => {
const getString = symbol =>
`https://min-api.cryptocompare.com/data/pricemulti?fsyms=${symbol}&tsyms=USD&api_key=API-KEY-HERE`;
function getAxious(id) {
const url = getString(id);
return axios.get(url);
}
const BUCKET_SIZE = 150;
const bucketArray = assets.reduce(
(arr, rec) => {
if (arr[arr.length - 1].length < BUCKET_SIZE) {
arr[arr.length - 1] = [...arr[arr.length - 1], rec];
return arr;
}
return [...arr, [rec]];
},
[[]]
);
bucketArray
.reduce((acc, rec) => {
return acc.then(results => {
return Promise.all(
rec.map(item =>
getAxious(item).then(({ data }) => {
return {
Symbol: item,
Open: data
};
})
)
).then(x => {
return [...x, ...results];
});
});
},
Promise.resolve([]))
.then(res => {
setDataApi(res);
});
};
Here in console I receive empty array - [] no errors showed, but email notification also stops from working.
I'm changing the code since I need to call whole array from API in one call. Before I was calling one symbol after another.
What I did wrong that console doesn't show the correct response?
EDIT1
Here is bucketArray value
const assets = ['ADA','KAVA','DOGE'];
I was not able to understand completely, but I think you want to collect all the results together and set it to the data using setDataApi.
Check the below code and let me know if it helps:
async function getApiData() {
const getString = (arr) =>
`https://min-api.cryptocompare.com/data/pricemulti?fsyms=${arr.join(
","
)}&tsyms=USD&api_key=API_KEY`;
function getAxious(arr) {
const url = getString(arr);
return axios.get(url);
}
const BUCKET_SIZE = 150;
const bucketArray = assets.reduce(
(arr, rec) => {
if (arr[arr.length - 1].length < BUCKET_SIZE) {
arr[arr.length - 1] = [...arr[arr.length - 1], rec];
return arr;
}
return [...arr, [rec]];
},
[[]]
);
const res = await getAxious(bucketArray);
console.log("res", res);
return res;
// after this you can set setDataApi(res);
}
// keep this useEffect sepearate
const [timer, setTimer] = useState(null);
useEffect(() => {
async function getApiDatahandler() {
const res = await getApiData();
console.log(res);
const timerId = setTimeout(() => {
getApiDatahandler();
}, 1000 * 60);
setTimer(timerId);
setDataApi(res)
// set the data setDataApi(res);
}
getApiDatahandler();
return () => {
window.clearTimeout(timer);
};
}, []);
// useEffect(() => {
// const timerId = setTimeout(() => {
// getApiData();
// }, 1000 * 60);
// }, [])
Checkout this codepen for a possible solution.
https://codepen.io/bcaure/pen/RwapqZW?editors=1011
In short, I don't know how to fix your code because it's quite a callback hell.
// Mock API and data
const bucketArray = [[{item: 'item1'}], [{item: 'item2'}], [{item: 'item3'}]];
const getAxious = item => {
return new Promise((resolve, reject) => resolve({data: 'API data'}));
}
// Return promise that combines API data + input item
const recToPromise = rec => rec.map(item => {
return new Promise((resolve, reject) => getAxious(item)
.then(data => resolve({item, data})));
});
// Flatten array
const recPromisesFlatten = bucketArray.flatMap(recToPromise);
Promise.all(recPromisesFlatten)
.then(res => {
const flattenRes = res.flatMap(({item, data}) => ({ Symbol: item, Open: data }));
console.log(JSON.Stringify(flattenRes))
});
What I'm suggesting to debug errors:
build your promise array first
then run Promise.all
then combine your data
Bonus: you can see flatMap instead of reduce for better readability.
I'm creating a process that converts multiple markdown files into a single pdf. It creates a pdf file for each .md file found in the source directory. Then it merges the individual pdf files into one pdf. It is this last step that is failing saying the individual pdf files do not exist.
const markdownpdf = require('markdown-pdf')
const path = require('path')
const PDFMerge = require('pdf-merge')
const fse = require('fs-extra')
const srcDir = '../manuscript'
const outDir = 'out'
const main = () => {
fse.pathExists(outDir)
.then(() => {
fse.remove(outDir).then(() => {
fse.ensureDir(outDir)
}).then(() => {
return fse.readdir(srcDir)
}).then((srcDirFiles) => {
console.log('source directory file count = ', srcDirFiles.length)
return srcDirFiles.filter(f => path.extname(f) === '.md')
}).then((mdFiles) => {
console.log('number of md files', mdFiles.length);
return mdFiles.map(file => {
const outFileName = `${path.basename(file, '.md')}.pdf`
fse.createReadStream(`${srcDir}/${file}`)
.pipe(markdownpdf())
.pipe(fse.createWriteStream(`${outDir}/${outFileName}`))
return `${outDir}/${outFileName}`
})
}).then(outFiles => {
console.log('number of pdf files created =', outFiles.length)
PDFMerge(outFiles, { output: `${__dirname}/3.pdf` })
})
})
}
main()
If I wrap the PDFMerge() line in setTimeout() it does work
setTimeout(() => {
PDFMerge(outFiles, { output: `${__dirname}/3.pdf` })
}, 1000)
I'm wondering why the setTimeout() is needed and what needs to be changed so it isn't.
I also wrote an async/await version that had the same problem and also worked with setTimeOut()
Edit
In response to Zach Holt's suggestion, here is the async/await version:
const markdownpdf = require('markdown-pdf')
const path = require('path')
const PDFMerge = require('pdf-merge')
const fse = require('fs-extra')
const srcDir = '../manuscript'
const outDir = 'out'
const createPdf = async (file) => {
try {
const outFileName = `${path.basename(file, '.md')}.pdf`
await fse.createReadStream(`${srcDir}/${file}`)
.pipe(markdownpdf())
.pipe(await fse.createWriteStream(`${outDir}/${outFileName}`))
}
catch (e) {
console.log(e)
}
}
const makePdfFiles = (files) => {
files.forEach(file => {
if (path.extname(file) === '.md') {
createPdf(file)
}
})
}
const mergeFiles = async (files) => {
try {
await PDFMerge(files, {output: `${__dirname}/3.pdf`})
}
catch (e) {
console.log(e)
}
}
const addPathToPdfFiles = (files) => {
return files.map(file => {
return `${outDir}/${file}`
})
}
const main = async () => {
try {
const exists = await fse.pathExists(outDir)
if (exists) {
await fse.remove(outDir)
}
await fse.ensureDir(outDir)
const mdFiles = await fse.readdir(srcDir)
const filesMade = await makePdfFiles(mdFiles)
const pdfFiles = await fse.readdir(outDir)
const pdfFilesWithPath = addPathToPdfFiles(pdfFiles)
mergeFiles(pdfFilesWithPath)
// setTimeout(() => {
// mergeFiles(pdfFilesWithPath)
// }, 1000)
} catch (e) {
console.log(e)
}
}
It has the same problem.
I also tried:
const makePdfFiles = files => {
return new Promise((resolve, reject) => {
try {
files.forEach(file => {
if (path.extname(file) === '.md') {
createPdf(file)
}
})
resolve(true)
} catch (e) {
reject(false)
console.log('makePdfFiles ERROR', e)
}
})
}
But it made no difference.
You need to return the promise from ensureDir() to make it wait for it.
I think the issue might be that you're creating a read stream for each of the .md files, but not waiting for the reads to finish before trying to merge outFiles.
You could likely wait until the outFiles length is the same as the number of md files found before merging.
Also, you should stick with async/await for this. It'll keep the code much clearer
Let me over-simplify your code to illustrate the problem:
p1.then(() => {
p2.then().then().then()
}).then(/* ??? */)
which is the same as:
p1.then(() => {
p2.then().then().then()
return undefined
}).then(/* undefined */)
What you need for chaining is to return the inner Promise:
p1.then(() => // no {code block} here, just return value
p2.then().then().then()
).then(/* ??? */)
which is the same as:
p1.then(() => {
p3 = p2.then()
p4 = p3.then()
p5 = p4.then()
return p5
}).then(/* p5 */)
As far as I can tell the original problem was the approach and not the obvious errors correctly pointed out by others. I found a much simpler solution to the overall goal of producing a single pdf from multiple md files.
const markdownpdf = require('markdown-pdf')
const path = require('path')
const fse = require('fs-extra')
const srcDir = '../manuscript'
const filterAndAddPath = (files) => {
try {
const mdFiles = files
.filter(f => path.extname(f) === '.md')
.map(f => `${srcDir}/${f}`)
return mdFiles
}
catch (e) {
console.log('filterAndAddPath', e)
}
}
const main4 = async () => {
const allFiles = await fse.readdir(srcDir)
const mdFiles = filterAndAddPath(allFiles)
const bookPath = 'book.pdf'
markdownpdf()
.concat.from(mdFiles)
.to(bookPath, function() {
console.log('Created', bookPath)
})
}
main4()