We Keep Coding

GroupBy Array elements from CSV file and help reduce code

Each csv file that is imported has the same data structure.
I need to sum the ['Net Charge Amount'] by each '[Service Type'].
I am currently doing this by assigning each unique ['Service Type'] to their own array. My current script is probably overkill but it is very easy to follow, however I am looking for a more compact way of doing this otherwise this script could get very long.
const fs = require('fs')
const { parse } = require('csv-parse')
// Arrays for each service type
const GroundShipments = []
const HomeDeliveryShipments = []
const SmartPostShipments = []
const Shipments = []
The [Shipments] array will hold all data and I would assume this is the array
we want to work with
//functions for each service type
function isGround(shipment) {
return shipment['Service Type'] === 'Ground'
}
function isHomeDelivery(data) {
return data['Service Type'] === 'Home Delivery'
}
function isSmartpost(shipment) {
return shipment['Service Type'] === 'SmartPost'
}
function isShipment(shipment) {
return shipment['Service Type'] === 'Ground' || shipment['Service Type'] === 'Home Delivery' ||
shipment['Service Type'] === 'SmartPost'
}
// Import csv file / perform business rules by service type
// output sum total by each service type
fs.createReadStream('repco.csv')
.pipe(parse({
columns: true
}))
.on('data', (data) => {
//push data to proper service type array
// Ground
if (isGround(data)) {
GroundShipments.push(data)
}
// Home Delivery
if (isHomeDelivery(data)) {
HomeDeliveryShipments.push(data)
}
// Smartpost
if (isSmartpost(data)) {
SmartPostShipments.push(data)
}
// All shipment types, including Ground, Home Delivery, and Smartpost
if (isShipment(data)) {
Shipments.push(data)
}
})
.on('error', (err) => {
console.log(err)
})
.on('end', (data) => {
// sum data by service type
// Ground Only
const sumGround = GroundShipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
// Home Delivery Only
const sumHomeDelivery = HomeDeliveryShipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
// SmartPost Only
const sumSmartPost = SmartPostShipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
// All services
const sumAllShipments = Shipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
//output sum by service type to console
console.log(`${GroundShipments.length} Ground shipments: ${sumGround}`)
console.log(`${HomeDeliveryShipments.length} Home Delivery shipments: ${sumHomeDelivery}`)
console.log(`${SmartPostShipments.length} Smartpost shipments: ${sumSmartPost}`)
console.log(`${Shipments.length} All shipments: ${sumAllShipments}`)
})
Here is the console output:
[1]: https://i.stack.imgur.com/FltTU.png
Instead of separating each ['Service Type'] by its own Array and Function, I would like one Array [Shipments] to output each unique ['Service Type'] and sum total of ['Net Charge Amount']

The two keys to simplifying this are:
separating the CSV parsing from the data processing
using a groupBy function
First, you should parse the CSV into a simple JS array. Then you can use regular JS utility functions to operate on the data, such as the groupBy function. It is a utility that can be found in the lodash and ramda libraries. It's probably going to be added to vanilla JS as the .group method but that's a while from now.
I was looking for a sample problem to play with my own JS evaluation framework, so I answered your question there:
You can explore the underlying val yourself: https://www.val.town/stevekrouse.exampleGroupByShppingCSV
There are a couple things about my answer that wouldn't make sense in a normal NodeJS codebase, but that I had to do to make it work in val.town (async/await, using a custom groupBy method instead of importing one). If you'd like help getting it to work in your application, just let me know.

A solution would be to use a Map instance to keep track of the stats of different service types.
For each shipment find the associated stats (based on service type), or create a new stats object { count: 0, sum: 0 }. Then increment the count, and add the amount to the sum.
When all data is iterated (on end), you can loop through the serviceTypeStats which and log the values. You can also use this loop to calculate the total by adding all count and sum of each service type group.
const serviceTypeStats = new Map();
// ...
.on('data', (shipment) => {
const serviceType = shipment['Service Type'];
const amount = parseFloat(shipment['Net Charge Amount']);
if (!serviceTypeStats.has(serviceType)) {
serviceTypeStats.set(serviceType, { count: 0, sum: 0 });
}
const stats = serviceTypeStats.get(serviceType);
stats.count += 1;
stats.sum += amount;
})
// ...
.on('end', () => {
const total = { count: 0, sum: 0 };
for (const [serviceType, stats] of serviceTypeStats) {
total.count += stats.count;
total.sum += stats.sum;
console.log(`${stats.count} ${shipmentType}: ${stats.sum}`);
}
console.log(`${total.count} All shipments: ${total.sum}`);
})
If you want to loop keys in a specific order you can define the order in an array, or sort the keys of the Map instance.
// pre-defined order
const serviceTypeOrder = ["Ground", "Home Delivery", "SmartPost"];
// or
// alphabetic order (case insensitive)
const serviceTypeOrder = Array.from(serviceTypeStats.keys());
serviceTypeOrder.sort((a, b) => a.localeCompare(b, undefined, { sensitivity: "base" }));
// ...
for (const serviceType of sericeTypeOrder) {
const stats = serviceTypeStats.get(serviceType);
// ...
}

Add data into Array in local storage, React

I'm trying to store the page Id in an array stored in local storage every time a user load a page.
I have my array, it create one if needed but for some reasons it does not update the array in new page load and keeps the first page Id.
I want to add the page id in that array on every page load if the id is not already in that array.
I've tried a lot of things but it seems like I don't understand something, any help ? Thanks
Here is my code
const [isPostId, setItems] = useState([postId]);
useEffect(() => {
//const items = JSON.parse(localStorage.getItem('items'));
if (JSON.parse(localStorage.getItem('isPostId')) == null) {
localStorage.setItem('isPostId', JSON.stringify(isPostId));
}
if (!isPostId.includes(postId)) {
JSON.parse(localStorage.getItem('isPostId'))
localStorage.setItem('isPostId', JSON.stringify(isPostId));
} },[isPostId]);
EDIT: It works now, looks like I was confused about how localStorage works, now it's clear thanks for your help everyone
Both are working:
useEffect(() => {
const storageKey = "isPostId";
const json = localStorage.getItem("isPostId");
const previousPosts = json ? JSON.parse(json) : [];
const filtered = previousPosts.filter((it) => it !== postId);
const updatedPosts = [...filtered, postId];
const stringifyed = JSON.stringify(updatedPosts);
localStorage.setItem("isPostId", stringifyed);
console.log('heu',filtered)
}, [])
useEffect(() => {
// options a - full replace
localStorage.setItem('isPostId', JSON.stringify(isPostId));
// option b - only add unique, don't remove previous
var currentIds = JSON.parse(localStorage.getItem('isPostId')) || [];
isPostId.map((e) => {
if (!currentIds.includes(e) {
currentIds.push(e);
}
})
localStorage.setItem('isPostId', JSON.stringify(currentIds));
}, [isPostId])

Right now the code in the first if statement will put ONE id in local storage if there isn't one already, but not as an array. The code in the second if statement will also only set one id. You need to be setting an array value as shown below
If isPostId is declared as an array:
useEffect(() => {
// options a - full replace
localStorage.setItem('isPostId', JSON.stringify(isPostId));
// option b - only add unique, don't remove previous
var currentIds = JSON.parse(localStorage.getItem('isPostId')) || [];
isPostId.map((e) => {
if (!currentIds.includes(e) {
currentIds.push(e);
}
})
localStorage.setItem('isPostId', JSON.stringify(currentIds));
}, [isPostId])
If isPostId is declared as a string:
If you are certain there will not be single string values in localStorage and there will only be null values or arrays, you can do this as such:
useEffect(() => {
var currentIds = JSON.parse(localStorage.getItem('isPostId')) || [];
if (!currentIds.includes(isPostId) {
currentIds.push(isPostId);
}
localStorage.setItem('isPostId', JSON.stringify(currentIds));
}, [isPostId])
If there is a possibility that there could be individual string values, you will need an additional check for the code inside the useEffect
var currentIds = JSON.parse(localStorage.getItem('isPostId'));
if (!currentIds?.length) {
currentIds = [];
} else if (typeof currentIds !== 'object') {
// value in localStorage is a single string/number rather than an array
currentIds = [currentIds]
);
if (!currentIds.includes(isPostId) {
currentIds.push(isPostId);
}
localStorage.setItem('isPostId', JSON.stringify(currentIds));
Could simplify the second chunk further if desired

If I understood the question correctly, then you need something like this solution.
useEffect(() => {
const storageKey = "isPostId";
const json = localStorage.getItem("isPostId");
const previousPosts = json ? JSON.parse(json) : [];
const updatedPosts = [...previousPosts, ...isPostId];
const uniquePosts = Array.from(new Set(updatedPosts))
const stringifyed = JSON.stringify(uniquePosts);
localStorage.setItem("isPostId", stringifyed);
}, [])

How to get all the 'items' of an API array from GitHub?

I am trying to get a list of repositories, that is my code does a search for repositories with a filter
The Javascript gets a result, with multiple items that contain the data for each repository that fit the filter using the URL: https://api.github.com/search/repositories?q=piccolowen+in:name.
I can do console.log(result.items[0].name) to get the first repository's name value, but I want get all of the repositories from the search printed to the console. I also want the code to be able to print all of the repositories and their values no matter how many repos fit the filter.
Here is the current code I want to add on to:
window.onload = func()
async function func() {
const url = 'https://api.github.com/search/repositories?q=piccolowen+in:name'
const response = await fetch(url);
const result = await response.json();
const apiresult = document.getElementById('thisisanid')
console.log(result)
}
Any ideas on how I could do this?
EDIT:
I found the answer to my problem using a while loop from this question: Get total number of items on Json object?
const resultLength = Object.keys(result.items).length
var arrnum = 0
while (arrnum < resultLength) {
//execute code
}
EDIT 2:
The code in my previous edit will crash a page. Still working on a solution for this huge bug.

Since results.items returns an array of objects, you can use Array.prototype.map to return all the item names, i.e.:
result.items.map(item => item.name)
If you want to simply filter out some properties, you can also do object destructuring. Let's say you want an array of items that only contain their name, id, and description, then you can do this:
result.items.map(({ name, id, description }) => ({ name, id, description }))
async function func() {
const url = 'https://api.github.com/search/repositories?q=piccolowen+in:name'
const response = await fetch(url);
const result = await response.json();
// Returns an array of item names
console.log(result.items.map(item => item.name));
// Returns an array of object with selected keys
console.log(result.items.map(({ name, id, description }) => ({ name, id, description })));
}
func();

The array has map function, which accepts a callback function. It iterate through all the elements and call the callback function and push data to the newly created array.
The map() method creates a new array populated with the results of calling a provided function on every element in the calling array.
More:
Array.map
const array1 = [1, 4, 9, 16];
// pass a function to map
const map1 = array1.map(x => x * 2);
console.log(map1);
// expected output: Array [2, 8, 18, 32]
window.load = main();
const nameMapper = (item) => item.name;
const liMapper = (item) => `<li>${item.name}</li>`;
async function main() {
const url = "https://api.github.com/search/repositories?q=piccolowen+in:name";
const result = await fetch(url).then((x) => x.json());
const names = result.items.map(nameMapper);
const apiresult = document.getElementById("thisisanid");
apiresult.textContent = names;
const ul = document.getElementById("list");
ul.innerHTML = result.items.map(liMapper).join("");
}
#list li{
list-style: none;
padding: 5px 10px;
border: 1px solid black;
max-width: 400px;
}
<div id="thisisanid"></div>
<ul id="list">
</ul>

You can use like!
let list = document.getElementById('list');
let htmlTemplate = result.items.map(function(item) {
return item.name
}).join("")
list.insertAdjacentHTML('beforeend', htmlTemplate)
or you can use template literal
foe example
when you returning value in items.map()
return `${item.id}: ${item.name}`

How to get object properties on element click?

I am building a weather app for practice. I get to that point that I have to make an autocomplete input field with data from JSON object. When someone makes an input, it displays the matched data, but on click I want to get two properties from the object. I need to get the longitude and latitude properties from JSON object to make an API request to return the object with the weather data. The content displays properly but I can't make that onClick event listener work. I tried very different things and failed, either was a scope problem or something else. It is one of my first projects and I am in a downfall right now. Please help me. :)
P.S. You can find it on this link: https://objective-edison-1d6da6.netlify.com/
// Testing
const search = document.querySelector('#search');
const matchList = document.querySelector('#match-list');
let states;
// Get states
const getStates = async () => {
const res = await fetch('../data/bg.json');
states = await res.json();
};
// Filter states
const searchStates = searchText => {
// Get matches to current text input
let matches = states.filter(state => {
const regex = new RegExp(`^${searchText}`, 'gi');
return state.city.match(regex);
});
// Clear when input or matches are empty
if (searchText.length === 0) {
matches = [];
matchList.innerHTML = '';
}
outputHtml(matches);
};
// Show results in HTML
const outputHtml = matches => {
if (matches.length > 0) {
const html = matches
.map(
match => `<div class="card match card-body mb-1">
<h4>${match.city}
<span class="text-primary">${match.country}</span></h4>
<small>Lat: ${match.lat} / Long: ${match.lng}</small>
</div>`
)
.join('');
matchList.innerHTML = html;
document.querySelector('.match').addEventListener('click', function() {});
//Wconsole.log(matches);
//let test = document.querySelectorAll('#match-list .card');
//const values = matches.values(city);
}
};
window.addEventListener('DOMContentLoaded', getStates);
search.addEventListener('input', () => searchStates(search.value));

If I understand correctly, you're trying to access the lat and lng values of the clicked match, if that is the case, here is one way of doing it:
const outputHtml = matches => {
if (matches.length > 0) {
const html = matches
.map(
match => `<div class="card match card-body mb-1" data-lat="`${match.lat}" data-lng="`${match.lng}">
<h4>${match.city}
<span class="text-primary">${match.country}</span></h4>
<small>Lat: ${match.lat} / Long: ${match.lng}</small>
</div>`
)
.join('');
matchList.innerHTML = html;
document.querySelectorAll('.match').forEach(matchElm => {
matchElm.addEventListener('click', function(event) {
const { currentTarget } = event;
const { lat, lng } = currentTarget.dataset;
});
});
}
};
I've used the data-lat and data-lng attributes to store the required values in the element's dataset and I've used document.querySelectorAll('.match') to get all the elements that have the class match not just the first one.

Filter/Reject Array of strings against multiple values using underscore

I'd like to _.filter or _.reject the cities array using the filters array using underscore.
var cities = ['USA/Aberdeen', 'USA/Abilene', 'USA/Akron', 'USA/Albany', 'USA/Albuquerque', 'China/Guangzhou', 'China/Fuzhou', 'China/Beijing', 'China/Baotou', 'China/Hohhot' ... ]
var filters = ['Akron', 'Albuquerque', 'Fuzhou', 'Baotou'];
My progress so far:
var filterList;
if (reject) {
filterList = angular.copy(cities);
_.each(filters, (filter) => {
filterList = _.reject(filterList, (city) => city.indexOf(filter) !== -1);
});
} else {
filterList = [];
_.each(filters, (filter) => {
filterList.push(_.filter(cities, (city) => city.indexOf(filter) !== -1));
});
}
filterList = _.flatten(filterList);
return filterList;
I'd like to DRY this up and use a more functional approach to achieve this if possible?

A somewhat more functional version using Underscore might look like this:
const cities = ['USA/Aberdeen', 'USA/Abilene', 'USA/Akron', 'USA/Albany',
'USA/Albuquerque', 'China/Guangzhou', 'China/Fuzhou',
'China/Beijing', 'China/Baotou', 'China/Hohhot']
const filters = ['Akron', 'Albuquerque', 'Fuzhou', 'Baotou'];
var inList = names => value => _.any(names, name => value.indexOf(name) > -1);
_.filter(cities, inList(filters));
//=> ["USA/Akron", "USA/Albuquerque", "China/Fuzhou", "China/Baotou"]
_.reject(cities, inList(filters));
//=> ["USA/Aberdeen", "USA/Abilene", "USA/Albany",
// "China/Guangzhou", "China/Beijing", "China/Hohhot"]

I'm using vanilla JavaScript here (some() and filter()) but I hope you get the idea:
const isValidCity = city => filters.some(filter => city.indexOf(filter) > -1)
const filteredCities = cities.filter(isValidCity)
Please note that this is a loop over a loop. So the time complexity is O(n * m) here.

In your example all city keys share the same pattern: country + / + city. Your filters are all an exact match to the city part of these names.
If this is a certainty in your data (which it probably isn't...), you could reduce the number of loops your code makes by creating a Map or object that stores each city per filter entry:
Create an object with an entry for each city name
Make the key the part that you want the filter to match
Make the value the original name
Loop through the filters and return the name at each key.
This approach always requires one loop through the data and one loop through the filters. For small array sizes, you won't notice a performance difference. When one of the arrays has length 1, you'll also not notice any differences.
Again, note that this only works if there's a constant relation between your filters and cities.
var cities = ['USA/Aberdeen', 'USA/Abilene', 'USA/Akron', 'USA/Albany', 'USA/Albuquerque', 'China/Guangzhou', 'China/Fuzhou', 'China/Beijing', 'China/Baotou', 'China/Hohhot' ]
var filters = ['Akron', 'Albuquerque', 'Fuzhou', 'Baotou'];
const makeMap = (arr, getKey) => arr.reduce(
(map, x) => Object.assign(map, {
[getKey(x)]: x
}), {}
);
const getProp = obj => k => obj[k];
const getKeys = (obj, keys) => keys.map(getProp(obj));
// Takes the part after the "/"
const cityKey = c => c.match(/\/(.*)/)[1];
const cityMap = makeMap(cities, cityKey);
const results = getKeys(cityMap, filters);
console.log(results);

Since you seem to be using AngularJS, you could utilize the built-in filter functionality. Assuming both the cities and filters array exist on your controller and you're displaying the cities array using ng-repeat, you could have something like this on your controller:
function cityFilter(city) {
var cityName = city.split('/')[1];
if (reject) {
return filters.indexOf(cityName) === -1;
} else {
return filters.indexOf(cityName) > -1;
}
}
And then in your template, you'd do something like this:
<div ng-repeat="city in cities | filter : cityFilter"></div>
Of course you'd have to modify your syntax a bit depending on your code style (for example, whether you use $scope or controllerAs).