Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
I try to build the Javascript equvivalent for Java's IntStream.range(0, 5).forEach(System.err::println); and reached
const IntStream = (function () {
function range(start, end, numbers = []) {
if (start === end) {
return numbers
}
return range(start + 1, end, numbers.concat(start))
}
return {
range
}
})()
IntStream.range(0, 5).forEach(number => console.log(number))
All the stream magic of Java is builtin in a normal JavaScript array. Why can't an ArrayList in Java do all same things as a Stream or is there a purpose I didn't figure out yet?
Array higher order functions will eagerly do the whole thing at each step.
const isOdd = v => v % 2 == 1;
const multiply = by => v => v * by;
const arrRange = IntStream.range(10, 20);
const arrOdd = arrRange.filter(isOdd);
const arrOddM3 = arrOdd.map(multiply(3));
Here all the bindings are distinct arrays created by each of the steps. Even when you chain them the intermediate arrays are always made and the whole array at each step need to be finished before the next can begin.
const arrOddM3 = IntStream.range(10, 20).filter(isOdd).map(multiply(3));
arrOddM3; // ==> [33, 39, 45, 51, 57]
Streams are different since they only compute values when they are accessed. A stream version would look very similar.
const streamOddM3 = Stream.range(10, Infinity).filter(isOdd).map(multiply(3));
streamOddM3; // ==> Stream
Notice I have changed the end to go to infinity. I can do that because at most it calculates the very first value and some implementations doesn't do any calculations at all until you ask for the values. To force the calculations you can take some values and get them returned as an array:
streamOddM3.take(3); // ==> [33, 39, 45]
Here is a Stream implementation loosely based on the one from the SICP videos which work similar to Java's streams.
class EmptyStream {
map() {
return this;
}
filter() {
return this;
}
take() {
return [];
}
}
class Stream extends EmptyStream {
constructor(value, next) {
super();
this._next = next;
this.value = value;
}
/**
* This prevents the value to be computed more than once
* #returns {EmptyStream|Stream}
*/
next() {
if( ! (this._next instanceof EmptyStream) ) {
this._next = this._next();
}
return this._next;
}
map(fn) {
return new Stream(fn(this.value), () => this.next().map(fn));
}
filter(fn) {
return fn(this.value) ?
new Stream(this.value, () => this.next().filter(fn)) :
this.next().filter(fn);
}
take(n) {
return n == 0 ? [] : [this.value, ...this.next().take(n && n - 1)];
}
static range(from, to, step = 1) {
if (to !== undefined && ( step > 0 && from > to || step < 0 && from < to )) {
return Stream.emptyStream;
}
return new Stream(from, () => Stream.range(from + step, to, step));
}
}
Stream.emptyStream = new EmptyStream();
There are alternatives to Stream that might work in their place.
In JavaScript you have generators (aka coroutines) and you can make a map and filter generator function that takes a generator source and becomes a new generator with that transformation. Since it is already in the language it might be a better match than Streams but I haven't studied it enough to make a generator example of the above.
In Clojure you have transducers that allows you to compose steps so that an eventual list making only happens for the elements that makes it to the final result. They are easily implemented in JavaScript.
Theres a big difference between Streams and Javasvript arrays:
[1,2,3,4]
.filter(el => {
console.log(el);
return el%2 === 0;
})
.forEach( el => console.log(el));
The result in javascript will be:
1,2,3,4 2,4
for a Stream it will be:
1,2 2 3,4 4
So as you can see javascript mutates the collection, then iterates the collection. An element passed into a Stream traverses the stream. If a collection is passed to a Stream, one element after another will be passed in the stream.
A possible Stream implementation would be:
class Stream {
constructor(){
this.queue = [];
}
//the modifying methods
forEach(func){
this.queue.push(["forEach",func]);
return this;
}
filter(func){
this.queue.push(["filter",func]);
return this;
}
map(func){
this.queue.push(["map",func]);
return this;
}
subStream(v){
this.forEach(d => v.get(d));
return this;
}
//data methods
get(value,cb){
for( let [type,func] of this.queue ){
switch(type){
case "forEach":
func(value);
break;
case "map":
value = func(value);
break;
case "filter":
if(! func(value)) return;
}
}
cb(value);
}
range(start,end){
const result = [];
Array.from({length:end-start})
.forEach((_,i)=> this.get(i+start, r => result.push(r)));
return result;
}
}
Usecase:
const nums = new Stream();
const even = new Stream();
even.filter(n => !(n%2) ).forEach(n => console.log(n));
const odd = new Stream();
even.filter(n => (n%2) ).forEach(n => console.log(n));
nums
.subStream(even)
.subStream(odd)
.range(0,100);
No they are not the same because of how they proccess the data.
In LINQ (C#) or javascript, each operation on a collection must end befor calling to the next operation in the pipeline.
In streams, its different. For example:
Arrays.asList(1,2,3).stream()
.filter((Integer x)-> x>1)
.map((Integer x)->x*10)
.forEach(System.out::println);
source collection: 1, 2 ,3
filter(1) -> You are not OK. Element 1 will not pass to the next operation
in the pipeline. Now deal with element 2.
filter(2) -> You are OK. element 2 pass to the next operation.
map(2) -> create new element 20 and put it in the new stream.
forEach(20) -> print 20. End dealing with element 2 in the source collection.
Now deal with element 3.
filter(3) -> You are OK. element 3 pass to the next operation
map(3) -> create new element 30 and put it in the new stream.
forEach(20) -> print 30. No more elements in the source collection.
finish excuting the stream.
output:
20
30
Illustration:
One of the outcome of this approach is sometimes some operations in the pipeline won't go over each element because some of them filtered out in the proccess.
This explanation were taken from: Streams In Depth By Stav Alfi
Related
Each csv file that is imported has the same data structure.
I need to sum the ['Net Charge Amount'] by each '[Service Type'].
I am currently doing this by assigning each unique ['Service Type'] to their own array. My current script is probably overkill but it is very easy to follow, however I am looking for a more compact way of doing this otherwise this script could get very long.
const fs = require('fs')
const { parse } = require('csv-parse')
// Arrays for each service type
const GroundShipments = []
const HomeDeliveryShipments = []
const SmartPostShipments = []
const Shipments = []
The [Shipments] array will hold all data and I would assume this is the array
we want to work with
//functions for each service type
function isGround(shipment) {
return shipment['Service Type'] === 'Ground'
}
function isHomeDelivery(data) {
return data['Service Type'] === 'Home Delivery'
}
function isSmartpost(shipment) {
return shipment['Service Type'] === 'SmartPost'
}
function isShipment(shipment) {
return shipment['Service Type'] === 'Ground' || shipment['Service Type'] === 'Home Delivery' ||
shipment['Service Type'] === 'SmartPost'
}
// Import csv file / perform business rules by service type
// output sum total by each service type
fs.createReadStream('repco.csv')
.pipe(parse({
columns: true
}))
.on('data', (data) => {
//push data to proper service type array
// Ground
if (isGround(data)) {
GroundShipments.push(data)
}
// Home Delivery
if (isHomeDelivery(data)) {
HomeDeliveryShipments.push(data)
}
// Smartpost
if (isSmartpost(data)) {
SmartPostShipments.push(data)
}
// All shipment types, including Ground, Home Delivery, and Smartpost
if (isShipment(data)) {
Shipments.push(data)
}
})
.on('error', (err) => {
console.log(err)
})
.on('end', (data) => {
// sum data by service type
// Ground Only
const sumGround = GroundShipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
// Home Delivery Only
const sumHomeDelivery = HomeDeliveryShipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
// SmartPost Only
const sumSmartPost = SmartPostShipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
// All services
const sumAllShipments = Shipments.reduce((acc, data) =>
acc + parseFloat(data['Net Charge Amount']), 0)
//output sum by service type to console
console.log(`${GroundShipments.length} Ground shipments: ${sumGround}`)
console.log(`${HomeDeliveryShipments.length} Home Delivery shipments: ${sumHomeDelivery}`)
console.log(`${SmartPostShipments.length} Smartpost shipments: ${sumSmartPost}`)
console.log(`${Shipments.length} All shipments: ${sumAllShipments}`)
})
Here is the console output:
[1]: https://i.stack.imgur.com/FltTU.png
Instead of separating each ['Service Type'] by its own Array and Function, I would like one Array [Shipments] to output each unique ['Service Type'] and sum total of ['Net Charge Amount']
The two keys to simplifying this are:
separating the CSV parsing from the data processing
using a groupBy function
First, you should parse the CSV into a simple JS array. Then you can use regular JS utility functions to operate on the data, such as the groupBy function. It is a utility that can be found in the lodash and ramda libraries. It's probably going to be added to vanilla JS as the .group method but that's a while from now.
I was looking for a sample problem to play with my own JS evaluation framework, so I answered your question there:
You can explore the underlying val yourself: https://www.val.town/stevekrouse.exampleGroupByShppingCSV
There are a couple things about my answer that wouldn't make sense in a normal NodeJS codebase, but that I had to do to make it work in val.town (async/await, using a custom groupBy method instead of importing one). If you'd like help getting it to work in your application, just let me know.
A solution would be to use a Map instance to keep track of the stats of different service types.
For each shipment find the associated stats (based on service type), or create a new stats object { count: 0, sum: 0 }. Then increment the count, and add the amount to the sum.
When all data is iterated (on end), you can loop through the serviceTypeStats which and log the values. You can also use this loop to calculate the total by adding all count and sum of each service type group.
const serviceTypeStats = new Map();
// ...
.on('data', (shipment) => {
const serviceType = shipment['Service Type'];
const amount = parseFloat(shipment['Net Charge Amount']);
if (!serviceTypeStats.has(serviceType)) {
serviceTypeStats.set(serviceType, { count: 0, sum: 0 });
}
const stats = serviceTypeStats.get(serviceType);
stats.count += 1;
stats.sum += amount;
})
// ...
.on('end', () => {
const total = { count: 0, sum: 0 };
for (const [serviceType, stats] of serviceTypeStats) {
total.count += stats.count;
total.sum += stats.sum;
console.log(`${stats.count} ${shipmentType}: ${stats.sum}`);
}
console.log(`${total.count} All shipments: ${total.sum}`);
})
If you want to loop keys in a specific order you can define the order in an array, or sort the keys of the Map instance.
// pre-defined order
const serviceTypeOrder = ["Ground", "Home Delivery", "SmartPost"];
// or
// alphabetic order (case insensitive)
const serviceTypeOrder = Array.from(serviceTypeStats.keys());
serviceTypeOrder.sort((a, b) => a.localeCompare(b, undefined, { sensitivity: "base" }));
// ...
for (const serviceType of sericeTypeOrder) {
const stats = serviceTypeStats.get(serviceType);
// ...
}
I am wondering what the best way is to prevent duplicate data from getting into a new array. I have a service call that returns the same array 3 times. I'm trying to take a number from inside the objects in the array and add them up to create a "total" number (fullRentAmt), but since the array gets returned 3 times I'm getting the total*3. I am thinking maybe .some() or .filter() could be of use here but I've never used those/am not sure how that would be implemented here. Thanks for any help!
What I tried, but it's not working/the new array isn't getting populated:
Component
properties = [];
fullRentAmt: number = 0;
const propertyDataSub = this.mainService.requestPropertyData()
.subscribe((pData: PropertyData[]) => {
if (pData) {
const propertyData = pData;
for (let i = 0; i < propertyData.length; i++) {
if (this.properties[i].propertyId !== propertyData[i].propertyId) {
this.properties.push(propertyData[i]);
}
}
for (let i = 0; i < this.properties.length; i++) {
this.fullRentAmt += this.properties[i].tenancyInformation[0].rentAmount;
}
});
Data returned from backend (array of 2 objects):
[
{
"tenantsData":[
{
"email":null,
"tenantNames":null,
"propertyId":2481,
}
],
"tenancyInformation":[
{
"id":2487,
"rentAmount":1000,
}
],
},
{
"tenantsData":[
{
"email":null,
"tenantNames":null,
"propertyId":3271,
}
],
"tenancyInformation":[
{
"id":3277,
"rentAmount":1200,
}
],
},
I'm not an angular developer, but I hope my answer will help you.
let the for loop duplicate the data as much as it wants. you just have to change the idea of storing the stuff from an array to a JavaScript Set
basically, it's very similar to arrays they're lists and iteratables that are very similar to arrays, the only difference is that they don't allow duplication,
usage:
const properties = new Set()
properties.add("yellow")
properties.add("blue")
properties.add("orange")
console.log(properties) // yellow, blue, orange
properties.add("blue")
properties.add("blue")
properties.add("blue")
console.log(properties) // yellow, blue, orange
after your for loop finishes, you may want to convert this set into a normal array, all you have to do is to use destructuring:
const propertiesArray = [...properties]
#YaYa is correct. I added this to show the correct code in Angular
properties = [];
fullRentAmt: number = 0;
const propertyDataSub = this.mainService.requestPropertyData()
.subscribe((pData: PropertyData[]) => {
if (pData && pData.length) {
let arrSet = new Set()
const propertyData = pData;
for (let i = 0; i < propertyData.length; i++) {
if (this.properties[i].propertyId !== propertyData[i].propertyId) {
arrSet.add(propertyData[i])
}
}
this.properties = Array.from(arrSet);
for (let i = 0; i < this.properties.length; i++) {
this.fullRentAmt += this.properties[i].tenancyInformation[0].rentAmount;
}
});
First thing you need to do is to fix your server and return the list once.
If server is out of your reach, you can leverage distinctUntilChanged pipe in combination with isEqual method in the frontend. You can either implement it yourself, or use a library such as lodash.
Also you do not have to subscribe, use async pipe in the template.
this.properties$ = this.mainService.requestPropertyData()
.pipe(
distinctUntilChanged(isEqual) // provide isEqual function somehow
);
this.totalRentAmount$ = properties$.pipe(
map(getTotalRentAmount)
);
// maybe in some other utility file:
export const getTotalRentAmount = (properties: Property[]): number => {
return properties
.map(property => property.tenancyInformation.rentAmount)
.reduce((total, amount) => total + amount, 0);
}
Then in the template:
<div>Total Rent Amount: {{ totalRentAmount | async }}</div>
Also if you really need to subscribe in the component and are only interested in the first emitted value of an observable, you can use first() or take(1) pipe to automatically unsubscribe after first value.
this.mainService.requestPropertyData()
.pipe(
first() // or take(1)
)
.subscribe(properties => this.properties = properties);
See the difference between first() and take(1)
What is the best way to implement a Set of coordinates in JavaScript? I would like to be able to do things like:
let s=new Set();
s.add([1,1]);
if (s.has([1,1])) // false, since these are different arrays
The above doesn't work, since the Set is storing a reference to the array instead of the contents.
You can subclass Set for more flexibility.
class ObjectSet extends Set{
add(elem){
return super.add(typeof elem === 'object' ? JSON.stringify(elem) : elem);
}
has(elem){
return super.has(typeof elem === 'object' ? JSON.stringify(elem) : elem);
}
}
let s=new ObjectSet();
s.add([1,1]);
console.log(s.has([1,1]))
console.log(s.has([1,2,3]));
console.log([...s]);
console.log([...s].map(JSON.parse));//get objects back
This can be done with strings:
let s=new Set();
s.add("1,1");
s.add("2,2");
console.log(s.has("1,1"), s.has("1,2")); // true false
However, I would prefer to do this with some type of numeric tuple to avoid repeated string conversion logic.
If you only plan to store pairs of coords, another possibility is to use a combination of a Map (for the first coord) and a Set (for the second coord).
function TupleSet() {
this.data = new Map();
this.add = function([first, second]) {
if (!this.data.has(first)) {
this.data.set(first, new Set());
}
this.data.get(first).add(second);
return this;
};
this.has = function([first, second]) {
return (
this.data.has(first) &&
this.data.get(first).has(second)
);
};
this.delete = function([first, second]) {
if (!this.data.has(first) ||
!this.data.get(first).has(second)
) return false;
this.data.get(first).delete(second);
if (this.data.get(first).size === 0) {
this.data.delete(first);
}
return true;
};
}
let mySet = new TupleSet();
mySet.add([0,2]);
mySet.add([1,2]);
mySet.add([0,3]);
console.log(mySet.has([1,3]));
console.log(mySet.has([0,2]));
mySet.delete([0,2]);
console.log(mySet.has([0,2]));
Unfortunately, unlike a normal Set for which:
You can iterate through the elements of a set in insertion order.
— MDN Set
This approach will, for the example above, iterate in the order:
[0,2]
[0,3]
[1,2]
I was building a game when I came across this problem.
Here's a typescript class that might be able to help you. It uses a tree to do its magic.
You should be able to easily modify this to use arrays instead of the x and y parameters
// uses an internal tree structure to simulate set-like behaviour
export default class CoordinateSet {
tree: Record<number, Record<number, boolean>> = {}
add(x: number, y: number) {
this.tree[x] ||= {}
this.tree[x][y] = true;
}
remove(x: number, y: number) {
// if the coordinate doesn't exist, don't do anything
if (!this.tree[x] || !this.tree[y]) {
return;
}
// otherwise, delete it
delete this.tree[x][y];
// if the branch has no leaves, delete the branch, too
if (!Object.keys(this.tree[x]).length) {
delete this.tree[x]
}
}
has(x: number, y: number) {
return !!this.tree[x]?.[y];
}
}
And tests, which will also show you how it works:
import CoordinateSet from "./CoordinateSet";
describe("CoordinateSet", () => {
it("Can add a coordinate", () => {
const cs = new CoordinateSet();
expect(cs.has(1,1)).toBeFalsy();
cs.add(1, 1);
expect(cs.has(1,1)).toBeTruthy();
});
it("Can remove a coordinate", () => {
const cs = new CoordinateSet();
cs.add(1, 1);
expect(cs.has(1,1)).toBeTruthy();
cs.remove(1,1);
expect(cs.has(1,1)).toBeFalsy();
})
})
If we can assume that our tuples are finite integers, they could be encoded as a float.
class TupleSet extends Set{
add(elem){
return super.add((typeof elem === 'object'
&& Number.isSafeInteger(elem[0])
&& Number.isSafeInteger(elem[1]))
? elem[0]+elem[1]/10000000 : elem);
}
has(elem){
return super.has((typeof elem === 'object'
&& Number.isSafeInteger(elem[0])
&& Number.isSafeInteger(elem[1]))
? elem[0]+elem[1]/10000000 : elem);
}
}
function TupleSetParse(elem){
return (Number.isFinite(elem)?
[Math.round(elem),Math.round((elem-Math.round(elem))*10000000)]:elem);
}
let s=new TupleSet();
s.add([1,5]);
s.add([1000000,1000000]);
s.add([-1000000,-1000000]);
console.log(s.has([1,5])); // true
console.log(s.has([1,2])); // false
console.log([...s].map(TupleSetParse));
// [ [ 1, 5 ], [ 1000000, 1000000 ], [ -1000000, -1000000 ] ]
Of course, this is limited in range. And it is fragile to some malformed input, so additional error checking should be added. However, after some testing, this method is only 25% better in speed and memory usage than the JSON.stringify approach. So, JSON is the preferred approach.
I am building a simple todo app, and I'm trying to get the assigned users for each task. But let's say that in my database, for some reason, the tasks id starts at 80, instead of starting at 1, and I have 5 tasks in total.
I wrote the following code to get the relationship between user and task, so I would expect that at the end it should return an array containing 5 keys, each key containing an array with the assigned users id to the specific task.
Problem is that I get an array with 85 keys in total, and the first 80 keys are undefined.
I've tried using .map() instead of .forEach() but I get the same result.
let assignedUsers = new Array();
this.taskLists.forEach(taskList => {
taskList.tasks.forEach(task => {
let taskId = task.id;
assignedUsers[taskId] = [];
task.users.forEach(user => {
if(taskId == user.pivot.task_id) {
assignedUsers[taskId].push(user.pivot.user_id);
}
});
});
});
return assignedUsers;
I assume the issue is at this line, but I don't understand why...
assignedUsers[taskId] = [];
I managed to filter and remove the empty keys from the array using the line below:
assignedUsers = assignedUsers.filter(e => e);
Still, I want to understand why this is happening and if there's any way I could avoid it from happening.
Looking forward to your comments!
If your taskId is not a Number or autoconvertable to a Number, you have to use a Object. assignedUsers = {};
This should work as you want it to. It also uses more of JS features for the sake of readability.
return this.taskLists.reduce((acc, taskList) => {
taskList.tasks.forEach(task => {
const taskId = task.id;
acc[taskId] = task.users.filter(user => taskId == user.pivot.task_id);
});
return acc;
}, []);
But you would probably want to use an object as the array would have "holes" between 0 and all unused indexes.
Your keys are task.id, so if there are undefined keys they must be from an undefined task id. Just skip if task id is falsey. If you expect the task id to possibly be 0, you can make a more specific check for typeof taskId === undefined
this.taskLists.forEach(taskList => {
taskList.tasks.forEach(task => {
let taskId = task.id;
// Skip this task if it doesn't have a defined id
if(!taskId) return;
assignedUsers[taskId] = [];
task.users.forEach(user => {
if(taskId == user.pivot.task_id) {
assignedUsers[taskId].push(user.pivot.user_id);
}
});
});
});
Hello I'm trying to figure out if there is an equivalent to the RxJs operator zip in xstream, or at least a way to get the same behaviour. In case anyone needs clarification on the difference the marble diagrams below will show.
zip in rxjs
|---1---2---3-----------5->
|-a------b------c---d----->
"zip"
|-1a----2b------3c-----5d->
whereas 'combineLatest' aka 'combine' in xstream does
|---1---2----------4---5->
|----a---b---c---d------->
"combine"
|-1a----2a-2b-2c-2d-4d-5d>
Any help is appreciated as I'm very new to programming with streams. Thank you in advance!
I also needed a zip operator for xstream. So I created my own from existing operators. It takes an arbitrary number of streams for zipping.
function zip(...streams) {
// Wrap the events on each stream with a label
// so that we can seperate them into buckets later.
const streamsLabeled = streams
.map((stream$, idx) => stream$.map(event => ({label: idx + 1, event: event})));
return (event$) => {
// Wrap the events on each stream with a label
// so that we can seperate them into buckets later.
const eventLabeled$ = event$.map(event => ({label: 0, event: event}));
const labeledStreams = [eventLabeled$, ...streamsLabeled];
// Create the buckets used to store stream events
const buckets = labeledStreams.map((stream, idx) => idx)
.reduce((buckets, label) => ({...buckets, [label]: []}), {});
// Initial value for the fold operation
const accumulator = {buckets, tuple: []};
// Merge all the streams together and accumulate them
return xs.merge(...labeledStreams).fold((acc, event) => {
// Buffer the events into seperate buckets
acc.buckets[event.label].push(event);
// Does the first value of all the buckets have something in it?
// If so, then there is a complete tuple.
const tupleComplete = Object.keys(acc.buckets)
.map(key => acc.buckets[key][0])
.reduce((hadValue, value) => value !== undefined
? true && hadValue
: false && hadValue,
true);
// Save completed tuple and remove it from the buckets
if (tupleComplete) {
acc.tuple = [...Object.keys(acc.buckets).map(key => acc.buckets[key][0].event)];
Object.keys(acc.buckets).map(key => acc.buckets[key].shift());
} else {
// Clear tuple since all columns weren't filled
acc.tuple = [];
}
return {...acc};
}, accumulator)
// Only emit when we have a complete tuple
.filter(buffer => buffer.tuple.length !== 0)
// Just return the complete tuple
.map(buffer => buffer.tuple);
};
}
This can be used with compose.
foo$.compose(zip(bar$)).map(([foo, bar]) => doSomething(foo, bar))