with my current project, I am dealing with large streams of numerical data and transformations that have to take place on them in a data-flow-programmable fashion.
I stumbled upon the idea of transducers, which promised to solve the difficulties to handle multiple transformations on large arrays. It seems that transducers don't suit exactly for what I'm trying to solve here.
I am looking for a pattern / concept for transducers which only collect a needed amount of lookback to then process out a result. Similar to the browser version of tensorflow, reaktor, max-msp (input outputs, flow-graphs, node-based, visual-programming)
Most of these modules, should be connected to a source, but should also be able to act as a source to chain those to other modules
source ( a stream ) =[new-value]|=> module1 => module2 => ...
|=> module3 => module4 // branch off here to a new chain
From my understanding, the transducers as explained in most blogs takes the whole array, and feeds each individual values trough chosen transformers.
Yet my modules/transformers don't require so much data to work, say the example of a simple moving average with a look back of 4 steps.
I imagine that module to collect enough data until it starts it's output.
I also don't need to hold the whole array in memory, I should only deal with the exact amounts needed. Results/Outputs would be optionally stored in a database.
stream =[sends-1-value]=> module[collects-values-until-processing-starts] =[sends-one-value]=>...
It should also be possible to connect multiple sources into a module (which transducers didn't seem to provide.
Would the transducer pattern here still apply or is something else out there?
To be honest, every programmer would have an idea to make this work, yet I am asking for some established way of doing it, just like transducers came to be.
The transducer pattern certainly applies here. You can create a floating point processor with transducers paired with the right data structure. I'll give you a baseline example, with one assumption:
the stream you are working with implements Symbol.asyncIterator
Consider a simple queue
function SimpleQueue({ size }) {
this.size = size
this.buffer = []
}
SimpleQueue.prototype.push = function(item) {
this.buffer.push(item)
if (this.buffer.length > this.size) {
this.buffer.shift()
}
return this
}
SimpleQueue.prototype[Symbol.iterator] = function*() {
for (const item of this.buffer) {
yield item
}
}
Our simple queue has one method push that pushes an item into its internal buffer (an array). The simple queue is also iterable, so you could do for (const x of simpleQueue) {/* stuff */}
We'll now use our SimpleQueue in our floating point processor.
const average = iterable => {
let sum = 0, count = 0
for (const item of iterable) {
sum += item
count += 1
}
return sum / count
}
const floatingPointAverage = ({ historySize }) => {
const queue = new SimpleQueue({ size: historySize })
return item => {
queue.push(item)
const avg = average(queue)
console.log(queue, avg) // this shows the average as the process runs
return avg
}
}
floatingPointAverage takes an item, pushes it into our SimpleQueue, and returns the current average of items in the queue.
Finally, we can implement and consume our transducer
const { pipe, map, transform } = require('rubico')
const numbersStream = {
[Symbol.asyncIterator]: async function*() {
for (let i = 0; i < 1000; i++) yield i
},
}
transform(
pipe([
map(floatingPointAverage({ historySize: 4 })),
/* transducers that do stuff with floating point average here */
]),
null,
)(numbersStream)
The transducer in this case is map(floatingPointAverage({ historySize: 4 })). This transducer is courtesy of rubico, a library I wrote to solve my own async problems. I write about transducers in the context of rubico here
Your output should look like this
SimpleQueue { size: 4, buffer: [ 0 ] } 0
SimpleQueue { size: 4, buffer: [ 0, 1 ] } 0.5
SimpleQueue { size: 4, buffer: [ 0, 1, 2 ] } 1
SimpleQueue { size: 4, buffer: [ 0, 1, 2, 3 ] } 1.5
SimpleQueue { size: 4, buffer: [ 1, 2, 3, 4 ] } 2.5
SimpleQueue { size: 4, buffer: [ 2, 3, 4, 5 ] } 3.5
SimpleQueue { size: 4, buffer: [ 3, 4, 5, 6 ] } 4.5
SimpleQueue { size: 4, buffer: [ 4, 5, 6, 7 ] } 5.5
SimpleQueue { size: 4, buffer: [ 5, 6, 7, 8 ] } 6.5
SimpleQueue { size: 4, buffer: [ 6, 7, 8, 9 ] } 7.5
Related
I'm trying to migrate a Python code to Javascript and I'm facing same issues on my calculations.
I also have no documentation to base on except this code:
var commonDenominator = ((myClassMap map (sub) -> sub reduce ((item, denominator = 0) -> item * perClassMultiplier[$$])) reduce ($$+$)) as Number
From what I understand, it's trying to read the myClassMap and trying to find the common denominator of everything.
I'm trying to convert but my code is doing a greatCommonDenominator and my calculations are wrong in the order of 100, I believe. Maybe I'm wrong...
Entry:
myClassMap -> [ [ 1, 150 ], [ 1, 0 ], [ 1.5, 8 ], [ 1.5, 0 ] ]
perClassMultiplier -> [ 1, 1, 1.5, 1.5 ]
Code
myClassMap.map(entry => {
const commonDenominator = greaterCommonDenominator(entry[0], entry[1]);
if(data.commonDenominator == null | commonDenominator > data.commonDenominator) {
data.commonDenominator = commonDenominator;
}
});
const greaterCommonDenominator = function(a, b) {
if (!b) {
return a;
}
return greaterCommonDenominator(b, a % b);
}
Any help? I want to make sure the code does the same calculation
I recently got into functional programming bit by bit; started trying to point-free every function I code today.
I was practicing using Ramda while I was coding a binary search algorithm.
const R = require("ramda");
const getMidIndex = R.compose(
R.call(R.invoker(1, "floor"), R.__, Math),
R.divide(R.__, 2),
R.apply(R.add),
R.props(["upper", "lower"])
);
const getMidElement = R.converge(R.nth, [getMidIndex, R.prop("list")]);
const getSearchValue = R.prop("searchValue");
var binarySearch = R.compose(
R.prop("ans"),
R.until(
R.either(
R.compose(R.not, R.compose(R.isNil, R.prop("ans"))),
R.compose(R.apply(R.gt), R.props(["lower", "upper"]))
),
R.cond([
[
R.converge(R.equals, [getSearchValue, getMidElement]),
R.converge(R.assoc("ans"), [getMidIndex, R.identity]),
],
[
R.converge(R.lt, [getSearchValue, getMidElement]),
R.converge(R.assoc("upper"), [
R.compose(R.dec, getMidIndex),
R.identity,
]),
],
[
R.converge(R.gt, [getSearchValue, getMidElement]),
R.converge(R.assoc("lower"), [
R.compose(R.inc, getMidIndex),
R.identity,
]),
],
])
),
R.converge(R.assoc("upper"), [
R.compose(R.dec, R.length, R.prop("list")),
R.assoc("lower", 0),
]),
R.assoc("ans", null),
R.converge(R.mergeLeft, [
R.compose(R.objOf("list"), R.nthArg(0)),
R.compose(R.objOf("searchValue"), R.nthArg(1)),
])
);
var binarySearch = R.curryN(2, binarySearch);
module.exports = binarySearch;
the function takes two inputs a list and a value(v) -then-> the first three composing collecting args in an object like this
{
searchValue: 7,
list: [ 2, 3, 7, 12 ],
ans: null,
lower: 0,
upper: 3
}
after finishing I started to wonder from the performance point of view is this worse than having just one local midElement for example instead to keep calculating it
am I on the right track or just wasting time is this even readable
I'm a Javascript beginner (more or less).
I've created a new array:
var genres = [
"metal",
"rockroll",
"funk",
"punk",
"country",
];
However, I'd like to put each genre in the array a specific number of times, not just once. I know I can just repeat each line as many times as I need, but I'm sure there's a better way.
It would be great if I could do something like this:
var genres = [
"metal" * 3,
"rockroll" * 5,
"funk" * 1,
"punk" * 0,
"country" * 4,
];
...but of course I've tried that, and it doesn't work.
Can anyone help me out? I wasn't able to find anything by googling.
Thanks!
You can build an array like this with reduce() if you start with some data structure that holds your counts and categories:
let cats = [[3, "metal"], [5, "rockroll"], [1, "funk"], [0, "punk"], [4, "country"] ]
// etc..
let arr = cats.reduce((arr, [n, cat]) => arr.concat(Array(n).fill(cat)), [])
console.log(arr)
let item = [
{
genres: "metal",
count: 3
},
{
genres: "rockroll",
count: 5
},
{
genres: "funk",
count: 1
},
{
genres: "punk",
count: 0
}
];
console.log(item);
item.map(i => {
for(let n = 0; n < i.count; n++){
console.log(i.genres);
}
});
How do you think about using the Object?
There's no built-in way to do this, but you could easily write a function to do it. For instance:
function addMultiples (input) {
const output = []
for (let key in input) {
for (let i = 0; i < input[key]; i++) {
output.push(key)
}
}
return output
}
Then you would pass in your values as an object:
console.log(addMultiples({
"metal": 3,
"rockroll": 5,
"funk": 1,
"punk": 0,
"country": 4
}).join(", "))
// prints "metal, metal, metal, rockroll, rockroll, rockroll, rockroll, rockroll, funk, country, country, country, country"
You can also use Array.from and keep your sub arrays filled. And only spread them when needed:
let cats = [[3, "metal"], [5, "rockroll"], [1, "funk"], [0, "punk"], [4, "country"]]
const filled = Array.from(cats, ([v,k]) => new Array(v).fill(k)) // fill arrays
console.log(filled.reduce((r,c) => [...r, ...c])) // spread for output
I am trying to define a function, merge, which, when given two sorted arrays containing numbers, returns a sorted array of the numbers from both lists.
merge([ 4, 5, 6 ], [ 1, 2, 3, 4 ]) => [ 1, 2, 3, 4, 4, 5, 6 ]
merge([ 4 ], [ 2, 5, 8 ]) => [ 2, 4, 5, 8 ]
merge([ 1, 2, 6 ], []) => [ 1, 2, 6 ]
This is my code:
function merge(arr1, arr2) {
return arr1.concat(arr2).sort(arr1, arr2);
}
While the output is correct, I am told -- from my studies, and its automated tests -- that this code is not in good style. It writes:
Does not handles two arrays of same length.
Doesn't handle shorter first array.
Doesn't handle shorter second array.
What is a better way I can write this code? What should I do better?
Your code looks ok, however the way you're using sort is incorrect.
One way to use sort is to supply a function that compares two values in the array, and returns a number (positive or negative) to dictate the sorting of those values. For more information on sort, see this article
Consider the following changes to your merge method:
function merge(arr1, arr2) {
return arr1.concat(arr2).sort(function(valueA, valueB) { return valueA - valueB; );
}
Other answers already give you the literal answer of how to make your code correct. However, it is possibly missing the point. The function that you described is used in building a "merge sort", a very important sorting algorithm, whose major advantage is that it only needs to read the input lists once, sequentially, resulting in complexity of O(N) per pass; this allows it to even sort things that can't fit into the memory. Your code does not do that - it relies on sort, which is O(N log(N)) each time you invoke your function, and it doesn't utilise the fact that both its inputs are already pre-sorted (which is a key requirement for merge sort).
The merge sort algorithm will take the first element from both lists, then append the smaller one. Then it compares the next element from that list with the other list's first element, and again takes the smaller one. Repeat until one list is exhausted, then append the rest of the surviving list. (You can find a more exhaustive explanation of the merge sort on the Wikipedia page).
Amadan's answer paid attention to the problem constraints and pointed out that this can be written in O(n) time. Essentially, when inputs are both sorted, the algorithm is the "merge" step in a merge sort. This is done in linear time and works in a very simple and pleasing manner: look at the first item of each list and take the smaller of the two until one or both lists are exhausted. Then, tack any remaining elements onto the result array and return it.
The other answers are fine and in good JS style, but are in O(n log n) time and ignore entirely that the arrays are pre-sorted without mention, which is almost certainly not the answer someone asking for this routine would be looking for.
Here's a merge step:
const merge = (a, b) => {
const result = [];
while (a.length && b.length) {
result.push(a[0] < b[0] ? a.shift() : b.shift());
}
return result.concat(a).concat(b);
};
console.log(merge([ 4, 5, 6 ], [ 1, 2, 3, 4 ])); // => [ 1, 2, 3, 4, 4, 5, 6 ]
console.log(merge([ 4 ], [ 2, 5, 8 ])); // => [ 2, 4, 5, 8 ]
console.log(merge([ 1, 2, 6 ], [])); // => [ 1, 2, 6 ]
This can also be done with indexes which preserves the original arrays and is faster, but a little uglier-looking.
Here's a benchmark at JSPerf.
Just concat the arrays and return after applying a simple sort function:
console.log(merge([4, 5, 6], [1, 2, 3, 4])); //[ 1, 2, 3, 4, 4, 5, 6 ]
console.log(merge([4], [2, 5, 8])); //[ 2, 4, 5, 8 ]
console.log(merge([1, 2, 6], [])); //[ 1, 2, 6]
function merge(a, b) {
return a.concat(b).sort((a, b) => a - b);
}
I'm taking a coding challenge for linked lists in javascript. I haven't written a solution because I'm still trying to understand the basics.
Here's one of about 20 tests:
Input:
l: [3, 1, 2, 3, 4, 5]
k: 3
Expected Output:
[1, 2, 4, 5]
As a sort of hack to make sure I'm doing this right, I tried running the tests with this
function removeKFromList(l, k) {
return { value: 1, next: { value: 2, next: { value: 4, next: { value: 5, next: null}}}};
}
Returning a linked list... but it doesn't pass. Then I simply returned the array
function removeKFromList(l, k) {
return [1,2,4,5]
}
and it passed the first test.
Here's the question:
"Given a singly linked list of integers l and an integer k, remove all elements from list l that have a value equal to k."
My question is: Does l = [3, 1, 2, 3, 4, 5] count as a "linked list of integers"?
No, it doesn't, it's an array. However, it seems they're just giving the seed data, to leave the implementation (if you want) of the specifics of the linked list (along with seeding it) up to you completely. With that, means they need a uniform return which would be a 'seed' back to them (an array in order)