Creating Lazy stream in CSharp

Hand-Toss a Stream
(Lazy List)
in C#
Dhaval Dalal
https://dhavaldalal.wordpress.com
@softwareartisan

Implementing Lazy List
IEnumerable<int> Naturals(int @from) { 
for (var i = @from; true ; i++) { 
yield return i; 
} 
} 
Naturals(0)
.Take(3)
.ForEach(Console.WriteLine); // 0 1 2
Using generators
This is the idiomatic approach.
There is yet another one as well.

Using Lambda
Wrap the tail of the
list in a closure.
1 ?

Using Lambda
list in a closure.
When we need the result
evaluate the tail by
invoking the closure.
Evaluate
tail
1 ?
1 *
2 ?

Using Lambda
list in a closure.
Additionally for
performance - Cache or
Memoize the closure
output.
Evaluate
tail
1 ?
1 *
2 ?

Using Lambda
list in a closure.
Additionally for
performance - Cache or
Memoize the closure
output.
Evaluate
tail
1 ?
1 *
2 ?
Stream implementation shown in these slides is available on:
https://github.com/CodeJugalbandi/FunctionalProgramming/blob/master/melodies/lazy_sequences/Stream.cs

Immutable Stream with
eager Head & lazy Tail
sealed class Stream<T> { 
private readonly T head; 
private readonly Func<Stream<T>> tail; 
 
public Stream(T head, Func<Stream<T>> tail) { 
this.head = head; 
this.tail = tail; 
} 
public T Head { 
get => head; 
} 
public Stream<T> Tail { 
get => tail(); // Need Memoization (for Perf) 
} 
public override string ToString() =>
$"Stream<{typeof(T)}>({head}, ?)"; 
}

private readonly Lazy<Stream<T>> tail; 
 
public Stream(T head, Lazy<Stream<T>> tail) { 
this.head = head; 
this.tail = tail; 
} 
public T Head { 
get => head; 
} 
public Stream<T> Tail { 
get => tail.Value; // Cached after first eval 
} 
public override string ToString() =>
$"Stream<{typeof(T)}>({head}, ?)"; 
}

var empty = new Stream<int>(0, null); 
Console.WriteLine(empty); // Stream<System.Int32>(0, ?) 
Console.WriteLine(empty.Head); // 0 
Console.WriteLine(empty.Tail); 
 
var singleton = new Stream<int>(1, new Lazy<Stream<int>>(() => empty)); 
Console.WriteLine(singleton); // Stream<System.Int32>(1, ?) 
Console.WriteLine(singleton.Head); // 1 
Console.WriteLine(singleton.Tail); // Stream<System.Int32>(0, ?) 
 
var couple = new Stream<int>(2, new Lazy<Stream<int>>(() => singleton)); 
Console.WriteLine(couple); // Stream<System.Int32>(2, ?) 
Console.WriteLine(couple.Head); // 2 
Console.WriteLine(couple.Tail); // Stream<System.Int32>(1, ?) 
Console.WriteLine(couple.Tail.Tail); // Stream<System.Int32>(0, ?) 
Console.WriteLine(couple.Tail.Tail.Tail);
As Streams are immutable
we can structurally share
the earlier stream.
Boom!
Boom!

Introduce Empty Stream
public static readonly Stream<T> Empty
= new Stream<T>(default(T), null); 
private readonly Lazy<Stream<T>> tail; 
public Stream(T head, Lazy<Stream<T>> tail) { … } 
public T Head { … } 
public Stream<T> Tail { … } 
public bool IsEmpty { 
get => tail == null; 
} 
public override string ToString() { 
if (IsEmpty)  
return "Empty"; 
 
return $"Stream<{typeof(T)}>({head}, ?)"; 
} 
}

Introduce Empty Stream
var empty = Stream<int>.Empty; 
Console.WriteLine(empty); // Empty 
Console.WriteLine(empty.IsEmpty); // True 
 
var singleton =
new Stream<int>(1, new Lazy<Stream<int>>(() => empty)); 
Console.WriteLine(singleton); // Stream(1, ?) 
Console.WriteLine(singleton.IsEmpty); // False 
 
var couple =
new Stream<int>(2, new Lazy<Stream<int>>(() => singleton)); 
Console.WriteLine(couple); // Stream(2, ?) 
Console.WriteLine(couple.IsEmpty); // False

Doing Side-effects
sealed class Stream<T> {
…
… 
public void ForEach(Action<T> action) { 
if (IsEmpty) 
return;
 
action(Head); 
Tail.ForEach(action); 
} 
}
var empty = Stream<int>.Empty; 
empty.ForEach(Console.WriteLine); // Prints Nothing 
var stream = new Stream<int>(2, new Lazy<Stream<int>>(new
Stream<int>(1, new Lazy<Stream<int>>(() => empty)))); 
stream.ForEach(Console.WriteLine); // 2 1

Consing to Stream
…
… 
// Cons using + 
public static Stream<T> operator + (Stream<T> s, T element) =>  
new Stream<T>(element, new Lazy<Stream<T>>(() => s)); 
}
var stream = Stream<int>.Empty + 1 + 2; 
Console.WriteLine(stream); // Stream(2, ?) 
Console.WriteLine(stream.Head); // 2 
Console.WriteLine(stream.Tail); // Stream(1, ?)
 
stream.ForEach(Console.WriteLine); // 2 1
Prepend (Cons)

Append to Stream
… 
// Append using + 
public static Stream<T> operator + (T element, Stream<T> s) =>  
new Stream<T>(element,
new Lazy<Stream<T>>(() => s.IsEmpty ? Stream<T>.Empty : s)); 
}
var stream = 1 + Stream<int>.Empty; 
stream.ForEach(Console.WriteLine); // 1 
 
var stream = 1 + (2 + (3 + (4 + Stream<int>.Empty))); 
stream.ForEach(Console.WriteLine); // 1 2 3 4

…
public static Stream<R> Of<R>(params R[] rs) { 
var stream = Stream<R>.Empty; 
var indices = rs.Length - 1; 
for (var i = indices; i >= 0; i--) { 
stream = stream + rs[i]; 
} 
return stream; 
} 
}
Stream<int>.Of<int>()
.ForEach(Console.WriteLine); // Prints Nothing
Stream<int>.Of(1, 2, 3, 4)
.ForEach(Console.WriteLine); // 1 2 3 4
Construct a Stream
from few elements

Concat another Stream
… 
public static Stream<T> operator + (Stream<T> @this,
Stream<T> other) {
if (@this.IsEmpty) 
return other; 
 
return new Stream<T>(@this.Head,
new Lazy<Stream<T>>(() => @this.Tail + other)); 
}
}
var concat1 = Stream<char>.Empty + Stream<char>.Of('a', 'b'); 
concat1.ForEach(Console.Write); // ab 
 
var concat2 = Stream<char>.Of('a', 'b') + Stream<char>.Empty; 
concat2.ForEach(Console.Write); // ab 
 
var concat3 = Stream<char>.Of('a', 'b') + Stream<char>.Of('c', 'd', 'e'); 
concat3.ForEach(Console.Write); // abcde

…
public Stream<T> Take(int howMany) { 
if (IsEmpty || howMany <= 0) 
return Stream<T>.Empty; 
 
return new Stream<T>(Head,
new Lazy<Stream<T>>(() => Tail.Take(howMany - 1))); 
} 
}
Stream<int>.Empty
.Take(2).ForEach(Console.WriteLine); // Prints Nothing 
.Take(2).ForEach(Console.WriteLine); // 1 2 
.Take(12).ForEach(Console.WriteLine); // 1 2 3 4 
.Take(0).ForEach(Console.WriteLine); // Prints Nothing
Take few elements

…
public Stream<T> Drop(int howMany) { 
if (IsEmpty || howMany <= 0) 
return this; 
 
return Tail.Drop(howMany - 1); 
} 
}
Stream<int>.Empty
.Drop(2).ForEach(Console.WriteLine); // Prints Nothing 
.Drop(2).ForEach(Console.WriteLine); // 3 4 
.Drop(20).ForEach(Console.WriteLine); // Prints Nothing 
.Drop(0).ForEach(Console.WriteLine); // 1 2 3 4
Drop few elements

…
…
public static Stream<R> Generate<R>(Func<R> fn) =>  
new Stream<R>(fn(), new Lazy<Stream<R>>(() => Generate(fn)));
}
var random = new Random(); 
Stream<int>.Generate(() => random.Next(100, 150))
.Take(4)
.ForEach(Console.WriteLine);
// Prints 4 random numbers bet [100, 150)
Construct a Stream
using lambda - 1

…
…
public static Stream<R> Iterate<R>(R initial, Func<R, R> fn) =>  
new Stream<R>(initial,
new Lazy<Stream<R>>(() => Iterate(fn(initial), fn))); 
}
Stream<int>.Iterate(9, x => x + 2)
.Take(4)
.ForEach(Console.WriteLine); // 9 11 13 15
Construct a Stream
using lambda - 2

…
…
public void Deconstruct(out T first, out Stream<T> rest) { 
if (IsEmpty)  
throw new ArgumentException("Collection is Empty!"); 
 
first = Head; 
rest = Drop(1); 
} 
}
var (head, rest) = Stream<int>.Of(1, 2, 3, 4); 
Console.WriteLine("Head = " + head); // 1 
Console.WriteLine("Rest = " + rest); // Stream(2, ?)
Deconstruct a Stream

…
public void Deconstruct(out T first, out Stream<T> rest) { 
if (IsEmpty)  
throw new ArgumentException("Collection is Empty!"); 
 
first = Head; 
rest = Drop(1); 
} 
public void Deconstruct(out T first, out T second,
out Stream<T> rest) =>
(first, (second, rest)) = this;  
}
var (head, second, rest) = Stream<int>.Of(1, 2, 3, 4);
Console.WriteLine("Head = " + head); // 1
Console.WriteLine("Second = " + second); // 2

…
public void Deconstruct(out T first, out T second,
out Stream<T> rest) => 
(first, (second, rest)) = this; 
 
public void Deconstruct(out T first, out T second, out T third,
out Stream<T> rest) => 
(first, second, (third, rest)) = this; 
}
var (head, second, third, rest) = Stream<int>.Of(1, 2, 3, 4); 
Console.WriteLine("Head = " + head); // 1 
Console.WriteLine("Second = " + second); // 2 
Console.WriteLine("Third = " + third); // 3

Transform each element
…
… 
public Stream<R> Select<R>(Func<T, R> fn) { 
if (IsEmpty) 
return Stream<R>.Empty; 
 
return new Stream<R>(fn(Head),
new Lazy<Stream<R>>(() => Tail.Select(fn))); 
} 
}
var empty = Stream<int>.Empty;
Console.WriteLine(empty.Select(x => x * x)); // Prints Nothing
var stream = Stream<int>.Of(1, 2); 
Console.WriteLine(stream.Select(x => x * x)); // 1 4

Filtering the Stream
… 
public Stream<T> Where(Predicate<T> pred) { 
if (IsEmpty) 
 
if (pred(Head))  
return new Stream<T>(Head,
new Lazy<Stream<T>>(() => Tail.Where(pred))); 
 
return Tail.Where(pred); 
} 
}
var empty = Stream<int>.Empty;
Console.WriteLine(empty.Where(x => x < 2)); // Prints Nothing
var stream = Stream<int>.Of(1, 2); 
Console.WriteLine(stream.Where(x => x < 2)); // 1

Flatmap the Stream
… 
public Stream<T> SelectMany(Func<T, Stream<R>> fn) { 
if (IsEmpty) 
 
return fn(Head) + Tail.SelectMany(fn); 
} 
}
Stream<char>.Of('a', 'b') 
.SelectMany(c => Stream<int>.Of(1, 2).Select(n => (c, n))) 
.ForEach(t => Console.Write(t)); // (a, 1)(a, 2)(b, 1)(b, 2) 
 
Stream<int>.Empty.SelectMany(c => Stream<int>.Of(1, 2).Select(n => (c, n))) 
.ForEach(t => Console.Write(t)); // Prints Nothing

Reverse
…
… 
public Stream<T> Reverse() { 
Stream<T> Reverse0(Stream<T> acc, Stream<T> source) { 
if (source.IsEmpty) 
return acc; 
 
return Reverse0(acc + source.Head, source.Tail); 
} 
return Reverse0(Stream<T>.Empty, this); 
} 
}
Stream<char>.Of('a', 'b', ‘c')
.Reverse().ForEach(Console.Write); // cba
 
Stream<int>.Empty
.Reverse().ForEach(Console.WriteLine); // Prints Nothing

Take while
predicate holds
… 
public Stream<T> TakeWhile(Predicate<T> pred) { 
if (IsEmpty) 
 
if (pred(Head)) 
return Head + Tail.TakeWhile(pred); 
 
} 
}
Stream<char>.Of('a', 'a', 'b', 'c').TakeWhile(c => c == 'a') 
.ForEach(Console.Write); // aa 
 
Stream<char>.Of('a', 'a', 'b', 'c').TakeWhile(c => c == 'b') 
.ForEach(Console.Write); // Prints Nothing

… 
public Stream<T> DropWhile(Predicate<T> pred) { 
if (IsEmpty) 
 
if (pred(Head)) 
return Tail.DropWhile(pred); 
 
return this; 
} 
}
Stream<char>.Of('a', 'a', 'b', 'c').DropWhile(c => c == 'a') 
.ForEach(Console.Write); // bc 
 
Stream<char>.Of('a', 'a', 'b', 'c').DropWhile(c => c == 'b') 
.ForEach(Console.Write); // aabc
Drop while
predicate holds

… 
public U Aggregate(U identity, Func<U, T, U> func) { 
if (IsEmpty) 
return identity; 
 
return Tail.Aggregate(func(identity, Head), func); 
} 
}
var sum1 = Stream<int>.Of(1, 2, 3, 4)
.Aggregate(0, (acc, elem) => acc + elem); 
Console.WriteLine($"sum = {sum1}"); // 10 
var sum2 = Stream<int>.Of<int>()
.Aggregate(0, (acc, elem) => acc + elem);  
Console.WriteLine($"sum = {sum2}"); // 0
Aggregate or Reduce

… 
public bool All(Predicate<T> pred) { 
bool All0(bool accumulator, Stream<T> stream) { 
if (stream.IsEmpty || accumulator == false) 
return accumulator; 
 
return All0(accumulator && pred(stream.Head), stream.Tail); 
} 
return All0(true, this); 
} 
}
Console.WriteLine(Stream<int>.Of<int>().All(x => x % 2 == 0));
// True 
Console.WriteLine(Stream<int>.Of<int>(2, 4).All(x => x % 2 == 0));
// True 
Console.WriteLine(Stream<int>.Of<int>(1, 2, 4).All(x => x % 2 == 0));
// False
All Don’t evaluate the entire
Stream, short-circuit if we
already determined
negation.

… 
public bool Any(Predicate<T> pred) { 
bool Any0(bool accumulator, Stream<T> stream) { 
if (stream.IsEmpty || accumulator == true) 
return accumulator; 
 
return Any0(accumulator || pred(stream.Head), stream.Tail); 
} 
return Any0(false, this); 
} 
}
Console.WriteLine(Stream<int>.Of<int>().Any(x => x % 2 == 0));
// False 
Console.WriteLine(Stream<int>.Of<int>(2, 4).Any(x => x % 2 == 0));
// True 
Console.WriteLine(Stream<int>.Of<int>(1, 2, 4).Any(x => x % 2 == 0));
// True 
Console.WriteLine(Stream<int>.Of<int>(1, 3).Any(x => x % 2 == 0));
// False
Any Don’t evaluate the entire
Stream, short-circuit if we
already determined
afﬁrmation.

… 
public Stream<T> Scan(U identity, Func<U, T, U> func) { 
if (IsEmpty) 
 
U newHead = func(identity, Head); 
return newHead + Tail.Scan(newHead, func); 
} 
}
// Prints running sum 
.Scan(0, (acc, elem) => acc + elem)
.ForEach(Console.WriteLine); // 1 3 6 10 
Stream<int>.Of<int>()
.Scan(0, (acc, elem) => acc + elem)
Scan

Zip two Streams
… 
public Stream<(T,U)> Zip(Stream that) { 
if (this.IsEmpty || that.IsEmpty) 
return Stream<(T,U)>.Empty; 
 
return (this.Head, that.Head) + this.Tail.Zip(that.Tail); 
} 
}
Stream<char>.Of('a', 'b').Zip(Stream<int>.Of(1, 2)) 
.ForEach(t => Console.Write(t)); // (a, 1)(b, 2) 
 
Stream<char>.Of('a', 'b').Zip(Stream<int>.Empty) 
.ForEach(t => Console.Write(t)); // Prints Nothing 
 
Stream<int>.Empty.Zip(Stream<char>.Of('a', 'b')) 
.ForEach(t => Console.Write(t)); // Prints Nothing

Zip with function
… 
public Stream<R> ZipWith<U, R>(Stream that, Func<T, U, R> fn) { 
if (this.IsEmpty || that.IsEmpty) 
 
return fn(this.Head, that.Head) +
this.Tail.ZipWith(that.Tail, fn); 
} 
}
var numbers = Stream<int>.Of(1, 2, 3); 
numbers.ZipWith(numbers, (n1, n2) => n1 * n2) 
.ForEach(Console.WriteLine); // 1 4 9 
 
numbers.ZipWith(Stream<int>.Empty, (n1, n2) => n1 * n2) 
.ForEach(Console.WriteLine); // Prints Nothing 
 
Stream<int>.Empty.ZipWith(numbers, (n1, n2) => n1 * n2)

Splitsealed class Stream<T> { 
… 
public (Stream<T>, Stream<T>) Split(Predicate<T> pred) { 
(Stream<T>, Stream<T>) Split0(Stream<T> yesAcc,
Stream<T> noAcc,
Stream<T> source) { 
if (source.IsEmpty)  
return (yesAcc.Reverse(), noAcc.Reverse()); 
 
var elem = source.Head; 
if (pred(elem)) 
return Split0(yesAcc + elem, noAcc, source.Tail); 
else 
return Split0(yesAcc, noAcc + elem, source.Tail); 
}  
return Split0(Stream<T>.Empty, Stream<T>.Empty, this); 
} 
}
var (evens, odds) = Stream<int>.Iterate(0, x => x + 1).Take(10)
.Split(x => x % 2 == 0); 
evens.ForEach(Console.Write); // 02468 
odds.ForEach(Console.Write); // 13579

To List
…
public List<T> ToList() { 
var result = new List<T>(); 
ForEach(result.Add); 
return result; 
} 
}
var list = Stream<int>.Iterate(1, x => x + 1)
.Take(4)
.ToList(); 
foreach (var item in list) { 
Console.WriteLine(item); 
}

Find first 6 primes using the Sieve of
Eratosthenes
Hint: Use Stream created earlier
http://world.mathigon.org/Prime_Numbers

Sieve
Stream<int> From(int start) =>
Stream<int>.Iterate(start, x => x + 1); 
Stream<int> Sieve(Stream<int> s) { 
var first = s.Head; 
var rest = s.Tail.Where(n => n % first != 0); 
return new Stream<int>(first,
new Lazy<Stream<int>>(() => rest)); 
} 
 
var primes = Sieve(From(2)).Take(6)
primes.ToList() // [2, 3, 5, 7, 11, 13]

First 10 Fibonacci Nos.
Write a function fibonacci which consumes an
integer and produces that many numbers in the
fibonacci series.
For Example: fibonacci(10) produces
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
Provide Solutions
Using Generator
Using IEnumerable
Using Stream that we developed

IEnumerable<int> Fibonacci(int howMany) {
var (first, second) = (0, 1); 
for (var i = 0; i < howMany; i++) { 
yield return first; 
(first, second) = (second, first + second); 
} 
} 
 
Fibonacci(10).ToList(); 
// [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
Using Generator

IEnumerable<int> Fibonacci(int howMany) {
return IEnumerableExtensions.Iterate<(int, int)>((0, 1), tuple => { 
var (first, second) = tuple; 
var next = first + second; 
return (second, next); 
}) 
.Select(tuple => { 
return first; 
}) 
.Take(howMany); 
}
Fibonacci(10).ToList();
// [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
Using IEnumerable
Definition as suggested by Christopher Grande

Using Stream
https://www.haskell.org/tutorial/functions.html
var seed = From(0).Take(2); 
// Start with seed elements 0 and 1 
Fibonacci(seed) 
.Take(10) 
.ForEach(Console.WriteLine);
// 0, 1, 1, 2, 3, 5, 8, 13, 21, 34
Stream<int> Fibonacci(Stream<int> s) { 
var next = s.Zip(s.Tail).Select(tuple => { 
return first + second; 
}); 
return new Stream<int>(s.Head,
new Lazy<Stream<int>>(() => Fibonacci(s + next.Head))); 
}

Prime Counts
Using Lazy Lists, write a
program that counts number of
primes up to a given number.
At every power of 10, it should
emit the count of primes
obtained thus far. The table
shows the output until 10
10
Implement this using:
Using Stream we developed earlier.
Using IEnumerable
Hint: Write an extension method Scan
on IEnumerable
i/p count
10 4
10
2 25
10
3 168
10
4 1,229
10
5 9,592
10
6 78,498
10
7 6,64,579
10
8 57,61,455
10
9 5,08,47,534
10
10 45,50,52,511

class Streams { 
public static Stream<int> Range(int start, int count) { 
if (count < 0) 
throw new ArgumentOutOfRangeException($"{count}"); 
 
return Stream<int>.Iterate(start, x => x + 1).Take(count); 
}  
} 
Streams.Range(1, 5).ForEach(Console.WriteLine);
// 1 2 3 4 5
Prime Counts using IEnumerable
First, write our own Range

Prime Counts Using Stream
Stream<(int, int)> PrimeCount(int howManyPowerOf10) { 
bool IsPrime(int x) => Streams.Range(2, x)
.Where(n => n < x)
.All(n => x % n != 0);
 
return Stream<int>.Iterate(2, x => x + 1) 
.TakeWhile(x => x <= Math.Pow(10, howManyPowerOf10)) 
.Select(x => (x, IsPrime(x))) 
.Scan((0, 0), (acc, tuple) => { 
var (x, isPrime) = tuple; 
var (_, count) = acc; 
return isPrime ? (x, count + 1): (x, count); 
}) 
.Where(tuple => { 
var (x, _) = tuple; 
return Streams.Range(1, howManyPowerOf10) 
.Any(n => Math.Pow(10, n) == x); 
}); 
}
PrimeCount(3).ForEach(t => Console.WriteLine(t));
(10, 4), (100, 25), (1000, 168)

First, write our own Scan
static class IEnumerableExtensions {
public static IEnumerable Scan<T, U>(this IEnumerable<T> @this,
U initial, Func<U, T, U> fn) { 
IEnumerable ScannedEnumerable() { 
var acc = seed; 
foreach (var item in @this) { 
acc = fn(acc, item); 
yield return acc; 
}  
if (@this == null) 
throw new ArgumentNullException("Require non-null list!"); 
if (fn == null) 
throw new ArgumentNullException("Require non-null function!”); 
return ScannedEnumerable(); 
}
}

IEnumerable<(int, int)> PrimeCount(int howManyPowerOf10) { 
bool IsPrime(int x) => Enumerable.Range(2, x)
.Where(n => n < x)
.All(n => x % n != 0);
 
return IEnumerableExtensions.Iterate(2, x => x + 1) 
.TakeWhile(x => x <= Math.Pow(10, howManyPowerOf10)) 
.Select(x => (x, IsPrime(x))) 
.Scan((0, 0), (acc, tuple) => { 
var (x, isPrime) = tuple; 
var (_, count) = acc; 
return isPrime ? (x, count + 1): (x, count); 
}) 
.Where(tuple => { 
var (x, _) = tuple; 
return Enumerable.Range(1, howManyPowerOf10) 
.Any(n => Math.Pow(10, n) == x); 
}); 
}
PrimeCount(3).ForEach(t => Console.WriteLine(t));
(10, 4), (100, 25), (1000, 168)

Drawbacks of this
Stream Implementation
As C# compiler does not support Tail-
Recursion, all the APIs that are
implemented recursively will cause a
stack blow-up at some point for large
values of stream.
Even-though it uses caching of tail
using Lazy<T>, this Stream is not
performant like IEnumerable! This is
because the recursive APIs don’t run in
constant stack space.

But still its a good
mental exercise to create
streams from first
principles!

Creating Lazy stream in CSharp

Recommended

More Related Content

What's hot (20)

Similar to Creating Lazy stream in CSharp (20)

More from Dhaval Dalal (20)

Recently uploaded (20)

Creating Lazy stream in CSharp