Skip to content

Add linear mutable array benchmarks #331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 24, 2021
Merged

Add linear mutable array benchmarks #331

merged 1 commit into from
Jun 24, 2021

Conversation

utdemir
Copy link
Contributor

@utdemir utdemir commented Jun 17, 2021

Slightly more details on issue #328.

I only spent time on reads benchmark, which creates an array
and iterates on it. On a recent GHC HEAD (378c0bba7d), here're
the current results:

$ cabal bench --benchmark-options 'arrays/reads'
...
benchmarked arrays/reads/Data.Array.Mutable.Linear
time                 191.7 ms   (185.3 ms .. 195.9 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 213.9 ms   (206.2 ms .. 226.2 ms)
std dev              16.61 ms   (9.796 ms .. 25.32 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarked arrays/reads/Data.Vector
time                 58.77 ms   (58.64 ms .. 58.88 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 67.59 ms   (64.36 ms .. 75.56 ms)
std dev              8.991 ms   (3.925 ms .. 14.61 ms)
variance introduced by outliers: 48% (moderately inflated)

The benchmark methodology is not ideal, since all benchmarks include
creation of the underlying array/vector. The reason is that the gauge
library we are using for benchmarks is not implemented with linear
arrows so we can not use its API easily to pass a linear value to a
benchmark. There might be a way to work around this implementation using
gauge (or another benchmarking library), but I didn't spend too much
time on it.

Slightly more details on issue #328.

I only spent time on `reads` benchmark, which creates an array
and iterates on it. On a recent GHC HEAD (`378c0bba7d`), here're
the current results:

```
$ cabal bench --benchmark-options 'arrays/reads'
...
benchmarked arrays/reads/Data.Array.Mutable.Linear
time                 191.7 ms   (185.3 ms .. 195.9 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 213.9 ms   (206.2 ms .. 226.2 ms)
std dev              16.61 ms   (9.796 ms .. 25.32 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarked arrays/reads/Data.Vector
time                 58.77 ms   (58.64 ms .. 58.88 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 67.59 ms   (64.36 ms .. 75.56 ms)
std dev              8.991 ms   (3.925 ms .. 14.61 ms)
variance introduced by outliers: 48% (moderately inflated)
```

The benchmark methodology is not ideal, since all benchmarks include
creation of the underlying array/vector. The reason is that the `gauge`
library we are using for benchmarks is not implemented with linear
arrows so we can not use its API easily to pass a linear value to a
benchmark. There might be a way to work around this implementation using
`gauge` (or another benchmarking library), but I didn't spend too much
time on it.
@utdemir utdemir requested a review from aspiwack June 17, 2021 22:25
@utdemir utdemir merged commit eb18d8e into master Jun 24, 2021
@utdemir utdemir deleted the ud/array-bench branch June 24, 2021 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants