Benchmarks

JuliaActuary is an ecosystem of packages that makes Julia the easiest language to get started for actuarial workflows.

The Life Modeling Problem

Inspired by the discussion in the ActuarialOpenSource GitHub community discussion, folks started submitted solutions to what someone referred to as the “Life Modeling Problem”. This user submitted a short snippet for consideration of a representative problem.

Benchmarks

After the original user submitted a proposal, others chimed in and submitted versions in their favorite languages. I have collected those versions, and run them on a consistent set of hardware.

Some submissions were excluded because from the benchmarks they involved an entirely different approach, such as memoizing the function calls[^1].

[ Info: Precompiling CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b]
Table 1: Benchmarks for the Life Modeling Problem in nanoseconds (lower times are better).
18×6 DataFrame
Row lang algorithm function_name median mean relative_mean
String15 String15 String15 Float64? Float64 Float64
1 Julia Accumulator npv9 6.388 6.375 1.0
2 Rust Accumulator npv3 7.0 7.0 1.09804
3 Julia Accumulator npv8 7.372 7.375 1.15686
4 Julia Accumulator npv7 7.92 7.917 1.24188
5 Julia Accumulator npv6 9.037 9.009 1.41318
6 Julia Accumulator npv4 10.764 10.761 1.688
7 Julia Accumulator npv5 11.49 11.469 1.79906
8 Rust Accumulator npv2 14.0 14.0 2.19608
9 Julia Accumulator npv3 14.507 14.487 2.27247
10 Rust Accumulator npv1 22.0 22.0 3.45098
11 Julia Vectorized npv2 235.758 218.391 34.2574
12 Julia Vectorized npv1 235.322 228.198 35.7958
13 Python (Numba) Accumulator npv_numba missing 626.0 98.1961
14 Python Accumulator npv_loop missing 2314.0 362.98
15 Python (NumPy) Vectorized npv missing 14261.0 2237.02
16 R Vectorized npv base 4264.0 46617.0 7312.47
17 R Accumulator npv_loop 4346.0 62275.7 9768.74
18 R (data.table) Vectorized npv 770554.0 8.42767e5 1.32199e5

To aid in visualizing results with such vast different orders of magnitude, this graph includes a physical length comparison to serve as a reference. The computation time is represented by the distance that light travels in the time for the computation to complete (comparing a nanosecond to one foot length goes at least back to Admiral Grace Hopper).

[ Info: Precompiling Plots [91a5bcdd-55d7-5caf-9e0b-520d859cae80]
[ Info: Precompiling FileIOExt [f5f51d8f-5827-5d2e-939b-192fcd6ec70c]
Precompiling UnitfulExt
  ✓ Plots → UnitfulExt
  1 dependency successfully precompiled in 27 seconds. 160 already precompiled.
[ Info: Precompiling UnitfulExt [0e51ec96-f580-5f12-a625-1297083d7970]
┌ Warning: Module Plots with build ID fafbfcfd-b4da-2e81-0002-191685c7cfe2 is missing from the cache.
│ This may mean Plots [91a5bcdd-55d7-5caf-9e0b-520d859cae80] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1948
[ Info: Skipping precompilation since __precompile__(false). Importing UnitfulExt [0e51ec96-f580-5f12-a625-1297083d7970].
[ Info: Precompiling GeometryBasicsExt [b238bd29-021f-5edc-8b0e-16b9cda5f63a]
┌ Warning: Module Plots with build ID fafbfcfd-b4da-2e81-0002-191685c7cfe2 is missing from the cache.
│ This may mean Plots [91a5bcdd-55d7-5caf-9e0b-520d859cae80] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1948
[ Info: Skipping precompilation since __precompile__(false). Importing GeometryBasicsExt [b238bd29-021f-5edc-8b0e-16b9cda5f63a].
[ Info: Precompiling IJuliaExt [2f4121a4-3b3a-5ce6-9c5e-1f2673ce168a]
┌ Warning: Module Plots with build ID fafbfcfd-b4da-2e81-0002-191685c7cfe2 is missing from the cache.
│ This may mean Plots [91a5bcdd-55d7-5caf-9e0b-520d859cae80] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1948
[ Info: Skipping precompilation since __precompile__(false). Importing IJuliaExt [2f4121a4-3b3a-5ce6-9c5e-1f2673ce168a].

Discussion

For more a more in-depth discussion of these results, see this post.

All of the benchmarked code can be found in the JuliaActuary Learn repository. Please file an issue or submit a PR request there for issues/suggestions.

IRRs

Task: determine the IRR for a series of cashflows 701 elements long (e.g. monthly cashflows for 60 years).

Benchmarks

Times are in nanoseconds:
┌──────────┬──────────────────┬───────────────────┬─────────┬─────────────┬───────────────┐
│ Language │          Package │          Function │  Median │        Mean │ Relative Mean │
├──────────┼──────────────────┼───────────────────┼─────────┼─────────────┼───────────────┤
│   Python │  numpy_financial │               irr │ missing │   519306422 │       123146x │
│   Python │           better │ irr_binary_search │ missing │     3045229 │          722x │
│   Python │           better │        irr_newton │ missing │      382166 │           91x │
│    Julia │ ActuaryUtilities │               irr │    4185 │        4217 │            1x │
└──────────┴──────────────────┴───────────────────┴─────────┴─────────────┴───────────────┘

Discussion

The ActuaryUtilities implementation is over 100,000 times faster than numpy_financial, and 91 to 722 times faster than the better Python package. The ActuaryUtilities.jl implementation is also more flexible, as it can be given an argument with timepoints, similar to Excel’s XIRR.

Excel was used to attempt a benchmark, but the IRR formula returned a #DIV/0! error.

All of the benchmarked code can be found in the JuliaActuary Learn repository. Please file an issue or submit a PR request there for issues/suggestions.

Black-Scholes-Merton European Option Pricing

Task: calculate the price of a vanilla european call option using the Black-Scholes-Merton formula.

\[\begin{align} C(S_t, t) &= N(d_1)S_t - N(d_2)Ke^{-r(T - t)} \\ d_1 &= \frac{1}{\sigma\sqrt{T - t}}\left[\ln\left(\frac{S_t}{K}\right) + \left(r + \frac{\sigma^2}{2}\right)(T - t)\right] \\ d_2 &= d_1 - \sigma\sqrt{T - t} \end{align}\]

Benchmarks

Times are in nanoseconds:
┌──────────┬─────────┬─────────────┬───────────────┐
│ Language │  Median │        Mean │ Relative Mean │
├──────────┼─────────┼─────────────┼───────────────┤
│   Python │ missing │    817000.0 │       19926.0 │
│        R │  3649.0 │      3855.2 │          92.7 │
│    Julia │    41.0 │        41.6 │           1.0 │
└──────────┴─────────┴─────────────┴───────────────┘

Discussion

Julia is nearly 20,000 times faster than Python, and two orders of magnitude faster than R.

Other benchmarks

These benchmarks have been performed by others, but provide relevant information for actuarial-related work:

Colophone

Code

All of the benchmarked code can be found in the JuliaActuary Learn repository. Please file an issue or submit a PR request there for issues/suggestions.

Footnotes

[^1] If benchmarking memoization, it’s essentially benchmarking how long it takes to perform hashing in a language. While interesting, especially in the context of incremental computing, it’s not the core issue at hand. Incremental computing libraries exist for all of the modern languages discussed here.

[^2] Note that not all languages have both a mean and median result in their benchmarking libraries. Mean is a better representation for a garbage-collected modern language, because sometimes the computation just takes longer than the median result. Where the mean is not available in the graph below, median is substituted.