Finance Models Overview

FinanceModels.jl provides a set of composable contracts, models, and functions that allow for modeling of both simple and complex financial instruments. The resulting models, such as discount rates or term structures, can then be used across the JuliaActuary ecosystem to perform actuarial and financial analysis.

1. Cashflow - a fundamental financial type

Say you wanted to model a contract that paid quarterly payments, and those payments occurred starting 15 days from the valuation date (first payment time = 15/365 = 0.057)

Previously, you had two options:

1. Choose a discrete timestep to model (e.g. monthly, quarterly, annual) and then lump the cashflows into those timesteps. E.g. with monthly timesteps of a unit payment of our contract, it might look like: [1,0,0,1,0,0...]
2. Keep track of two vectors: one for the payment and one for the times. In this case, that might look like: cfs = [1,1,...];times = [0.057, 0.307...]

The former has inaccuracies due to the simplified timing and logical complication related to mapping the contracts natural periodicity into an arbitrary modeling choice. The latter becomes unwieldy and fails to take advantage of Julia’s type system.

The new solution: Cashflows. Our example above would become: [Cashflow(1,0.057), Cashflow(1,0.307),...]

2. Contracts - A composable way to represent financial instruments

Contracts are a composable way to represent financial instruments. They are, in essence, anything that is a collection of cashflows. Contracts can be combined to represent more complex instruments. For example, a bond can be represented as a collection of cashflows that correspond to the coupon payments and the principal repayment.

Examples:

• a Cashflow
• Bonds:
  • Bond.Fixed, Bond.Floating
• Options:
  • Option.EuroCall and Option.EuroPut
• Compositional contracts:
  • Forwardto represent an instrument that is relative to a forward point in time.
  • Composite to represent the combination of two other instruments.

In the future, this notion may be extended to liabilities (e.g. insurance policies in LifeContingencies.jl)

using FinanceModels,FinanceCore

# Transducers is used to provide a more powerful, composable way to construct collections than the basic iteration interface
using Transducers: __foldl__, @next, complete

"""
A bond which pays down its par (one unit) in equal payments. 
"""
struct PrincipalOnlyBond{F<:FinanceCore.Frequency} <: FinanceModels.Bond.AbstractBond
    frequency::F
    maturity::Float64
end

# We extend the interface to say what should happen as the bond is projected
# There's two parts to customize:
# 1. any initialization or state to keep track of
# 2. The loop where we decide what gets returned at each timestep
function Transducers.__foldl__(rf, val, p::Projection{C,M,K}) where {C<:PrincipalOnlyBond,M,K}
    # initialization stuff
    b = p.contract # the contract within a projection
    ts = Bond.coupon_times(b) # works since it's a FinanceModels.Bond.AbstractBond with a frequency and maturity
    pmt = 1 / length(ts)

    for t in ts
        # the loop which returns a value
        cf = Cashflow(pmt, t)
        val = @next(rf, val, cf) # the value to return is the last argument
    end
    return complete(rf, val)
end

julia> PrincipalOnlyBond(Periodic(2),5.) |> collect
10-element Vector{Cashflow{Float64, Float64}}:
 Cashflow{Float64, Float64}(0.1, 0.5)
 Cashflow{Float64, Float64}(0.1, 1.0)
 Cashflow{Float64, Float64}(0.1, 1.5)
 Cashflow{Float64, Float64}(0.1, 2.0)
 Cashflow{Float64, Float64}(0.1, 2.5)
 Cashflow{Float64, Float64}(0.1, 3.0)
 Cashflow{Float64, Float64}(0.1, 3.5)
 Cashflow{Float64, Float64}(0.1, 4.0)
 Cashflow{Float64, Float64}(0.1, 4.5)
 Cashflow{Float64, Float64}(0.1, 5.0)

3. Quotes - The observed price we need to fit a model to

Quotes are the observed prices that we need to fit a model to. They represent the market prices of financial instruments, such as bonds or swaps. In the context of the package, a quote is defined as a pair of a contract and a price.

For example, a par yield bond paying a 4% coupon (paid as 2% twice per annum) implies a price at par (i.e. 1.0):

julia> ParYield(Periodic(0.04,2),10)
Quote{Float64, FinanceModels.Bond.Fixed{Periodic, Float64, Int64}}(
1.0, 
FinanceModels.Bond.Fixed{Periodic, Float64, Int64}(0.040000000000000036, Periodic(2), 10))

A number of convenience functions are included to construct a Quote:

• ZCBPrice and ZCBYield
• ParYield
• CMTYield
• OISYield
• ForwardYields

4. Models - Not just yield curves anymore

• Yield Curves: all of Yields.jl yield models are included in the initial FinanceModels.jl release
• Equities and Options: The initial release includes BlackScholesMerton option pricing and one can use constant or spline volatility models
• Others more to come in the future

 using FinanceModels, FinanceCore
 using AccessibleOptimization 
 using IntervalSets
 
struct ABDiscountLine{A} <: FinanceModels.Yield.AbstractYieldModel
    a::A
    b::A
end

ABDiscountLine() = ABDiscountLine(0.,0.)

function FinanceCore.discount(m::ABDiscountLine,t)
    #discount rate is approximated by a straight line, floored at 0.0 and capped at 1.0
    clamp(m.a*t + m.b, 0.0,1.0) 
end


# `@optic` indicates what in our model variables needs to be updated (from AccessibleOptimization.jl)
# `-1.0 .. 1.0` says to bound the search from negative to positive one (from IntervalSets.jl)
FinanceModels.__default_optic(m::ABDiscountLine) = OptArgs([
    @optic(_.a) => -1.0 .. 1.0,
    @optic(_.b) => -1.0 .. 1.0,
]...)

quotes = ZCBPrice([0.9, 0.8, 0.7,0.6])

m = fit(ABDiscountLine(),quotes)

julia> map(q -> pv(m,q.instrument),quotes) 
4-element Vector{Float64}:
 0.9
 0.8
 0.7
 0.6

5. fit - The standardized API for all models, quotes, and methods

       Model                                                               Method
          |                                                                   |
    |------------|                                                     |---------------|
fit(Spline.Cubic(), CMTYield.([0.04,0.05,0.055,0.06,0055],[1,2,3,4,5]), Fit.Bootstrap())
                    |-------------------------------------------------|
                                              |
                                              Quotes

• Model could be Spline.Linear(), Yield.NelsonSiegelSvensson(), Equity.BlackScholesMerton(...), etc.
• Quote could be CMTYields, ParYields, Option.Eurocall, etc.
• Method could be Fit.Loss(x->x^2), Fit.Loss(x->abs(x)), Fit.Bootstrap(), etc.

The benefit of this versus the old Yields.jl API is:

• Without a generic fit method, no obvious way to expose different curve construction methods (e.g. choice of model and method)
• The fit is extensible. Users or other packages could define their own Models, Quotes, or Methods and integrate into the JuliaActuary ecosystem.
• The fit formulation is very generic: the required methods are minimal to integrate in order to extend the functionality.

function FinanceModels.fit(m::ABDiscountLine, quotes, ...)
   # custom code for fitting your model here
end

6. Projections

A Projection is a generic way to work with various data that you can project forward. For example, getting the series of cashflows associated with a contract.

What is a Projection?

struct Projection{C,M,K} <: AbstractProjection
    contract::C    # the contract (or set of contracts) we want to project
    model::M       # the model that defines how the contract will behave
    kind::K           # what kind of projection do we want? only cashflows? 
end

contract is obvious, so let’s talk more about the second two:

• model is the same kind of thing we discussed above. Some contracts (e.g. a floating rate bond). We can still decompose a floating rate bond into a set of cashflows, but we need a model.
  • There are also projections which don’t need a model (e.g. fixed bonds) and for that there’s the generic NullModel()
• kind defines what we’ll return from the projection.
  • CashflowProjection() says we just want a Cashflow[...] vector
  • … but if we wanted to extend this such that we got a vector containing cashflows, capital factors, default rates, etc we could define a new projection type (e.g. we might call the above AssetDetailProjection()
  • As of the time of announcement, only CashflowProjection() is defined by FinanceModels.jl

struct Floating{F<:FinanceCore.Frequency,N<:Real,M<:Timepoint,K} <: AbstractBond
    coupon_rate::N # coupon_rate / frequency is the actual payment amount
    frequency::F
    maturity::M
    key::K
end

@inline function Transducers.__foldl__(rf, val, p::Projection{C,M,K}) where {C<:Bond.Floating,M,K}
    b = p.contract
    ts = Bond.coupon_times(b)
    for t in ts
        freq = b.frequency # e.g. `Periodic(2)`
        freq_scalar = freq.frequency  # the 2 from `Periodic(2)`

        # get the rate from the current time to next payment 
        # out of the model and convert it to the contract's periodicity
        model = p.model[b.key]
        reference_rate = rate(freq(forward(model, t, t + 1 / freq_scalar)))
        coup = (reference_rate + b.coupon_rate) / freq_scalar
        amt = if t == last(ts)
            1.0 + coup
        else
            coup
        end
        cf = Cashflow(amt, t)
        val = @next(rf, val, cf)
    end
    return complete(rf, val)
end

7. ProjectionKinds

While CashflowProjection is the most common (and the only one built into the initial release of FinanceModels), a Projection can be created which handles different kinds of outputs in the same manner as projecting just basic cashflows. For example, you may want to output an amortization schedule, or a financial statement, or an account value roll-forward. The Projection is able to handle these custom outputs by dispatching on the third element in a Projection.

Let’s extend the example of a principle-only bond from section 2 above. Our goal is to create a basic amortization schedule which shows the payment made and outstanding balance.

First, we create a new subtype of ProjectionKind:

struct AmortizationSchedule <: FinanceModels.ProjectionKind
end

And then define the loop for the amortization schedule output:

# note the dispatch on `AmortizationSchedule` in the next line
function Transducers.__foldl__(rf, val, p::Projection{C,M,K}) where {C<:PrincipalOnlyBond,M,K<:AmortizationSchedule}
    # initialization stuff
    b = p.contract # the contract within a projection
    ts = Bond.coupon_times(b) # works since it's a FinanceModels.Bond.AbstractBond with a frequency and maturity
    pmt = 1 / length(ts)
    balance = 1.0
    for t in ts
        # the loop which returns a tuple of the relevant data
        balance -= pmt
        result = (time=t,payment=pmt,outstanding=balance)
        val = @next(rf, val, result) # the value to return is the last argument
    end
    return complete(rf, val)
end

We can now define the projection:

julia> p = Projection(
           PrincipalOnlyBond(Periodic(2),5.),  # our contract
           NullModel(),                       # the projection doesn't need a model, so use the null model
           AmortizationSchedule(),            # specify the amortization schedule output
           );

And then collect the values:

julia> collect(p)
10-element Vector{NamedTuple{(:time, :payment, :outstanding), Tuple{Float64, Float64, Float64}}}:
 (time = 0.5, payment = 0.1, outstanding = 0.9)
 (time = 1.0, payment = 0.1, outstanding = 0.8)
 (time = 1.5, payment = 0.1, outstnding = 0.7000000000000001)
 (time = 2.0, payment = 0.1, outstanding = 0.6000000000000001)
 (time = 2.5, payment = 0.1, outstanding = 0.5000000000000001)
 (time = 3.0, payment = 0.1, outstanding = 0.40000000000000013)
 (time = 3.5, payment = 0.1, outstanding = 0.30000000000000016)
 (time = 4.0, payment = 0.1, outstanding = 0.20000000000000015)
 (time = 4.5, payment = 0.1, outstanding = 0.10000000000000014)
 (time = 5.0, payment = 0.1, outstanding = 1.3877787807814457e-16)

Development Benefits

In addition to the more composable code for the end-user, the package itself has been able to be simplified. Compared to Yields.jl, the lines of source code have been reduced by 30% while the number of lines of documentation has increased by over 20%.

Migration Guide

For those looking to upgrade from Yields (v3.x.x) to FinanceModels (v4+), there is a migration guide here. Associated packages ActuaryUtilities.jl and FinanceCore.jl had major version releases for minor breaking changes where most code should remain unaffected (FinanceCore is not intended to be user-facing).

Some tutorials or examples on the site may still use Yields.jl - that’s okay as they will still work given Julia’s strong degree of reproducibility and dependency management tools. Please open an issue on the JuliaActuary.org repository if you have trouble with any of the old example code.

Conclusion

In this post we’ve now defined two assets that can work seamlessly with projecting cashflows, fitting models, and determining valuations :)

FinanceModel.jl should provide the basis for a performant and composable design to facilitate further development and use by actuaries and other financial professionals.

What is Modern Bayesian Statistics?

A Bayesian statistical model has four main components to focus on:

1. Prior encoding assumptions about the random variables related to the problem at hand, before conditioning on the data.
2. A Model that defines how the random variables give rise to the observed outcome.
3. Data that we use to update our prior assumptions.
4. Posterior distributions of our random variables, conditioned on the observed data and our model

Having defined the first two components and collected our data, the workflow involves computationally sampling the posterior distribution, often using a technique called Markov Chain Monte-Carlo (MCMC). The result is a series of values that are sampled statistically from the posterior distribution.

Advantages of the Bayesian Approach

The main advantages of this approach over traditional actuarial techniques are:

1. Focus on distributions rather than point estimates of the posterior’s mean or mode. We are often interested in the distribution of the parameters and a focus on a single parameter estimate will understate the risk distribution.
2. Model flexibility. A Bayesian model can be as simple as an ordinary linear regression, but as complex as modeling a full insurance mechanics.
3. Simpler mental model. Fundamentally, Bayes’ theorem could be distilled down to an approach where you count the ways that things could occur and update the probabilities accordingly.
4. Explicit Assumptions.: Enumerating the random variables in your model and explicitly parameterizing prior assumptions avoids ambiguity of the assumptions inside the statistical model.

Challenges with the Bayesian Approach

With the Bayesian approach, there are a handful of things that are challenging. Many of the listed items are not unique to the Bayesian approach, but there are different facets of the issues that arise.

1. Model Construction. One must be thoughtful about the model and how variables interact. However, with the flexibility of modeling, you can apply (actuarial) science to makes better models!
2. Model Diagnostics. Instead of R^2 values, there are unique diagnostics that one must monitor to ensure that the posterior sampling worked as intended.
3. Model Complexity and Size of Data. The sampling algorithms are computationally intensive - as the amount of data grows and model complexity grows, the runtime demands cluster computing.
4. Model Representation. The statistical derivation of the posterior can only reflect the complexity of the world as defined by your model. A Bayesian model won’t automatically infer all possible real-world relationships and constraints.

Why Now?

There are both philosophical and practical reasons why Bayesian analysis is rapidly changing the statistical landscape.

Philosophically, one of the main reasons why Bayesian thinking is appealing is its ability to provide a straightforward interpretation of statistical conclusions.

For example, when estimating an unknown quantity, a Bayesian probability interval can be directly understood as having a high probability of containing that quantity. In contrast, a frequentist confidence interval is typically interpreted only in the context of a series of similar inferences that could be made in repeated practice. In recent years, there has been a growing emphasis on interval estimation rather than hypothesis testing in applied statistics. This shift has strengthened the Bayesian perspective since it is likely that many users of standard confidence intervals intuitively interpret them in a manner consistent with Bayesian thinking.

Another meaningful way to understand the contrast between Bayesian and frequentist approaches is through the lens of decision theory, specifically how each view treats the concept of randomness. This perspective pertains to whether you regard the data being random or the parameters being random.

Frequentist statistics treats parameters as fixed and unknown, and the data as random — this is reflective of the view that data you collect is but one realization of an infinitely repeatable random process. Consequently, frequentist procedures, like hypothesis testing or confidence intervals, are generally based on the idea of long-run frequency or repeatable sampling.

Conversely, Bayesian statistics turns this on its head by treating the data as fixed — after all, once you’ve collected your data, it’s no longer random but a fixed observed quantity. Parameters, which are unknown, are treated as random variables. The Bayesian approach then allows us to use probability to quantify our uncertainty about these parameters.

The Bayesian approach tends to align more closely with our intuitive way of reasoning about problems. Often, you are given specific data and you want to understand what that particular set of data tells you about the world. You’re likely less interested in what might happen if you had infinite data, but rather in drawing the best conclusions you can from the data you do have.

Practically, recent advances in computational power, algorithm development, and open-source libraries have enabled practitioners to adapt the Bayesian workflow.

Deriving the posterior distribution is analytically intractable so computational methods must be used. Advances in raw computing power only in the 1990’s made non-trivial Bayesian analysis possible, and recent advances in algorithms have made the computations more efficient. For example, one of the most popular algorithms, NUTS, was only published in the 2010’s.

Many problems require the use of compute clusters to manage runtime, but if there is any place to invest in understanding posterior probability distributions, it’s insurance companies trying to manage risk!

Moreover, the availability of open-source libraries, such as Turing.jl, PyMC3, and Stan provide access to the core routines in an accessible interface.

Subjectivity of the Priors?

There are two ways one might react to subjectivity in a Bayesian context: It’s a feature that should be embraced or it’s a flaw that should be avoided.

Bayesian Versus Machine Learning

Machine learning (ML) is fully compatible with Bayesian analysis - one can derive posterior distributions for the ML parameters like any other statistical model and the combination of approaches may be fruitful in practice.

However, to the extent that actuaries have leaned on ML approaches due to the shortcomings of traditional actuarial approaches, Bayesian modeling may provide an attractive alternative without resorting to notoriously finicky and difficult-to-explain ML models. The Bayesian framework provides an explainable model and offers several analytic extensions beyond the scope of this introductory article:

• Causal Modeling: Identifying not just correlated relationships, but causal ones, in contexts where a traditional experiment is unavailable.
• Bayes Action: Optimizing a parameter for, e.g., a CTE95 level instead of a parameter mean.
• Information Criterion: Principled techniques to compare model fit and complexity.
• Missing data: Mechanisms to handle the different kinds of missing data.
• Model averaging: Posteriors can be combined from different models to synthesize different approaches.

Implications for Risk Management

Like Bayes’ Formula itself, another aspect of actuarial literature that is taught but often glossed over in practice is the difference between process risk (volatility), parameter risk, and model formulation risk. Often when performing analysis that relies on stochastic result, in practice only process/volatility risk is assessed.

Bayesian statistics provides the tools to help actuaries address parameter risk and model formulation. The posterior distribution of parameters derived is consistent with the observed data and modeled relationships. This posterior distribution of parameters can then be run as an additional dimension to the risk analysis.

Additionally, best practices include skepticism of the model construction itself, and testing different formulation of the modeled relationships and variable combinations to identify models which are best fit for purpose. Tools such as Information Criterion, posterior predictive checks, Bayes factors, and other statistical diagnostics can inform the actuary about tradeoffs between different choices of model.

Paving the Way Forward for Actuaries

Bayesian approaches to statistical problems are rapidly changing the professional statistical field. To the extent that the actuarial profession incorporates statistical procedures we should consider adopting the same practices. The benefits of this are a better understanding of the distribution of risks, results that are more interpretable and explainable, and techniques that can be applied to a wider range of problems. The combination of these things would serve to enhance actuarial best practices related to understanding and communicating about risk.

For actuaries interested in learning more, there are number of available resources to be found. Textbooks recommended by the author are:

• Statistical Rethinking (McElreath)
• Bayes Rules! (Johnson, Ott, Dogucu)
• Bayesian Data Analysis (Gelman, et. al.)

Additionally, the author has published a few examples of Bayesian analysis in an actuarial context on JuliaActuary.org.

JuliaActuary participation

JuliaActuary has a number of “issues” with the hacktoberfest tag on Github across all the repositories which identify “easier” or introductory contribution ideas. See this project board for items with the hacktoberfest tag or browse individual packages’ issues page.

If you participate and submit at least two of your contributions to JuliaActuary, we will mail you a JuliaActuary sticker as thanks!

Low-code or no-code contributions

We welcome contributions that have low or no-code in them. That could be as simple as new documentation, changing some website CSS, writing a blog post about your experience, etc. Hacktoberfest has a list of low/no-code suggestions.

If you have ideas but aren’t sure where to begin, start a discussion here: https://github.com/orgs/JuliaActuary/discussions.

Reminder of “Why Julia?”

In the 2021 Stack Overflow Survey, Julia was the 5th most loved language. This was ahead of other languages commonly used in actuarial contexts, such as Python (6th), R (28th), C++ (25th), Matlab (36th), or VBA (37th).

There are three main reasons to consider using Julia:

1. The language itself offers expressiveness, pleasant syntax, and less boilerplate than many alternatives. Multiple dispatch is a programming paradigm that is an evolution of object-oriented approaches that’s more amenable to a wide range of programming styles, including functional and vectorized approaches. It affords Julia code a high level of composability and is what makes the Julia ecosystem so powerful.

2. High performant Julia code, instead of needing libraries written in C/Cython/etc. For lots of problems, especially “toy” problems as you learn a language, the speed of Matlab/Python/R is fast enough. However, in real usage, particularly actuarial problems, you might find that when you need the performance, it’s too late¹

3. The language, tooling, and ecosystem is very modern, mature, and powerful. A built-in package manager, packages that work together without needing to know about each other, differentiable programming, meta-programming (macros), first-class GPU/parallel support, and wide range of packages relevant to actuarial workloads.

More detail of why Julia works so well in actuarial contexts was discussed in the prior article, Julia for Actuaries. The rest of this article is going to focus on getting oriented to using Julia, and the next article in the series will introduce actuarial packages available.

The language itself

Julia is a high level language, with syntax that should feel familiar to someone coming from R, Python, or Matlab. This article is too short for a true introduction to the language. Fortunately, there’s a ton of great references online, such as:

• Matlab-Python-Julia Cheatsheet
• Learn Julia in Y minutes
• Julia Official Documentation

Julia code is compiled on-the-fly, generating efficient code for the specific data that you are currently working with. This is kind of an in-between of a fully interpreted language (like pure Python or R) and a complied language like C++ which must compile everything in advance.

Born of a desire to have the niceties of high level languages with the performance of low level compiled code, there are many built-in data structures and functions related to numerical computing like that used in finance, insurance, and statistics.

If you just want a quick introduction for beginners, Julia For Data Science is a great resource with easy, digestible tutorials. If you want a ground-up introduction, the e-book Think Julia starts simple and builds up. If you prefer to learn-by-example: the interactive, free, online course Introduction to Computational Thinking by MIT will have you working on everything from data science to climate modeling.

Lastly, JuliaAcademy.com has a number of free courses that introduce the language via data science, machine learning, or “for nervous beginners.”

ols(x,y) = inv(x'x)x'y

# alternatively, one can write this using matrix division:
ols(x,y) = x \ y

Installation and Tooling

A Whirlwind Tour of General-Purpose Packages

The Julia ecosystem favors composability and interoperability, enabled by multiple dispatch. In other words, because it’s easy to automatically specialize functionality based on the type of data being used, there’s much less need to bundle a lot of features within a single package.

As you’ll see, Julia packages tend to be less vertically integrated because it’s easier to pass data around. Counterexamples of this in Python and R:

• Numpy-compatible packages that are designed to work with a subset of numerically fast libraries in Python
• special functions in Pandas to read CSV, JSON, database connections, etc.
• The Tidyverse in R has a tightly coupled set of packages that works well together but has limitations with some other R packages

Julia is not perfect in this regard, but it’s neat to see how frequently things just work. It’s not magic, but because of Julia features outside the scope of this article it’s easy for package developers (and you!) to do this.

Julia also has language-level support for documentation, so packages can follow a consistent style of help-text and have the docs be auto-generated into web pages available locally or online.

The following highlighted packages were chosen for their relevance to typical actuarial work, with a bias towards those used regularly by the authors. This is a small sampling of the over 6000 registered Julia Packages⁴

julia> using Zygote

julia> function policy_av(pol)
    COIs = [0.00319, 0.00345, 0.0038, 0.00419, 0.0047, 0.00532]
    av = 0.0
    for (i,coi) in enumerate(COIs)
        av += av * pol.credit_rate
        av += pol.annual_premium
        av -= pol.face * coi
    end
    return av                # return the final account value
end

julia> pol = (annual_premium = 1000, face = 100_000, credit_rate = 0.05);

julia> policy_av(pol)        # the ending account value
4048.08

julia> policy_av'(pol)       # the derivative of the account value with respect to the inputs
(annual_premium = 6.802, face = -0.0275, credit_rate = 10972.52)

julia> using ActuaryUtilities, BenchmarkTools

julia> @benchmark present_value(0.05,[10,10,10])

BenchmarkTools.Trial: 10000 samples with 994 evaluations.
 Range (min … max):  33.492 ns … 829.015 ns  ┊ GC (min … max): 0.00% … 95.40%
 Time  (median):     34.708 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   36.599 ns ±  33.686 ns  ┊ GC (mean ± σ):  4.40% ±  4.55%

  ▁▃▆▆▆██▇▄▃▂         ▁                                        ▂
  █████████████▆▆▇█▇████▇██▇█▇█▇▇▆▆▅▅▅▅▅▄▅▄▄▅▅▅▅▄▄▁▅▄▄▅▄▄▅▅▆▅▆ █
  33.5 ns       Histogram: log(frequency) by time      45.6 ns <

 Memory estimate: 112 bytes, allocs estimate: 1.

Getting Help

Aside from the usual StackOverflow, there is a community page with links to the Discourse forum, Slack, and Zulip. The latter two have a dedicated #actuary channel.

Summary

This article introduced Julia, getting setup with running and editing, and pointed toward a number of general-purpose features and packages useful to actuaries. The next article in the series will focus on the range of actuarial-specific packages available.

The Life Modeling Problem

Inspired by the discussion in the ActuarialOpenSource GitHub community discussion, folks started submitted solutions to what someone referred to as the “Life Modeling Problem”. This user submitted a short snippet for consideration of a representative problem.

My take on the characteristics are that modeling life actuarial science problems breaks down to the following items:

• Calculations are recursive in nature
• Computationally intensive calculations
• Performance matters given large volumes of data to process
• Readability and usability aids in controls and risk

csubs←{0=x←⊃⍸⍺⍷⍵:0 ⋄ 1+⍺∇(¯1+x+⍴⍺)↓⍵}

function countsubstring(s1, s2) result(c)
  character(*), intent(in) :: s1, s2
  integer :: c, p, posn
 
  c = 0
  if(len(s2) == 0) return
  p = 1
  do 
    posn = index(s1(p:), s2)
    if(posn == 0) return
    c = c + 1
    p = p + posn + len(s2) - 1
  end do
end function

using System;
 
class SubStringTestClass
{
   public static int CountSubStrings(this string testString, string testSubstring)
   {
        int count = 0;
 
        if (testString.Contains(testSubstring))
        {
            for (int i = 0; i < testString.Length; i++)
            {
                if (testString.Substring(i).Length >= testSubstring.Length)
                {
                    bool equals = testString.Substring(i, testSubstring.Length).Equals(testSubstring);
                    if (equals)
                    {
                        count++;
                        i += testSubstring.Length - 1;
                    }
                }
            }
        }
        return count;
   }
}

Benchmarks

After the original user submitted a proposal, others chimed in and submitted versions in their favorite languages. I have collected those versions, and run them on a consistent set of hardware[^3].

Some “submissions” were excluded because they involved an entirely different approach, such as memoizing the function calls[^2].

To aid in visualizing results with such vast different orders of magnitude, this graph includes a physical length comparison to serve as a reference. The computation time is represented by the distance that light travels in the time for the computation to complete (comparing a nanosecond to one foot length goes at least back to Admiral Grace Hopper).

To aid in visualizing results with such vast different orders of magnitude, this graph includes a physical length comparison to serve as a reference. The computation time is represented by the distance that light travels in the time for the computation to complete (comparing a nanosecond to one foot length goes at least back to Admiral Grace Hopper).

Benchmark Discussion

# via Houstonwp
q <- c(0.001,0.002,0.003,0.003,0.004,0.004,0.005,0.007,0.009,0.011)
w <- c(0.05,0.07,0.08,0.10,0.14,0.20,0.20,0.20,0.10,0.04)
P <- 100
S <- 25000
r <- 0.02

base_r_npv <- function(q,w,P,S,r) {
  inforce <- c(1,head(cumprod(1-q-w), -1))
  ncf <- inforce * P - inforce * q * S
  d <- (1/(1+r)) ^ seq_along(ncf)
  sum(ncf * d)
}

base_r_npv(q,w,P,S,r)
#> [1] 50.32483
microbenchmark::microbenchmark(base_r_npv(q,w,P,S,r))

#![feature(test)]
extern crate test;
use test::Bencher;

// Via Paddy Horan
pub fn npv(mortality_rates: &Vec<f64>, lapse_rates: &Vec<f64>, interest_rate: f64, sum_assured: f64, premium: f64, init_pols: f64, term: Option<usize>) -> f64 {

    let term = term.unwrap_or_else(|| mortality_rates.len());
    let mut result = 0.0;
    let mut inforce = init_pols;
    let v: f64 = 1.0 / (1.0 + interest_rate);

    for (t, (q, w)) in mortality_rates.iter().zip(lapse_rates).enumerate() {
        let no_deaths = if t < term {inforce * q} else {0.0};
        let no_lapses = if t < term {inforce * w} else {0.0};
        let premiums = inforce * premium;
        let claims = no_deaths * sum_assured;
        let net_cashflow = premiums - claims;
        result += net_cashflow * v.powi(t as i32);
        inforce = inforce - no_deaths - no_lapses;
    }

    result
}
#[bench]
fn bench_xor_1000_ints(b: &mut Bencher) {

let q: Vec<f64> = vec![0.001,0.002,0.003,0.003,0.004,0.004,0.005,0.007,0.009,0.011];
let w: Vec<f64> = vec![0.05,0.07,0.08,0.10,0.14,0.20,0.20,0.20,0.10,0.04];
let p: f64 = 100.0;
let s: f64 = 25000.0;
let r: f64 = 0.02;
    b.iter(|| {
        // use `test::black_box` to prevent compiler optimizations from disregarding
        // unused values
        test::black_box(npv(&q,&w,r,s,p,1.0,Some(10)));
    });
}

import timeit
setup='''
import numpy as np
q = np.array([0.001,0.002,0.003,0.003,0.004,0.004,0.005,0.007,0.009,0.011])
w = np.array([0.05,0.07,0.08,0.10,0.14,0.20,0.20,0.20,0.10,0.04])
P = 100
S = 25000
r = 0.02
def npv(q,w,P,S,r):
    decrements = np.cumprod(1-q-w)
    inforce = np.empty_like(decrements)
    inforce[:1] = 1
    inforce[1:] = decrements[:-1]
    ncf = inforce * P - inforce * q * S
    t = np.arange(np.size(q))
    d = np.power(1/(1+r), t)
    return np.sum(ncf * d)
'''
benchmark = '''npv(q,w,P,S,r)'''

print(timeit.timeit(stmt=benchmark,setup=setup,number = 1000000))

using BenchmarkTools

q = [0.001,0.002,0.003,0.003,0.004,0.004,0.005,0.007,0.009,0.011]
w = [0.05,0.07,0.08,0.10,0.14,0.20,0.20,0.20,0.10,0.04]
P = 100
S = 25000
r = 0.02

function npv1(q,w,P,S,r) 
    inforce = [1.; cumprod(1 .- q .- w)[1:end-1]] 
    ncf = inforce .* P .- inforce .* q .* S
    d = (1 ./ (1 + r)) .^ (1:length(ncf))
    return sum(ncf .* d)
end

@benchmark npv($q,$w,$P,$S,$r)

function npv5(q,w,P,S,r,term=nothing)
    term = term === nothing ? length(q) : term
    inforce = 1.0
    result = 0.0
    v = (1 / ( 1 + r))
    v_t = v
    for (q,w,_) in zip(q,w,1:term)
        result += inforce * (P - S * q) * v_t
        inforce -= inforce * q + inforce * w
        v_t *= v
    end
    return result
end

More flexibility, more performance from Julia

I wanted to go a little bit deeper and show how 1) Julia just runs fast even if your not explicitly focused on performance. But for where it really matters, you can go even deeper. This is a little advanced, but I think it can be useful to introduce some basics as to why some languages and approaches are going to be fundamentally slower than others.

Notes: The accumulator approach vectors and allocations talk about stack/heap? pick a faster approach and explain why it’s even faster.

Colophon

Julia Version 1.7.0-DEV.938
Commit 2b4c088ee7* (2021-04-16 20:37 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin20.3.0)
  CPU: Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, cyclone)

1.53.0-nightly (b0c818c5e 2021-04-16)

Python 3.9.4 (default, Apr  4 2021, 17:42:23) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin

Footnotes

[^1] A take on Andy and Bill’s law

[^2] If benchmarking memoization, it’s essentially benchmarking how long it takes to perform hashing in a language. While interesting, especially in the context of incremental computing, it’s not the core issue at hand. Incremental computing libraries exist for all of the modern languages discussed here.

[^3] Note that not all languages have both a mean and median result in their benchmarking libraries. Mean is a better representation for a garbage-collected modern language, because sometimes the computation just takes longer than the median result. Where the mean is not available in the graph below, median is substituted.

[^4] Don’t prematurely optimize. But in the long run avoid, re-writing your code in a faster language too many times!

[^5] I’ve seen 50+ line, irregular Excel formulas. To Nick: it probably started out as a good idea but it was a beast to understand and modify! At least with code you can look at the code with variable names and syntax highlighting! Comments, if you are lucky!

[^6] This is not legal advice, consult a lawyer for more details.

Julia Overview

Julia is relatively new¹, and it shows. It is evident in its pragmatic, productivity-focused design choices, pleasant syntax, rich ecosystem, thriving communities, and its ability to be both very general purpose and power cutting edge computing.

With Julia: math-heavy code looks like math; it’s easy to pick up, and quick-to-prototype. Packages are well-integrated, with excellent visualization libraries and pragmatic design choices.

Julia’s popularity continues to grow across many fields and there’s a growing body of online references and tutorials, videos, and print media to learn from.

Large financial services organizations have already started realizing gains: BlackRock’s Aladdin portfolio modeling, the Federal Reserve’s economic simulations, and Aviva’s Solvency II-compliant modeling. The last of these has a great talk on YouTube by Aviva’s Tim Thornham, which showcases an on-the-ground view of what difference the right choice of technology and programming language can make. Moving from their vendor-supplied modeling solution was 1000x faster, took 1/10 the amount of code, and was implemented 10x faster².

The language is not just great for data science — but also modeling, ETL, visualizations, package control/version management, machine learning, string manipulation, web-backends, and many other use cases.

For the Actuary

Julia is well suited for actuarial work: easy to read and write and very performant for large amounts of data/modeling.

# Define our data structures
struct Life
  policies
end

struct Policy
  face
  cessions
 end

struct Cession
  ceded
end

# define retention
function retained(pol::Policy)
  pol.face - sum(cession.ceded for cession in pol.cessions)
end

function retained(l::Life)
  sum(retained(policy) for policy in life.policies)
end

julia> pol_1 = Policy( 1000, [ Cession(100), Cession(500)] )
julia> pol_2 = Policy( 2500, [ Cession(1000) ] )

julia> life = Life([pol_1, pol_2])

julia> retained(pol_1)
400

julia> retained(life)
1900

julia> retained.(life.policies) # retained amount for each policy
[400 ,  1500]

using Distributions

rand(100)
rand(Normal(), 100)
rand(Bernoulli(0.5), 100)
rand(["Preferred","Standard"], 100)

runif(100)
rnorm(100)
rbern(100, 0.5)
sample(c("Preferred","Standard"),
100, replace=TRUE)

import scipy.stats as sps
import numpy as np


sps.uniform.rvs(size=100)
sps.norm.rvs(size=100)
sps.bernoulli.rvs(p=0.5,size=100)
np.random.choice(["Preferred","Standard"],
size=100)

Conclusion

Looking at other great tools like R and Python, it can be difficult to summarize a single reason to motivate a switch to Julia, but hopefully this article piqued an interest to try it for your next project.

That said, Julia shouldn’t be the only tool in your tool-kit. SQL will remain an important way to interact with databases. R and Python aren’t going anywhere in the short term and will always offer a different perspective on things!

In an earlier article, I talked about becoming a 10x Actuary which meant being proficient in the language of computers so that you could build and implement great things. In a large way, the choice of tools and paradigms shape your focus. Productivity is one aspect, expressiveness is another, speed one more. There are many reasons to think about what tools you use and trying out different ones is probably the best way to find what works best for you.

It is said that you cannot fully conceptualize something unless your language has a word for it. Similar to spoken language, you may find that breaking out of spreadsheet coordinates (and even a dataframe-centric view of the world) reveals different questions to ask and enables innovated ways to solve problems. In this way, you reward your intellect while building more meaningful and relevant models and analysis.

Footnotes

The insurance industry: yesterday, today, and tomorrow

It might be odd to say that technology and its use in insurance is on a one-hundred-year cycle, but that seems to be the case.

130 years ago, actuaries crowded into a room at a meeting of the Actuarial Society of America to watch a demonstration that would revolutionize the industry: Herman Hollerith’s tabulating punch card machine².

For the next half-century, the increasing automation — from tabulating machines to early-adopting mainframes and computers — was a critical competitive differentiator. Companies like Prudential, MetLife, and others partnered with technology companies in the development of hardware and software³.

The dramatic embodiment of this information-driven cycle was portrayed in the infamous Billion Dollar Bubble movie, which showcased the power and abstraction of the computer to commit millions of dollars of fraud by creating and maintaining fake insurance policies.

The movie also starts to hint at the oscillation away from the technological-competitive focus of insurance companies. I argue that the focus on technology was lost over the last 50 years with the rise of Wall Street finance, investment-oriented life insurance, industry consolidation, and the explosion of financial structuring like derivatives, reserve financing, or other advanced forms of reinsurance.

Value-add came from the C-Suite, not from the underlying business processes, operations, and analysis. The result is, e.g., ever-more complicated reinsurance treaties layered into mainframes and admin systems older than most of the actuaries interfacing with them.

The pace of strategic value-add isn’t slowing, though it must stretch further (in complexity and risk) to find comparable opportunities as the past. Having more agile, data-oriented operations enables companies to be able to react to and implement those opportunities. Technological value-add can improve a company’s bottom line through lower expenses and higher top-line growth, but often with a more favorable risk profile than some of the “strategic” opportunities.

Today, there is a trend reverting back to technological value-creation and is evident across many traditional sectors. Tesla claims that it’s a technology company; Amazon is the #1 product retailer because of its vehement focus on internal information sharing⁴; Airlines are so dependent on their systems that the skies become quieter on the rare occasion that their computers give way.

Why is it, that companies that are so involved in things (cars, shopping) and physical services (flights) are so much more focused on improving their technological operations than insurance companies whose very focus is ‘information-based’? The market has rewarded those who have prioritized their internal technological solutions.

Commoditized investing services and low yield environments have reduced insurance companies’ comparative advantage to “manage money”. Yield compression and the explosion of consumer-oriented investment services means a more competitive focus on the ability to manage the entire policy lifecycle efficiently (digitally), perform more real-time analysis of experience and risk management, and handle the growing product and regulatory complexity.

These are problems that have technological solutions and are waiting for insurance company adoption.

Companies that treat data like coordinates on a grid (spreadsheets) will get left behind. Two main hurdles have prevented technology companies from breaking into insurance:

1. High regulatory barriers to entry, and
2. Difficulty in selling complex insurance products without traditional distribution.

Once those two walls are breached, traditional insurance companies without a strong technology core will struggle to keep up. The key to thriving is not just adding “developers” to an organization; it’s going to be getting domain experts like actuaries to be an integral part of the technology transformation.

What’s coding got to do with this?

Everything. Programming is the optimal way to interact between the computer and actuary — and importantly between computer and computer. Programming is the actionable expression of ideas, math, analysis, and information. Think of programming as the 21st-century leap in the actuary’s toolkit, just as spreadsheets were in the preceding 40 years. Versus a spreadsheet-oriented workflow:

• More natural automation of, and between processes
• Better reproducibility
• Scaling to fit any size dataset and workload
• Statistics and machine learning capabilities
• Advanced visualizations to garner new views into your data

This list isn’t comprehensive and some benefits are subtle — when you are code-oriented instead of spreadsheet-oriented, you tend to want to structure your data in a portable and shareable way. For example, relying more on data warehouses instead of email attachments. This, in turn, enables data discovery and insights that otherwise wouldn’t be there. Investing in a code-oriented workflow is playing the long-game.

The actuary of the future needs to have coding as one of their core skills. Already today, the advances of business processes, insurance products, and financial ingenuity are written with lines of code — not spreadsheets. Not being able to code necessarily means that you are following what others are doing today.

It’s commonly accepted now that to gather insights from your data, you need to know how to code. Similar to your data, your business architecture, modeling needs, and product peculiarities are often better suited to customized solutions. Why stop at data science when learning how to solve problems with a computer?

The 10x Actuary

As we swing back to a technological focus, we do not leave the finance-driven complexity behind. The increasingly complex business needs will highlight a large productivity difference between an actuary who can code and one who can’t — simply because the former can react, create, synthesize, and model faster than the latter. From the efficiency of transforming administration extracts, summarizing and aggregating valuation output, to analyzing claims data in ways that spreadsheets simply can’t handle, you can become a “10x Actuary”⁵.

Flipping switches in a graphical user interface versus being able to build models is the difference between having a surface-level familiarity and having full command over the analysis and the concepts involved — with the flexibility to do what your software can’t.

Your current software might be able to perform the first layer of analysis but be at a loss when you want to visualize, perform sensitivity analysis, statistics, stochastic analysis, or process automation. Things that, when done programmatically, are often just a few lines of additional code.

Do I advocate dropping the license for your software vendor? No, not yet anyway. But the ability to supplement and break out of the modeling box has been an increasingly important part of most actuaries’ work.

Additionally, code-based solutions can leverage the entire-technology sector’s progress to solve problems that are hard otherwise: scalability, data workflows, integration across functional areas, version control and versioning, model change governance, reproducibility, and more.

30-40 years ago, there were no vendor-supplied modeling solutions and so you had no choice but to build models internally. This shifted with the advent of vendor-supplied modeling solutions. Today, it’s never been better for companies to leverage open source to support their custom modeling, risk analysis/monitoring, and reporting workflows.

Risk Governance

Code-based workflows are highly conducive to risk governance frameworks as well. If a modern software project has all of the following benefits, then why not a modern insurance product and associated processes?

• Access control and approval processes
• Version control, version management, and reproducibility
• Continuous testing and validation of results
• Open and transparent design
• Minimization of manual overrides, intervention, and opportunity for user error
• Automated trending analysis, system metrics, and summary statistics
• Continuously updated, integrated, and self-generating documentation
• Integration with other business processes through a formal boundary (e.g. via an API)
• Tools to manage collaboration in parallel and in sequence

Managing and Leading the Transformation

The ability to understand the concepts, capabilities, challenges, and lingo is not a dichotomy, it’s a spectrum. Most actuaries, even at fairly high levels, are still often involved in analytical work. Still above that, it’s difficult to lead something that you don’t understand.

Conversely, the skill and practice of coding enhances managerial capabilities. When you are really skilled at pulling apart a problem or process into its constituent parts and designing optimal solutions; that’s a core attribute of leadership: having the vision of where the organization should be instead of thinking about where it is now.

Nor is the skillset described here limiting in any other aspect of career development any more than mathematical ability, project collaboration, or financial acumen — just to name a few.

Outlook

It will increasingly be essential for companies to modernize to remain competitive. That modernization isn’t built with big black-box software packages; it will be with domain experts who can translate the expertise into new forms of analysis - doing it faster and more robustly than the competition.

SpaceX doesn’t just hire rocket scientists - they hire rocket scientists who code.

Be an actuary who codes.

Future Articles

The forthcoming series of articles will help illustrate what this can look like in practice: examining the business case, acquainting with ways of interacting with problems outside of spreadsheets and dataframes, and approaching work in a way that removes the boring parts and focuses on the concepts and insights.

The next article in this series will discuss what tools enable an actuary to deliver on the vision outlined in this article. In particular, it will highlight the Julia programming language, a tool well positioned to enable actuaries to develop the analysis, systems, and models of tomorrow.

Footnotes

Instructions to Run

Because JuliaActuary doesn’t have an active server to run this on, you have to run it locally. Assuming that you already have Julia installed but still need to install Pluto notebooks:

1. Open a Julia REPL and copy and paste the following:

# install these dependencies
import Pkg; Pkg.add(["Pluto"]) 

# use and start Pluto
using Pluto; Pluto.run()

2. In the Pluto window that opens, enter this URL into the Open from file: box:

https://raw.githubusercontent.com/JuliaActuary/Learn/master/CashflowAnalysis.jl

Runing the system

This results in the following plot. The tracked output variables u1 and u2 represent the two vars that we tracked above: account value and cumulative premium.

begin
    p = params(
        8000.0, # 8,000 annual premium
        0.08    # 8% interest
    )

    # calculate the number of days to project
    projection_end_date = p.issue_date + Year(p.projection_years)
    days_to_project = Dates.value(projection_end_date - p.issue_date)

    # the [0.0,..] are the initial conditions for the tracked variables
    prob = DiscreteProblem(policy_projection, [0.0, 0.0, 0], (0, days_to_project), p)
    proj = solve(prob, FunctionMap())

    plot(proj)
end

Moving up the ladder of abstraction

An excellent way to understand the behavior of a model is to move up the ladder of abstraction. Below, we will see what happens to the projection at varying levels of credit rates and annual premiums.

md”The full range of simulated outcomes can take a couple of minutes to run:

begin
    prem_range = 1000.0:100.0:9000.0
    int_range = 0.02:0.0025:0.08

    function ending_av(ann_prem, int, days_to_project)
        p = params(ann_prem, int)
        prob = DiscreteProblem(policy_projection, [0.0, 0.0, 0], (0, days_to_project), p)
        proj = solve(prob, FunctionMap())
        end_av = proj[end][1]
        if end_av == 0.0
            lapse_time = findfirst(isequal(0.0), proj[1, 2:end])
        else
            lapse_time = length(proj)
        end
        duration = proj[3, lapse_time]
        end_age = p.issue_age + duration - 1.0

        return end_av, end_age

    end

    end_age = zeros(length(prem_range), length(int_range))
    end_av = zeros(length(prem_range), length(int_range))

    # loop through each projection we did and fill our ranges with the ending AV and ending age
    for (i, vp) in enumerate(prem_range)
        for (j, vi) in enumerate(int_range)
            end_av[i, j], end_age[i, j] = ending_av(vp, vi, days_to_project)
        end
    end
end

Now let’s plot the result. Not surprising, interest has a huge effect on the policy projection. Premium is also a major influence.

One thing that’s remarkable is how going from 2000 premium to just ~2200 of premium results in about a $5m difference at 8% interest. The power of compound interest!

viz = plot(layout=2) # side by side plot

# 
contour!(viz[1], int_range,
    prem_range,
    end_av ./ 1e6, # scale to millions for readability
    contour_labels=true,
    c=cgrad(ColorSchemes.turbo.colors),
    fill=true,
    title="AV at age 100 (M)",
    ylabel=L"Annual Premium (\$)",
    xlabel="credit rate")

contour!(viz[2], int_range,
    prem_range,
    end_age, contour_labels=true,
    c=cgrad(ColorSchemes.turbo.colors),
    fill=true,
    yaxis=false,
    title="Age at Lapse",
    xlabel="credit rate"
)

annotate!(viz[2], (0.055, 7000, Plots.text("Doesn't lapse \nbefore age 100", 8, :white, :center)))