Proof-of-concept parallel tempering

[richfitz]

Merge after #104, contains those commits

Closes #24

A basic parallel tempering sampler. There's much more to add (particularly embedding other samplers and sorting out sampling from the hot chain), but this is a start at least

This PR does not include tests of statistical correctness; I'm not really sure how we'll do these generally. Marc and I have picked over this pretty closely with some long runs and it's performing well at the moment.

There's quite a lot of refactoring work to do around the samplers once this is in; see comments for details

old comments from a fairly protracted period of validation follow:

---

Marc, a simple example to get you going. This actually looks pretty good now I think:

set.seed(1) # so we see the same thing
pkgload::load_all()
likelihood <- ex_mixture(5)
prior <- monty_dsl({
  x ~ Normal(0, 10)
})
posterior <- likelihood + prior
s <- monty_sampler_parallel_tempering(n_rungs = 10, vcv = matrix(0.1))
res <- monty_sample(posterior, s, 10000, n_chains = 4)

hist(c(res$pars), freq = FALSE)
x <- seq(min(res$pars), max(res$pars), length.out = 1001)
y <- exp(posterior$density(rbind(x)))
lines(x, y / sum(y) / diff(x)[[1]], col = "red")

(takes 17s on my machine for 40,000 steps, so 2k steps/s which is ok, especially as this is really 11 times that many samples).

I've run out a much longer chain and this is what I see:

set.seed(42) # so we see the same thing
pkgload::load_all()
likelihood <- ex_mixture(5)
prior <- monty_dsl({
  x ~ Normal(0, 10)
})
posterior <- likelihood + prior
s <- monty_sampler_parallel_tempering(n_rungs = 10, vcv = matrix(0.1))
r <- monty_runner_callr(dust2::dust_openmp_threads(10, "fix"))
res <- monty_sample(posterior, s, 50000, n_chains = 10, runner = r)

hist(c(res$pars), freq = FALSE)
x <- seq(min(res$pars), max(res$pars), length.out = 1001)
y <- exp(posterior$density(rbind(x)))
lines(x, y / sum(y) / diff(x)[[1]], col = "red")

(27s for 500,000 points, I can live with that)

I suspect we're out by a factor related to the beta swaps at the moment?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.79%. Comparing base (7dc0478) to head (a5bc87c).

Additional details and impacted files

@@           Coverage Diff            @@
##             main     #106    +/-   ##
========================================
  Coverage   99.79%   99.79%            
========================================
  Files          66       67     +1     
  Lines        5240     5356   +116     
========================================
+ Hits         5229     5345   +116     
  Misses         11       11            

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.