# Permutation symmetry and finding a noise model’

If we have a time series of data points $\{ x_i\},$ how can we tell if these come from some stationary distribution or if there is some secular process coupled to an uncertainty inducing step? Many statistical theorems start out by assuming stationarity. Suppose we ask for the probability of a model $P(f,\eta|x_i)$ where $x_i = f(t_i) + \eta_i,$  $t_i$ is the time coordinate’ of the $i^{\rm th}$ data point, and $\eta$ comes from a stationary distribution. How do we disentangle $f$ and $\eta?$ By definition, the $f$ likelihood should be invariant under replacing $\eta_i \rightarrow \eta_{\sigma(i)}, \sigma \in P_N,$ the permutation group on $N$ data points. In other words, $P(f,\eta|D) = P(f,\eta_\sigma|D),$ but then this is going to require $P(D|f,\eta) = P(D|f,\eta_\sigma)$ which suggests the trivial solution of just defining $P(D|f,\langle\eta\rangle)$ as the product over $\sigma,$ which seems a bit heavy-handed. It is easily achieved, of course, but this will be a computational nightmare. If this could be done, I think it would be a pretty strong constraint on what $\eta$ and $f$ could be.

One way to try to do this would be to do Monte-Carlo sampling over $P_N$ in calculating the cost function at each step of the MCMC for finding $f, \eta.$ We really want to evaluate something like

$- \log P(x|f,\langle\eta\rangle) \equiv \sum_{\sigma\in P_N} \sum_i (x_i - f(t_i) - \eta_{\sigma(i)})^2,$

and I think that for $N$ large enough the sum over $\sigma$ could be approximated by a probabilistic sampling over some permutations. The worry here is that $\eta$ will come out all constant, so the noise model preferred will be the trivial distribution, so all the variation will try to be fit by $f.$ The other extreme is that $f$ will be a constant and all the data will be fit by $\eta.$

The quick way might be to use something like singular spectrum analysis on each permuted summand and demand that the SSA interpolations all agree: Pick $\eta,$ compute SSA for some sample of permutations, vary $\eta$ in a standard MCMC to sample over possible $\eta.$ The end result should be a de-noised prediction for the actual functional dependence $f.$

Advertisements

## One comment on “Permutation symmetry and finding a `noise model’”

1. Carson Chow says:

This is certainly ill-posed for an arbitrary f. Using the SSA is implicitly assuming a prior of smoothness. But there could be other priors.