Permutation symmetry and finding a `noise model’

If we have a time series of data points \{ x_i\}, how can we tell if these come from some stationary distribution or if there is some secular process coupled to an uncertainty inducing step? Many statistical theorems start out by assuming stationarity. Suppose we ask for the probability of a model P(f,\eta|x_i) where x_i = f(t_i) + \eta_i,  t_i is the `time coordinate’ of the i^{\rm th} data point, and \eta comes from a stationary distribution. How do we disentangle f and \eta? By definition, the f likelihood should be invariant under replacing \eta_i \rightarrow \eta_{\sigma(i)}, \sigma \in P_N, the permutation group on N data points. In other words, P(f,\eta|D) = P(f,\eta_\sigma|D), but then this is going to require P(D|f,\eta) = P(D|f,\eta_\sigma) which suggests the trivial solution of just defining P(D|f,\langle\eta\rangle) as the product over \sigma, which seems a bit heavy-handed. It is easily achieved, of course, but this will be a computational nightmare. If this could be done, I think it would be a pretty strong constraint on what \eta and f could be.

One way to try to do this would be to do Monte-Carlo sampling over P_N in calculating the cost function at each step of the MCMC for finding f, \eta. We really want to evaluate something like

- \log P(x|f,\langle\eta\rangle) \equiv \sum_{\sigma\in P_N} \sum_i (x_i - f(t_i) - \eta_{\sigma(i)})^2,

and I think that for N large enough the sum over \sigma could be approximated by a probabilistic sampling over some permutations. The worry here is that \eta will come out all constant, so the noise model preferred will be the trivial distribution, so all the variation will try to be fit by f. The other extreme is that f will be a constant and all the data will be fit by \eta.

The quick way might be to use something like singular spectrum analysis on each permuted summand and demand that the SSA interpolations all agree: Pick \eta, compute SSA for some sample of permutations, vary \eta in a standard MCMC to sample over possible \eta. The end result should be a de-noised prediction for the actual functional dependence f.

Advertisements

One comment on “Permutation symmetry and finding a `noise model’

  1. Carson Chow says:

    This is certainly ill-posed for an arbitrary f. Using the SSA is implicitly assuming a prior of smoothness. But there could be other priors.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s