Acceptance Ratio in the MH algorithm

Dear Peter,

The tuning the scale parameter so that the acceptance rate is between .2 and .4 is only an heuristic. From very simple models, we know that the acceptance rate should depend on dimension of the problem (the number of estimated parameters), and should be in this region. You can find an introduction on this in the book by Robert and Casella (“Monte Carlo Statistical Methods”, chapter 7, in particular section 7.8.4).

We only know for sure that acceptance ratios close to 0 or 1 are bad. I usually target one third. But we cannot be sure, controlling only one parameter, that the acceptance rates will be the same across chains. Normally in the end, if the chains are long enough, the acceptance ratios should be similar across chains.

A small acceptance rate does not mean that the MCMC is trapped in a low density region. Imagine that the current state of the MCMC is the posterior mode. If the jumps provided by the proposal distribution are large it is very likely that all the proposals will be rejected (resulting in a low acceptance rate).

Best,
Stéphane.