I’m not sure but it seems to me that when either the priors or the likelihood are assigned penalty values (e.g. no saddle or out of pdf range) the posterior is still assigned the value of the sum of likelihood and priors.
Would it not be better to assign a penalty to the posterior directly? (The optimization algorithm takes marginal improvements on the penalty as true improvements)
we try to impose quadratic penalties in a smooth manner: we add to the last acceptable value of the objective function, the sum of the square of the eigenvalues that are on the wrong side of one.
So, a penalized value should always be larger than the last acceptable one.
Tell me if you have an exemple that shows that it doesn’t happen that way in practice.