I have to run a series of estimation exercises and I am looking for rule of thumbs on how to proceed.
My understanding is the Dynare Bayesian estimation command follows the steps below:
1-a) Compute the likelihood of the observed sample;
1-b) Compute the “posterior” and the “posterior mode” using a mode solver (global or local);
2 ) Use the mode computed at (1-b) as a starting point for the MC-MC which provides an estimate for the parameters;
My questions are:
i) What is the relationship between the posterior at (1-b) and the posterior at step 2?
I have the feeling that when the posterior at (1-b) is flat or truncated (as shown in the mode_check plots),
I can already expect that the prior-posterior graph at (2) will not be smooth and so problematic.
ii) In light of this, I was wondering to run many estimations with few mh_replic for step 2 while varying the
priors - maybe, using a global finder as mode_compute 2 or 6.
Observe the mode_check plots, discard the cases in which the posterior looks weird.
Then, only for the selected cases, run again the estimation with an higher number of draws (maybe, using
mode_compute=0 to upload the previous mode_file).
Would it be a good idea?
Starting the MCMC at the true posterior mode makes the estimation more efficient, i.e. you need fewer draws. But any starting point with any positive definite covariance matrix will work with enough draws. The problem with not being at the true mode is that the inverse Hessian computed at that point is not necessarily positive definite. So you will get a crash when you try to use that matrix as the jumping proposal covariance.
1 - The mode_check option shows a plot with a likelihood and a posterior. When we are lucky these are smooth - i.e. bell-shaped - or in general with an evident maximum. When we are not lucky, it is difficult to see whether there is a maximum (i.e. the lines are piece-wise flat, there are corner solutions, etc. ).
I have noticed that, based on this information, I can have already a feeling of whether the MCMC algorithm (step 2 in my initial post) will provide -in the prior-posterior graph- nice and smooth posteriors or not.
Can I use the information presented in the mode_check plots to avoid running estimations with many draws during my initial estimation attempts i.e. when I am not sure my priors are well fine-tuned?
2 - The second question was related to the previous. As I have mentioned, I am looking for best practices for estimation. At the moment, I am following an iterative process:
i) Choose a set of priors, run the estimation routine and check whether the green dashed-line in the prior-posterior graphs is on a global maximum.
ii-a) If the posteriors are smooth and the mode is on the maximum, I accept the estimation results.
ii-b) If the posteriors are NOT-smooth for ONE parameter. I change the hyper-parameters of the prior
distribution for that parameter and hope things improve. Generally, I repeat this step for several values
of the hyper-parameters.
ii-c) If the posteriors are NOT-smooth for MANY parameters. I start adjusting the hyper-parameters of one
prior. Then, I try with another one, etc.
This way of proceeding is time consuming, so, I was wondering whether one can extrapolate some anticipated conclusions by looking at the “densities” of the mode_check plot. For instance, If I see that the posterior in that plot is ugly, I stop the estimation procedure and try directly with a different prior definition.
Would this be a good way of proceeding?
In general, what kind of best practices would you suggest not to waste time during the estimation exercises?
With corner solutions or failed mode-finding, estimation will typically not run, due to a non-positive definite inverse hessian. So yes, you can infer from the mode_check-plots whether there will be problems.
Your approach is the opposite of a best practice. Your prior must be independent of the data. When you tweak the prior based on the estimation, you are violating that condition.
My recommendation: spend sufficient time in debugging to make sure both your model and your data treatment/observation equations are correct, that all parameters are identifed, etc. After that, use global mode-finders to search for a mode. Potentially, run a sequence of mode-finders. If you do this, estimation should rarely be an big issue.