# Question about the evaluation of estimation results?

**URL:** <https://forum.dynare.org/t/question-about-the-evaluation-of-estimation-results/15835>\
**Category:** ML/Bayesian estimation\
**Created:** [18 May 2020 07:28 UTC](https://forum.dynare.org/t/question-about-the-evaluation-of-estimation-results/15835 "2020-05-18T07:28:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jamess](https://forum.dynare.org/letter_avatar_proxy/v4/letter/j/87869e/32.png) [@jamess](https://forum.dynare.org/u/jamess)\
**Post date:** [18 May 2020 07:28 UTC](https://forum.dynare.org/t/question-about-the-evaluation-of-estimation-results/15835/1 "2020-05-18T07:28:38Z")

</div>

Dear Professor Pfeifer,

I set a DSGE model and run the Bayesian Estimation. The mode plots and the convergence of the two chains all looks OK. I thought the estimation results are good until I compare the distance between the model-implied moments and their empirical counterparts. The empirical moment are not in the 95% probability interval of model-implied moment as I wish.

Then,  
(1) Does this mean that there is some problem with my model setting ?  
(2) How should I improve this ? Is there any method to adjust the model-implied moment and make it close to the empirical ones ?  
(3) Could I get better results if I try simulated method of moments estimation ?

Thank you for your precious time !  
Best regards

---

<div class="post-metadata">

**Author:** ![jpfeifer](https://forum.dynare.org/user_avatar/forum.dynare.org/jpfeifer/32/5044_2.png) [@jpfeifer](https://forum.dynare.org/u/jpfeifer)\
**Post date:** [18 May 2020 08:07 UTC](https://forum.dynare.org/t/question-about-the-evaluation-of-estimation-results/15835/2 "2020-05-18T08:07:12Z")

</div>

That usually happens if the model has a hard time fitting the data. While the model has uncorrelated shocks, they may be correlated in sample. This is typically a sign of model misspecification, but not unheard of.

---

<div class="post-metadata">

**Author:** ![jamess](https://forum.dynare.org/letter_avatar_proxy/v4/letter/j/87869e/32.png) [@jamess](https://forum.dynare.org/u/jamess)\
**Post date:** [19 May 2020 07:02 UTC](https://forum.dynare.org/t/question-about-the-evaluation-of-estimation-results/15835/3 "2020-05-19T07:02:15Z")

</div>

Dear Professor Pfeifer,

Thank you for your reply!

(1) Do you mean that the only way to improve is resetting the model ? However, the problem still exists after I simplified the model to the best of my ability. Is there any other possible mistake that may cause this type of problem ?

(2) When doing the comparison between model-implied moments and their empirical counterparts, I use posterior mean and command  
“stoch\_simul (order=1,irf=0,periods=2500000)”  
and then calculate with 10,000 simulations of the same length as the data. For every simulated sample, I do the first-difference of every interested variable(e.g. y c i pi ) and calculate the standard deviation. Am I doing it correctly ?

Also, I see some paper use “order=3,pruning” in stoch\_simul command. What’s the difference between them? I tried “order=3,pruning” and get very huge value of moments.

Thanks again for your time! I really appreciate your kindness.  
Best regards

---

<div class="post-metadata">

**Author:** ![jpfeifer](https://forum.dynare.org/user_avatar/forum.dynare.org/jpfeifer/32/5044_2.png) [@jpfeifer](https://forum.dynare.org/u/jpfeifer)\
**Post date:** [19 May 2020 10:19 UTC](https://forum.dynare.org/t/question-about-the-evaluation-of-estimation-results/15835/4 "2020-05-19T10:19:25Z")

</div>

If the problem persists despite a correct comparison, then you can only modify your model.  
Regarding the comparison, you should compare the theoretical moments (periods=0) of the observed variables to the ones in the data.
