Validation analysis

Dear all, I have estimated my DSGE model (bayesian estimation), the mcmc graphs, the validation ratio of brooks and gelman and also the posterior distribution graphs, all confirm the validation of model, now I have some questions:

  1. Are the IRF plots vaild if they simply fulfill the expectations from theories? or do I have to check the moments, if they are close to moments for real data or not?
  2. If I have to check for moments, how close should they be to the moments of real data? how is it for y gap bellow in your opinion?
    1st moments 2nd moments correlation coef.
    1st model -0.00002 0.0088 0.2716
    2nd model 0.00003 0.0082 0.4129
    real data -0.00064 0.0274 0.483
  3. for analyzing the irf plots, are the numbers meaningful (in log linearized model) or should we just care for the directions and overall responses?

Thanks in advance

With full information estimation, you consider your model to be the data generating process. If the moments of your model are very different from the actual data, that is indeed a problem. Some differences may occur, but having a variable being four times as volatile in the data as in the model is worrisome.

And yes, numbers matter. Why else would you want to use a quantitative model?

hello… thanks indeed for your quick response … can u please offer a reference to tell me …which moments are more important?( mean? variance or the correlation coefficient?, for example if for model A mean was closer to real data’s mean and for model B standard error was closer… which model is better?)

question2: how different, is considered worrisome in each moment?

There is no good guidance here. As I said: you are claiming your estimated model is the DGP. If simulated data from the estimated model is very different that the actual data, then you will have a hard time convincing readers that your model indeed is the DGP.

Regarding which moments: people usually just like at the variances and the contemporaneous and first order autocorrelations.

I appreciate your guidance. I am comparing a model with two priors for a parameter: share of rule of thumb households equal to zero ( homogeneous model ) and equal to 0.35 (heterogeneous model ) comparing the moments could not help me choose one, there is no clear difference , can I use the compare command? what information does it provide and is that a manual I could use?
thanks in advance

IF your model is estimated on the same data, you can use model_comparison