Comparing actual data with simulated data

Hi Prof @jpfeifer

I hope you are well and thanks for your assistance so far. I am making good progress.

Please, I would like to know which of the following is the right way to compare the actual data with the simulated data. I estimated a DSGE model with 41 endogenous variables, including 7 observable variables, using the estimation command in Dynare 4.6.

Method 1: In the results folder, I saw 41 rolls of data in the file named oo_.endo_simul. The 41 corresponds to the number of endo variables in the var block. My understanding is that the order of the rolls follows the order in which the endogenous variables were declared in the var block. Because my observables were the last 7 variables in the var block, I copied the last 7 rolls in oo_.endo_simul as my simulated data. The period of the simulated data is 400 (very long) but my actual data is 60 quarters. Hence I compared the moments of the first 60 simulated data for each variable to the corresponding actual data to check whether the model fits each data well. The model fit was good enough.

Method 2: The second thing I tried was calibrate the model using the estimated parameters and standard errors. I then copied the simulated data from the oo_.endo_simul file and compared the data moments with that of my actual data. However the model fit was bad.

I just would like to know which of these two is the right way to compare actual data to simulated data.

1 Like

Generally, have a look at How to see whether the model fit the real data well?

Method 1 is wrong. When a model is estimated with full information techniques, the model will perfectly match the observables. What you are currently doing is comparing 400 periods of artificially simulated data to the observed data. That means there is a stoch_simul somewhere that created this data.


I have read the suggested chat and it is was very informative. Putting my post and the other comments together, my understanding is that option two is correct. Which is estimate the model, use the estimated parameters to calibrate the model, use the stoch_simul command to estimate the calibrated model and generate the simulated data and compare the moments of the actual data and the simulated data. Can you please confirm is my understanding is correct. I also understand that some researchers do more than an eyeballing approach.

You also noted in your comment that

I used the command below for my estimation:
estimation(datafile=Data_demean, prefilter=1, mode_compute = 4,
mh_jscale = 0.30, mode_check,
irf = 40, bayesian_irf, conf_sig = 0.95) y pi i ;

And I used this command to estimate the calibrated model;
stoch_simul(order = 1, irf=40) y pi i;

That looks correct.

Thank you very much @jpfeifer for your response. The fit is bad but i will work on the model to fix that. I have another question.

What is the best practice when calling dynare_sensitivity? do I add the command after the estimation command, or should I estimate the model, calibrate the model with the estimated parameter and add the dynare_sensitivity command after the stoch_simul command. I check the manual and watched a few videos but did not see an example where the dynare_sensitivity command was used after the estimation command.

The question is what you are trying to achieve with that command.

I am examining which parameters in the Taylor rule drives indeterminacy.

Have a look at DSGE_mod/Ascari_Sbordone_2014.mod at master · JohannesPfeifer/DSGE_mod · GitHub

I have followed the steps in this post Comparing actual data with simulated data - #3 by SamBons to generate simulated data after estimation and compare the variance of the series in the data and the model.

I am using consumption growth as one of my observation equations. My issue is that the variance of the consumption growth in the data and the model are very different (an order of magnitude greater in the model). For other observation equation that I have a shock directly effecting that variable, I obtain a much better result. Does that mean there is an error in the estimation? How is it possible that the historical data and the smoothed variables are the same after the estimation, but the simulated model generates very different series for the consumption growth.
Appreciate any helps!

See e.g.

1 Like