Model_comparison

[harumihas]

Dear all,

I compared four models by the dynare command: model_comparison (marginal_density=laplace). The four models are equaly weighted.

My question is, why the Log Marginal Density reported by the command slighty differs from the one reported in the estimation results in the command window in Matlab.

Thank you very much

[jpfeifer]

How big is the difference? Is it more than rounding?

[harumihas]

Hello Johannes:

Yes, it is. In fact, in some cases, the ranking of the models I am comparing changes. For example:

These are the log densitys of the four models I am comparing that one can see in the command window in Matlab, when all posteriors all displayed too. The “best” model would be M3 (model with higher log density).
M1: 418.4
M2: 410.5
M3: 421.2
M4: 408.1

And using the model_comparison command, I get this:
M1: 419.9
M2: 411.7
M3: 419.5
M4: 408.9

And the “best” model according to this is he M1

[jpfeifer]

Are you comparing the right objects? The marginal data density is reported using the Laplace approach and the modified harmonic mean.

[harumihas]

Well, I am using this:
ESTIMATION RESULTS

Log data density is 418.410999.

parameters
prior mean post. mean 90% HPD interval prior pstdev

rho_rrf 0.650 0.6672 0.5117 0.8383 invg 0.1000
rho_mup 0.300 0.2603 0.1718 0.3585 invg 0.1000
rho_rr 0.300 0.2999 0.2811 0.3161 invg 0.0100
phix 0.800 0.5805 0.4501 0.7241 invg 0.2000
phi 1.500 1.5225 1.2588 1.7569 invg 0.1500

And the command I am using is:
model_comparison(marginal_density = laplace) mod_2018_4_10_17(0.25), mod_2018_1_10_17(0.25), mod_2018_2_10_17(0.25), mod_2018_3_10_17(0.25);

Would that comparison be ok?

Thank you very much

[jpfeifer]

Please provide the full log-file of that model.

[harumihas]

Now I see what you mean. Thank you very much. But, what is exactly the difference between the log data density lkaplace approximation and the other log data density?

Log data density [Laplace approximation] is 419.450586.

Estimation::mcmc: Multiple chains mode.
Estimation::mcmc: Old mh-files successfully erased!
Estimation::mcmc: Old metropolis.log file successfully erased!
Estimation::mcmc: Creation of a new metropolis.log file.
Estimation::mcmc: Searching for initial values…
Estimation::mcmc: Initial values found!

Estimation::mcmc: Write details about the MCMC… Ok!
Estimation::mcmc: Details about the MCMC are available in mod_2018_3_10_17/metropolis\mod_2018_3_10_17_mh_history_0.mat

Estimation::mcmc: Number of mh files: 1 per block.
Estimation::mcmc: Total number of generated files: 2.
Estimation::mcmc: Total number of iterations: 10000.
Estimation::mcmc: Current acceptance ratio per chain:
Chain 1: 4.13%
Chain 2: 22.47%
Estimation::mcmc: Total number of MH draws per chain: 10000.
Estimation::mcmc: Total number of generated MH files: 1.
Estimation::mcmc: I’ll use mh-files 1 to 1.
Estimation::mcmc: In MH-file number 1 I’ll start at line 2501.
Estimation::mcmc: Finally I keep 7500 draws per chain.

MCMC Inefficiency factors per block
Parameter Block 1 Block 2
SE_eps_RR 633.425 227.863
SE_eps_TOT 503.365 189.160
SE_eps_AA_x 621.390 144.295
SE_eps_YY 309.409 85.365
SE_eps_AA 375.983 103.200
SE_eps_INV 317.652 200.704
SE_eps_CC 479.942 221.775
SE_eps_YYx_OBS 538.629 98.647
rho_rrf 627.784 140.894
rho_mup 126.576 333.789
rho_rr 543.699 124.894
phix 449.613 61.778
phi 584.109 106.923
pssi 558.992 131.561
beta 439.276 104.352
alpha_x 549.047 90.655
theta 558.874 77.896
omega 591.967 143.880
sigma 539.671 124.633
epx 576.526 94.695
phi_gdp 297.325 128.103
gama 598.781 96.679
lamda_q 460.611 100.704
lamda_s 434.732 390.720
lamda_gg 414.808 123.975
lamda_xi 582.371 119.177
rho_tot 460.594 109.986
rho_aa_x 557.228 122.367
rho_aa 587.918 141.905
phi_p 524.589 78.587

Estimation::mcmc::diagnostics: Univariate convergence diagnostic, Brooks and Gelman (1998):
Parameter 1… Done!
Parameter 2… Done!
Parameter 3… Done!
Parameter 4… Done!
Parameter 5… Done!
Parameter 6… Done!
Parameter 7… Done!
Parameter 8… Done!
Parameter 9… Done!
Parameter 10… Done!
Parameter 11… Done!
Parameter 12… Done!
Parameter 13… Done!
Parameter 14… Done!
Parameter 15… Done!
Parameter 16… Done!
Parameter 17… Done!
Parameter 18… Done!
Parameter 19… Done!
Parameter 20… Done!
Parameter 21… Done!
Parameter 22… Done!
Parameter 23… Done!
Parameter 24… Done!
Parameter 25… Done!
Parameter 26… Done!
Parameter 27… Done!
Parameter 28… Done!
Parameter 29… Done!
Parameter 30… Done!

Estimation::marginal density: I’m computing the posterior mean and covariance… Done!
Estimation::marginal density: I’m computing the posterior log marginal density (modified harmonic mean)… Done!

ESTIMATION RESULTS

Log data density is 421.220810.

parameters
prior mean post. mean 90% HPD interval prior pstdev

rho_rrf 0.650 0.7009 0.5229 0.8790 invg 0.1000
rho_mup 0.300 0.6615 0.2230 0.9999 invg 0.1000
rho_rr 0.300 0.2973 0.2849 0.3093 invg 0.0100
phix 0.800 0.5256 0.4114 0.6508 invg 0.2000
phi 1.500 1.6852 1.3563 1.9670 invg 0.1500
pssi 0.300 0.2272 0.0950 0.3611 beta 0.1000
beta 1.000 0.7395 0.6373 0.8327 invg 0.2000
alpha_x 0.500 0.5349 0.3587 0.6731 beta 0.1000
theta 1.500 1.4838 1.3247 1.6218 invg 0.1000
omega 2.300 2.2199 1.9436 2.5803 invg 0.2000
sigma 8.500 8.4488 7.7769 9.0810 invg 0.5000
epx 0.500 0.4991 0.4824 0.5129 invg 0.0100
phi_gdp 1.300 1.3157 1.1755 1.4751 invg 0.1000
gama 0.100 0.0805 0.0225 0.1352 beta 0.0500
lamda_q 0.360 0.1935 0.1241 0.2762 beta 0.1000
lamda_s 0.600 0.9643 0.9353 0.9974 beta 0.2000
lamda_gg 0.700 0.6701 0.5231 0.8389 beta 0.1000
lamda_xi 0.300 0.2424 0.1446 0.3706 beta 0.1000
rho_tot 0.950 0.9509 0.9351 0.9648 beta 0.0100
rho_aa_x 0.830 0.8297 0.8153 0.8462 beta 0.0100
rho_aa 0.800 0.8547 0.8028 0.9132 beta 0.1000
phi_p 2.200 2.1964 2.1800 2.2132 invg 0.0100

standard deviation of shocks
prior mean post. mean 90% HPD interval prior pstdev

eps_RR 0.010 0.0305 0.0168 0.0430 invg 2.0000
eps_TOT 0.300 0.0425 0.0357 0.0509 invg 2.0000
eps_AA_x 0.010 0.0052 0.0026 0.0080 invg 2.0000
eps_YY 0.010 0.0128 0.0101 0.0148 invg 2.0000
eps_AA 0.100 0.0309 0.0216 0.0415 invg 2.0000
eps_INV 0.010 0.0285 0.0053 0.0555 invg 2.0000
eps_CC 0.010 0.0169 0.0126 0.0217 invg 2.0000
eps_YYx_OBS 0.010 0.0613 0.0514 0.0743 invg 2.0000
Estimation::mcmc: Posterior (dsge) IRFs…
Estimation::mcmc: Posterior IRFs, done!
Estimation::compute_moments_varendo: I’m computing endogenous moments (this may take a while)…

[jpfeifer]

See

The log marginal data density in mode_computer=6

Both the Laplace approximation and the modified harmonic mean estimator are ways to compute the marginal data density. As computing the marginal data density involves solving a complicated integral, these two methods for tackling the issue have been proposed. In theory, they should yield identical results as they measure the same thing. In practice, both involve approximations and may yield differing results. It is hard to tell which one to prefer. Footnote 11 of Smets/Wouters (2007) states: