Comparing models using log data density vs using moments

[kofiemma]

I have two estimated models I want to compare, which I can do using, say, the log data density. Everything is the same in both models except for different monetary policy rules. I have the following results:

Model 1: larger log data density, but data moments using the estimated parameters are off by about 50-100% relative to moments from actual data.

Model 2: smaller log data density, but data moments using the estimated parameters are off by about 10-30% relative to moments from actual data.

Is Model 1 still better?

[jpfeifer]

We had a similar discussion in

How to see whether the model fit the real data well?

You are confusing fit in the context of full information and limited information estimation techniques. They imply a different weighting of moments. See above: The marginal data density will take the moments into account, but in a not obvious way. Also, you may not care about the moments that the MDD assigns the highest weight.

[kofiemma]

Many thanks, Prof Pfeifer!! I understood but I still have a little confusion. So more like, use MDD to first select features and compare models. But there is no guarantee that moments would be matched under the estimated parameters. So secondly, I can use, say, SMM to estimate the model again and match moments better, right? Which I guess would be necessary only if the estimated model using Bayesian methods did not match data moments well…

[jpfeifer]

The point is which moments you match. SMM will match the targeted moments, MDD will target all available moments, not just the ones you care about most.

[kofiemma]

Thanks, Prof. Pfeifer, for the reply. I understand. But I guess I do not know which moments I want to match. :). I am doing something related to optimal monetary policy. Is there is rule of thumb or some guide on which moments you should care about? When I was in graduate school, we kind of never talked about other moments…and it was all about matching seconds moments. Now that I am thinking about it, maybe it was because a lot of the models were calibrated and we cannot really calibrate the model to match the 10th moment, for example. But estimation does that, right? So let’s say data correlation is like -0.4 between x and y variables, and that of the model (using estimated parameters) is like -0.7, the discrepancy may be due to the estimated model trying to much all moments, yeah?

@jpfeifer

The point is which moments you match.

I was thinking before that maybe I should match the second moment really well to use the model for any analysis…even after having estimated the model. But it seems that is not correct thinking…and thus, it may be better to use the estimated parameters just like that even if second moments based on those parameters are a little off…because some other higher moments may be well matched.

Indeed, I do not know which moments to match for which task. Any comments will be greatly appreciated.

[jpfeifer]

But if you already estimated your model using full information techniques, then you are done. There is nothing else you need to do. Theoretically, that’s the best you can do (unless the model is misspecified)