I’m running the Gertler Karadi (2011) model, using the mod file taken from Peter’s webpage modified to work in Dynare++ (code attached for reference; shouldn’t really make any difference, but just for background; original mod file was called Step0FA1++.mod but the pluses didn’t survive the upload, apparently).Step0FA1 .mod (6.0 KB)
ISSUE: Dynare++ reports irfs to positive and negative shocks and I’m finding them to be asymmetric (i.e. positive ones are not mirror images of negative ones). See also figures below where positive and negative IRFs are in red and blue respectively, and where asymmetries are clearly visible (these are IRFs to a monetary policy shock e_i, btw.).
I’ve tried to replicate this in standard dynare, but there everything works as expected - i.e. positive and negative IRFs sum to zero.
The differences in IRFs in dynare++ visible above look too large to be numerical noise (at least to me, see e.g. C top left, or nu…). I’ve also had a look at Euler equation residuals, but nothing struck me as immediately suspicious. So I’m puzzled… Is this a bug, or am I missing something?
Thoughts / suggestions appreciated (I’m using dynare++ distributed with version 4.5.6 and haven’t checked whether the behavior persists in previous builds as well).
With apologies for the delay, I have now had a bit more time to look into this.
I believe that what I’ve plotted above is a bug that was introduced into dynare++ when moving from the version distributed with dynare 4.2.5 to the version distributed with 4.3.0. Specifically, in version 4.2.5 (and all the previous ones I checked, including dynare++ v.1.3.6 and 1.0.1) first order IRFs were still symmetric to machine precision. In contrast, large asymmetries appear at first order from version 4.3.0 onwards, and they’re still there in the version of dynare++ distributed with 4.5.6.Not sure what exactly went wrong, but something appears to have done so…
To see where I’m coming from, I’m plotting the sum of positive and negative impulse responses from a first order approximation of the .mod file attached to my original post. These should equal 0 (or close to). A representative subset of IRFs from dynare++ distributed with dynare v. 4.2.5 looks as follows:
To me these are ‘numerical’ zeros, which is what I would expect.
Keeping everything constant except for the fact that I’m now switching to dynare++ distributed with dynare v.4.3.0 generates the following (again these are sums of IRFs to a positive and negative shock):
For the avoidance of doubt, the exact same thing seems to be happening in the version of dynare++ distributed in the latest stable release of dynare (i.e. 4.5.6):
I’d be grateful if someone on the dynare team could confirm that this is indeed a bug (obviously, I’m happy to stand corrected if I’m missing something and the behavior above is, in fact, desired; for whatever reason…). Presumably the recommendation would then be to use the version of dynare++ distributed with dynare v.4.2.5?
@paw I can confirm that there is a significant change in behavior between Dynare++ 4.2.5 and 4.3
Unfortunately, I am not yet sure what the correct version is. If you use
!dynare++ --sim 10 Step0FA1.mod
In 4.2.5 you will see extremely small differences between the two IRFs and will see that the variance across replication dyn_irfm_e_i_var are very small. If you simulate GIRFs with stochastic shocks, this is highly unusual.
Dynare 4.3 gives you large differences between IRFs, but a reasonable variance across replications. Increasing the number of simulations decreases the differences significantly as expected, but it seems to not be the whole story.
@MichelJuillard@sebastien Do you know what changed? I could not find any information on changes in Dynare++ across versions.
Thanks a lot for looking into this Johannes - appreciated!
On the first point you raise (if I understand it correctly): could this not be due to the fact that the shock sequences used for negative GIRFs are equal to minus the shock sequences used for the positive ones? That way of computing generalized impulse responses would ensure that any differences are only caused by policy function asymmetries, as opposed to sampling uncertainty? This would be why I’d expect the sum of these in a linear model to give something close to (machine precision) zero, and why the differences in variances would also be small?
Even if not the case however, shouldn’t IRFs in a linear model be base independent? Meaning that even if the underlying shocks were not mirror images of one another, the stochastic ‘base’ used for IRF computation should play no role (numerically, probably slightly larger differences than if the procedure above was used, but still close to zero)? I may be missing your point, of course…
I’ve now quickly compared the output of dynare++ v.4.2.5 and v.4.3.0 (generated by compiling the same .mod file attached initially) - and I think the results are quite interesting:
Specifically, the comparison suggests that the only things that differ are the IRFs - both the positive and negative ones. Things like policy function coefficients, Euler equation residuals, means, variances etc. are identical between both versions… Not sure this is helpful or helps narrow the issue down - but thought I’d throw it into the mix.
I was also able to reproduce the bug with the Linux version of Dynare++. Something looks wrong with the impulse used for the positive and the negative IRFs. I will look into it.
@paw I was talking about computing the IRFs at first order as Generalized IRFs. In that case, the IRF is the difference between two stochastic simulations, the one with a given impulse minus the one without the impulse. The random shocks other than the impulse are then averaged/integrated out by averaging over multiple replications. If the random numbers used for positive and negative IRFs differ in these simulations, you would get bigger differences that vanish asympotically.
@jpfeifer Thanks for clarifying Johannes - appreciated. What I was trying to say is that I believe that first order is actually a bit different from all the others and, numerical inaccuracies aside, sampling / law of large numbers should play no role when computing GIRFs (which is presumably one reason why no one computes them in linear models?).
While I may be missing something, if the linear solution is y(t) = Ay(t-1)+Be(t) and the impulse vector is i(1), then the IRF in period t would be A^(t-1)Bi(1) for every series of shocks e(t) underlying a GIRF. Sure, in practice you’re going to get some numerical inaccuracies and the IRFs corresponding to each shock draw won’t be exactly identical, but that would be only due to numerical imprecision, in contrast to higher orders where base effects matter directly (in any case, the numerical inaccuracies should be much smaller than anything to do with sampling; and I’m not sure the law of large numbers would apply to them…)? Anyway, just so we’re on the same page, and as an aside to the thread topic.
@MichelJuillard Many thanks for looking into this Michel - really appreciated. I guess one of my concerns was whether whatever is causing the IRFs to look a bit off could also show up in standard dynare, given that (if my understanding is correct) the third order algorithms build on those underlying dynare++? Any thoughts on this would be welcome.
in Matlab/Octave dynare, we don’t use dynare++ to compute the IRFs. We use the same Matlab code for all orders. We only use dynare++ code to compute the solution of 3rd order and simulate for a given set of shocks.
I still have to locate precisely where is the problem with IRFs in dynare++ code.
Best
Michel
@paw You are right. At first order, the IRFs are linear so there is nothing to integrate out. The GIRFs are the simple IRFs. No need to simulate. So that cannot explain the difference.
What is your ultimate goal? If you are just looking for GIRFs to asymmetric shocks at third order, have a look at Regime switching based on volatility
@jpfeifer Thanks a lot Johannes! Ultimately what I’m after is accuracy stats associated with pruned simulation paths (at 2nd and 3rd order).
My main reason for going back to dynare++ was that it reports three types of numerical accuracy checks (which is what I’m interested in). Unfortunately, the unpruned dynare++ simulations appear unstable for the model I’m working with (a tweak of GK), so I’m not really getting much useful info out of these (they’re infinite). So, I was wondering if you’re aware of existing code reporting related accuracy stats, which you could recommend (it would save me having to write this from scratch)?
Thanks in advance for your help,
Pawel
ps. As an aside: I was looking at the (G)IRFs as a compliment to the accuracy checks, and that’s when I spotted the puzzling asymmetries reported in this thread.
I definitely intend to look at this, but not immediately. As you may have noticed, I’m currently in the process of reviewing and improving the Dynare++ codebase, which is going to take some time.
The problem has been isolated, and will be fixed in the upcoming Dynare 4.5.7.
Basically all IRFs computed by Dynare++ at any order were wrong since version 4.3.0. Essentially the two simulations used for computing the generalized IRF algorithm were not using the same starting point, hence the bogus results.
Fortunately this bug did not affect Dynare itself.
in the code!