Java Heap Space/Out of Memory

Hi everyone,

when running a large MCMC (several million draws, more than 50 parameters) in Dynare 4.1 on Matlab 2009b (64bit) I always encounter the following error message after some time (about 1.2 million draws):

[quote]java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedWriter.(Unknown Source)
at java.io.BufferedWriter.(Unknown Source)
at java.io.PrintWriter.(Unknown Source)
at
com.mathworks.jmi.OpaqueJavaInterface.getExceptionMessage(OpaqueJavaInterface.java:1313)
??? Error using ==> drawnow
Failed to retrieve Exception Message

Error in ==> waitbar at 108
drawnow;

Error in ==> random_walk_metropolis_hastings_core at 116
waitbar(prtfrc,hh, ‘(’ int2str(b) '/‘
int2str(options_.mh_nblck) ‘) ’ sprintf(’%f done, acceptation rate
%f’,prtfrc,isux/j)]);

Error in ==> random_walk_metropolis_hastings at 73
fout = random_walk_metropolis_hastings_core(localVars, fblck,
nblck, 0);

Error in ==> dynare_estimation_1 at 1050

feval(options_.posterior_sampling_method,‘DsgeLikelihood’,options_.proposal_distribution,xparam1,invhess,bounds,gend,data,…

Error in ==> dynare_estimation at 62
dynare_estimation_1(var_list,varargin{:});

Error in ==> MYMODEL at 698
dynare_estimation(var_list_);

Error in ==> dynare at 132
evalin(‘base’,fname) ;

Exception in thread “AWT-EventQueue-0” java.lang.OutOfMemoryError: Java
heap space
at java.awt.image.DataBufferInt.(Unknown Source)
at java.awt.image.Raster.createPackedRaster(Unknown Source)
at
java.awt.image.DirectColorModel.createCompatibleWritableRaster(Unknown
Source)
at java.awt.image.BufferedImage.(Unknown Source)
at
com.sun.java.swing.plaf.windows.XPStyle$SkinPainter.createImage(Unknown
Source)
at sun.swing.CachedPainter.paint0(Unknown Source)
at sun.swing.CachedPainter.paint(Unknown Source)
at
com.sun.java.swing.plaf.windows.XPStyle$Skin.paintSkinRaw(Unknown Source)
at
com.sun.java.swing.plaf.windows.AnimationController.paintSkin(Unknown
Source)
at com.sun.java.swing.plaf.windows.XPStyle$Skin.paintSkin(Unknown
Source)
at com.sun.java.swing.plaf.windows.XPStyle$Skin.paintSkin(Unknown
Source)
at
com.sun.java.swing.plaf.windows.WindowsScrollBarUI.paintTrack(Unknown
Source)
at javax.swing.plaf.basic.BasicScrollBarUI.paint(Unknown Source)
at javax.swing.plaf.ComponentUI.update(Unknown Source)
at javax.swing.JComponent.paintComponent(Unknown Source)
at javax.swing.JComponent.paint(Unknown Source)
at javax.swing.JComponent.paintChildren(Unknown Source)
at javax.swing.JComponent.paint(Unknown Source)
at javax.swing.JComponent.paintToOffscreen(Unknown Source)
at
javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
at javax.swing.RepaintManager.paint(Unknown Source)
at javax.swing.JComponent._paintImmediately(Unknown Source)
at javax.swing.JComponent.paintImmediately(Unknown Source)
at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
at
javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown
Source)
at java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
Exception in thread “AWT-EventQueue-0” java.lang.OutOfMemoryError: Java
heap space
Exception in thread “AWT-EventQueue-0” java.lang.OutOfMemoryError: Java
heap space[/quote]

Apparently, the Jave heap space maximum set by Matlab at startup (196 MB for my system) is not large enough. The problem could be solved by following the suggestions from Mathworks (mathworks.com/support/solutions/en/data/1-18I2C/index.html?solution=1-18I2C) to increase the heap space manually.

You have to place a text-file called “java.opts” into your Matlab startup folder or the “\Matlab\2009b\bin\win64”-folder (or “\Matlab\2009b\bin\win32” on 32-Bit systems). The file should contain a single line

, which sets the maximum heap space to 1024MB. You can use any number instead of the 1024. Note that the Matlab default is 196MB for 64-bit systems and 128MB for 32-bit systems. As this default was insufficient in my case and lead to a crash after a quarter of the MCMC, I simply increased it fivefold to be on the safe side (having 4GB of memory).

While the solution above seems to work fine for me, I was wondering if this problem is caused by a bug in Dynare/incompatibility with my system? From what I understand, the MCMC chain is split up and saved in small chunks. Hence, there should be no reason why Java needs so much memory as the length of the MCMC increases. Or am I missing a configuration option that I have to invoke to avoid this issue?

Johannes

Hi Johannes, Thanks for the tip. I never had this problem with java, but when I run very large mcmc I often use -nojvm option (I use matlab in a terminal under linux and don’t need java. With this option you won’t see the progress bar, if you want to get some idea about the progression of the mcmc you can open and read the file metropolis.log). Your understanding is correct the number of iterations in the metropolis should not increase the needed amount of RAM.

Best,
Stéphane.

The same problem occurs under Dynare 4.1, Win XP 32bit, Matlab 2009a, only with a slightly different error message:

[quote]java.lang.OutOfMemoryError: Java heap space
at com.mathworks.jmi.OpaqueJavaInterface.getExceptionMessage(OpaqueJavaInterface.java:1305)
??? Error using ==> drawnow
Failed to retrieve Exception Message

Error in ==> waitbar at 256
drawnow;

Error in ==> random_walk_metropolis_hastings_core at 116
waitbar(prtfrc,hh, ‘(’ int2str(b) ‘/’ int2str(options_.mh_nblck) ‘) ’ sprintf(’%f done, acceptation rate
%f’,prtfrc,isux/j)]);

Error in ==> random_walk_metropolis_hastings at 73
fout = random_walk_metropolis_hastings_core(localVars, fblck, nblck, 0);

Error in ==> dynare_estimation_1 at 1057
feval(options_.posterior_sampling_method,‘DsgeLikelihood’,options_.proposal_distribution,xparam1,invhess,bounds,gend,data,…

Error in ==> dynare_estimation at 62
dynare_estimation_1(var_list,varargin{:});

Error in ==> MYMODEL at 501
dynare_estimation(var_list_);

Error in ==> dynare at 132
evalin(‘base’,fname) ;

Exception in thread “AWT-EventQueue-0” java.lang.OutOfMemoryError: Java heap space[/quote]

Hence, a compatibility issue with 64bit systems can be excluded.