I’m trying to run dynare 4.4.3 in parallel with the MCMC routine on Mac OS X 10.9.5 Mavericks with Matlab R2014a Student Version. The parallel_test call completes fine, but when trying to run the code with parallel call locally it gets stuck at
ssh @localhost rm -f //random_walk_metropolis_hastings_core_output_*.mat: Signal 127
ssh @localhost rm -f //comp_status_random_walk_metropolis_hastings_core*.mat: Signal 127
ssh @localhost rm -f //slaveParallel_break.mat: Signal 127
The MCMC window with status bars is stuck saying initialize with 4 bars corresponding to 4 cores on my machine.
I also had to make a change to the AnalyseComputationalEnvironment.m file at line 548:
It used to say
[si0 de0]=system('grep processor /proc/cpuinfo');
I had to change it to
[si0 de0]=system('sysctl -a | grep machdep.cpu | grep core_count');
To get it to work
Any idea why it gets stuck at this spot? Help would be appreciated.
I attached the .mod file I’m working with and the config file. I tried attaching the data but I see I can’t seem to upload it. Hopefully you can find some data to make it work for you if you need to run the code test_small_base2.mod (11.9 KB) Model.txt (7.5 KB) AnalyseComputationalEnvironment.m (22.8 KB) dynare_parallel.txt (89 Bytes)
I’m using OS X 10.9.5 (Mavericks), Dynare 4.4.3, Matlab R2014a Student Version.
I added the data as tab delimited .txt file to the main post. It will need to be converted to a .mat file to be used in the .mod file.
Please simply just rename the .mat file as a .txt file and upload it again.
Please also upload DYNARE_estimdata
As long as you can type “ssh localhost” into Terminal (you’re running this in parallel on your computer, right?) and it works, everything should be fine.
I uploaded an updated version of the file. It will read in ‘Model.txt’ and use the data in it directly. No need to change anything to get it to read the file anymore. I also removed the step that used Dynare_estimdata. Hope this helps.
Unfortunately, not yet. You can keep track at github.com/DynareTeam/dynare/issues/838. If you have any insights into what is going wrong, they are highly appreciated. As I am not a Mac user, it is hard for me to help.
I am not using a snapshot. I wish I could, but under Mac OS Sierra there are some MEX incompatibility issues between Matlab R2017a and the pre-compiled versions of Dynare (including the snapshots). So, based on another thread (i.e. Saving the estimated shocks, where you have also posted) I had to use Homebrew to compile Dynare, which works nicely following the hints in the aforementioned thread.
However, I only managed to compile the stable version, but not the HEAD version. Below I give the output of the installation command
"brew install dynare --HEAD --with-matlab=/Applications/MATLAB_R2017a.app --with-matlab-version=9.2 --without-octave --with-tex"
Compilation of the stable version (so without the --HEAD option) works fine, but unfortunately not the HEAD version. I guess the fix for parallel Dynare under Mac OS is only implemented in the HEAD version and not in the stable one (at least parallel Dynare doesn’t work under the stable version on my laptop with Mac OS Sierra, Matlab R2017a).
Is there some other solution? For example, is it possible to replace a couple of Dynare files by newer versions from the GitHub repo?
Any advice would be greatly appreciated!!
Best, Joris
Output from HEAD install:
==> Installing dynare from homebrew/science
==> Installing dependencies for homebrew/science/dynare: boost
==> Installing homebrew/science/dynare dependency: boost
==> Downloading homebrew.bintray.com/bottles/bo … tle.tar.gz
######################################################################## 100.0%
==> Pouring boost-1.64.0.sierra.bottle.tar.gz
/usr/local/Cellar/boost/1.64.0: 12,628 files, 395.7MB
==> Installing homebrew/science/dynare --HEAD --with-matlab= --with-matlab-version= --with-tex --without-octave
==> Using Homebrew-provided fortran compiler.
This may be changed by setting the FC environment variable.
==> Cloning github.com/DynareTeam/dynare.git
Updating /Users/Joris/Library/Caches/Homebrew/dynare–git
==> Checking out branch master
==> autoreconf -fvi
Last 15 lines from /Users/Joris/Library/Logs/Homebrew/dynare/01.autoreconf:
2017-04-21 09:28:45 +0200
Besides, the full 01.autoreconf file is included below. Still don’t know why the HEAD version does not compile (while the stable version does compile fine).
Thanks for pointing this out. The people at Homebrew update the formulas as other things change in their project. It’s a huge undertaking and I think that’s how this bug appeared. I fixed it here and it has been merged into homebrew-science. So, you should now be able to install the latest version of Dynare by running:
Thanks for the quick response! With your instructions, I managed to install the HEAD version of Dynare.
Initially, even with the HEAD version, I could not run Dynare in parallel. However, after experimenting with some settings, I finally managed to run Dynare in parallel. I was not aware that I had to set the MatlabOctavePath option in the .dynare config file (the various PDFs with documentation seem to be a little bit confusing about this), but this option did the trick for me. See below how my .dynare config file looks like.
So far so good. But then I realised that the HEAD version (unstable 4.5) runs considerably slower than the current stable version. Given that it was my own mistake that the stable version did not run in parallel, I thought maybe I could go back to the stable version and use the appropriate .dynare config file, and run the stable version in parallel after all. This works nicely. So, I recorded the timings of some example estimation with the following results (I did run every version twice to be sure):
Parallel 4.5: 3m38s
Parallel 4.4.3: 2m16s
Normal 4.5: 6m11s
Normal 4.4.3: 4m49s
So, at least on my laptop and for my example estimation, parallel Dynare is considerably faster than normal Dynare (as is the purpose of using parallel Dynare in the first place). But the bad news seems to be that the newer version of Dynare is considerably slower than the current stable version. I thought this is useful information to share …
What is your experience with the relative speed of Dynare 4.5 (unstable) vs Dynare 4.4.3 (stable)? Do you have an explanation for my finding?
works. That should be documented in the manual of the unstable version.
I did a quick profiling and the reason seems to be a commit making the Kalman filter more robust github.com/DynareTeam/dynare/pull/1088 by properly conditioning a matrix. The increases the runtime of the Kalman filter by 30 to 40 percent in my tests. Thanks for reporting this. I opened a ticket: github.com/DynareTeam/dynare/issues/1437