# Parallel estimation in Linux gets stuck after mode
**URL:** https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991
**Category:** ML/Bayesian estimation
**Created:** [30 August 2022 15:58 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991 "2022-08-30T15:58:53Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@manuelgonzalez](https://forum.dynare.org/u/manuelgonzalez)
#### Post date: [30 August 2022 15:58 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/1 "2022-08-30T15:58:53Z")
I set up the configuration file and ran the parallel\_test step with satisfactory results (Test for Cluster computation, computer localhost … Passed!) However, when I actually run the estimation, Dynare gets stuck after computing the mode and just before starting the MCMC chains. The last messages on the screen are:
Estimation::mcmc: Write details about the MCMC… Ok!
Estimation::mcmc: Details about the MCMC are available in SOE\_Estimation/metropolis/SOE\_Estimation\_mh\_history\_0.mat
Matlab shows it is “Busy”, but nothing happens after (no window showing the progress of the MH algorithm). Without the parallel setup, the estimation runs fine. I connect remotely to a virtual machine to run Dynare 4.6.4. Any idea what could be going on?
---
### Author:  [@jpfeifer](https://forum.dynare.org/u/jpfeifer)
#### Post date: [30 August 2022 19:43 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/2 "2022-08-30T19:43:24Z")
Is the config file correctly set up, particularly with respect to the Dynare version?
---
### Author:  [@manuelgonzalez](https://forum.dynare.org/u/manuelgonzalez)
#### Post date: [30 August 2022 20:02 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/3 "2022-08-30T20:02:43Z")
Hi Johannes,
This is the config file:
[cluster]
Name = Local
Members = n1
[node]
Name = n1
ComputerName = localhost
CPUnbr = 12
NumberOfThreadsPerJob = 6
I would have presumed the parallel\_test step would have picked up any mistakes, but it passes the test seamlessly. Thanks!
---
### Author:  [@jpfeifer](https://forum.dynare.org/u/jpfeifer)
#### Post date: [31 August 2022 08:04 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/4 "2022-08-31T08:04:39Z")
No, it does not test everything. Usually, you should also have
```auto
#path to matlab.exe; on Windows, the MATLAB bin folder is in the system path
#so we only need to provide the name of the exe file
MatlabOctavePath=matlab
#Dynare path you are using
DynarePath=C:/Users/Dynare/5.2/matlab
```
---
### Author:  [@manuelgonzalez](https://forum.dynare.org/u/manuelgonzalez)
#### Post date: [31 August 2022 17:52 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/5 "2022-08-31T17:52:12Z")
Thanks, Johannes. I added those lines and after waiting for about 30 minutes after computing the mode, Matlab eventually showed the window with the progress of the MCMC chains (I am not sure the lines added made a difference because I had not waited that long before). This happened when I used fmincon for the mode (mode\_compute=1). Because I wanted to use mode\_compute=6, I restarted the estimation with this configuration. After about 3 hours, Matlab hadn’t shown the progress of the MCMC chains after successfully computing the mode (this is the configuration under which I wrote the present forum post). This is puzzling.
---
### Author:  [@jpfeifer](https://forum.dynare.org/u/jpfeifer)
#### Post date: [1 September 2022 08:01 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/6 "2022-09-01T08:01:27Z")
That is indeed strange. Is your model big? The time between mode-finding and the MCMC is typically used to compute the Hessian when using optimizers that do not directly provide it. That can take some time for big models.
I guess you killed the Dynare run when nothing changed for a long time. Do you know in which function it was stuck?
---
### Author:  [@manuelgonzalez](https://forum.dynare.org/u/manuelgonzalez)
#### Post date: [1 September 2022 12:52 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/7 "2022-09-01T12:52:14Z")
The model has 30ish coefficients and 15 observable variables. I am not sure that is a big model. I tried providing the mode and the covariance matrix obtained with mode\_compute=6 (single core estimation), by doing mode\_compute=0 and mode\_file=model\_name\_mh\_mode.mat, but Matlab spent the whole night stuck after reporting the mode. The message I get after stopping the program is the following:
```auto
Operation terminated by user during dos
In dynareParallelDelete (line 55)
[~, ~] = system(['ssh ',ssh_token,username,Parallel(indPC).ComputerName,' ''/bin/bash --norc -c "rm -f ',directory,pname,fname,'"''']);
In masterParallel (line 205)
dynareParallelDelete([fname,'_output_*.mat'],PRCDir,Parallel);
In posterior_sampler (line 150)
[fout, nBlockPerCPU, totCPU] = masterParallel(options_.parallel, fblck, nblck,NamFileInput,'posterior_sampler_core', localVars, globalVars, options_.parallel_info);
In dynare_estimation_1 (line 474)
posterior_sampler(objective_function,posterior_sampler_options.proposal_distribution,xparam1,posterior_sampler_options,bounds,dataset_,dataset_info,options_,M_,estim_params_,bayestopt_,oo_);
In dynare_estimation (line 105)
dynare_estimation_1(var_list,dname);
In SOE_Estimation.driver (line 1493)
oo_recursive_=dynare_estimation(var_list_);
In dynare (line 293)
evalin('base',[fname '.driver']) ;
```
---
### Author:  [@jpfeifer](https://forum.dynare.org/u/jpfeifer)
#### Post date: [2 September 2022 07:22 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/8 "2022-09-02T07:22:44Z")
So the code gets stuck when trying to delete the `_output_*.mat`-files. Could it be a problem with write permissions?
---
### Author:  [@manuelgonzalez](https://forum.dynare.org/u/manuelgonzalez)
#### Post date: [13 September 2022 17:48 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/9 "2022-09-13T17:48:31Z")
After experimenting a bit, we made the parallel estimation work in Linux by using 4.5.6. Thanks for the feedback.
---
### Author:  [@jpfeifer](https://forum.dynare.org/u/jpfeifer)
#### Post date: [14 September 2022 09:17 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/10 "2022-09-14T09:17:58Z")
That is strange. We experienced this issue in the past when trying to combine parallel estimation on different Dynare versions, i.e. the host had a different version than the respective nodes. That’s why the `DynarePath` was essential.
---
### Author:  [@manuelgonzalez](https://forum.dynare.org/u/manuelgonzalez)
#### Post date: [14 September 2022 11:42 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/11 "2022-09-14T11:42:40Z")
Correct. Actually, I was running on 4.5.6 and had forgotten to change the path in the configuration file (it was still 4.6.4). The results were bad. After aligning both to 4.5.6, things worked well.
---
### Author:  [@jgslazzaro](https://forum.dynare.org/u/jgslazzaro)
#### Post date: [16 September 2022 18:17 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/12 "2022-09-16T18:17:05Z")
I am having the same issue, the error is exactly the same as the one above. I only have 5.0 installed here. This is my conf file:
```auto
[cluster]
Name=Local
Members=n1
[node]
Name=n1
ComputerName=localhost
CPUnbr=4
NumberOfThreadsPerJob=1
MatlabOctavePath=matlab
DynarePath=/usr/lib/dynare/matlab
```
If I run parallel\_test, I get the following error:
```auto
Testing computer -> localhost <- ...
Check on Local Variable ..... Ok!
Checking Hardware please wait ...
Hardware has 16 Cpu/Cores!
User requires 4 Cpu/Cores!
Warning! There are unused CPU's!
Test for Cluster computation, computer localhost ..... Passed!
AnalyseComputationalEnvironment returned with Error Code: 2.2
```
---
### Author:  [@jpfeifer](https://forum.dynare.org/u/jpfeifer)
#### Post date: [19 September 2022 07:04 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/13 "2022-09-19T07:04:08Z")
That is not an true error, just the notice that you are using fewer cores than are available.
---
### Author:  [@sehyoun](https://forum.dynare.org/u/sehyoun)
#### Post date: [14 August 2026 14:05 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/14 "2026-08-14T14:05:20Z")
I ran into a similar problem, but I isolated a different problem than the version conflict noted above.
If you have residual tasks in FevalQueue in your pre-existing parallel pool, estimation will just get “stuck/hang” once it hits parfeval (line 167 of \ file) because new parallel evals are queued behind those (potentially aborted) tasks. For example, this will happen if you abort a parallel estimation with the “stop” button (not the “x” in the dynare progress bar) or ctrl+c, and restart estimation.
This can be fixed in dynare with
- “cancelAll(pool.FevalQueue);” can be added right behind “run\_with\_pct=true;” (line 103 of \) to flush FevalQueue if parallel pool is being reused. This assumes that people do not want to wait for the queued tasks to finish before running dynare, but I consider this to be unlikely. The example above of stopping and restarting estimation runs is a more likely usage pattern.
- Alternately, a warning can be issue prior to parfeval. Something like “If the progress bar does not appear shortly, dynare is waiting for prior queued tasks to finish. If this is not intended, clear parallel eval queue prior to running dynare with cancelALL(gcp(‘nocreate’).FevalQueue)”
- Alternately, a warning can be issued for the most common case that “breaking/stopping” a parallel estimation run doesn’t clear the parallel evalution queue.
I prefer the first option. In the meantime, users can just
- add “cancelAll(gcp(‘nocreate’).FevalQueue);” before running dynare to clear the FevalQueue.
- close parallel pool “delete(gcp(‘nocreate’));” before running dynare (this will be slower)
- Unfortunately, “clear all” doesn’t seem to clear pool.FevalQueue, so you need that ugly(?) line even with clear all.
---
### Author:  [@jpfeifer](https://forum.dynare.org/u/jpfeifer)
#### Post date: [16 September 2026 20:45 UTC](https://forum.dynare.org/t/parallel-estimation-in-linux-gets-stuck-after-mode/20991/15 "2026-09-16T20:45:22Z")
Upon conversing with Mathworks, we opted for [Making sure you're not a bot!](https://git.dynare.org/Dynare/dynare/-/merge_requests/2643).
In theory, the behaviour described is expected. A submitted job goes into the queue, and it will get picked up once the queue reaches it. However, if you “stop” MATLAB, it will not signal to stop the async queue.
Adding a cancel all before jobs start is a bit dangerous, because there is no guarantee that whatever job is running does indeed come from Dynare. Hence, we add an “onCleanup” task that fires at the end of that function and cancels any job that is still running.