Parallel Dynare: Windows network (SMB)

Aldo · September 16, 2017, 2:48am

I used parallel dynare in my laptop in order to estimate my model with four cores. Now, I want to use two laptops. So I suppose it should be in network.

In manual of parallel dynare says:

"3.1 Requirements
3.1.1 For a Windows grid

A standard Windows network (SMB) must be in place;"

So, my question is how do I get

“SMB”? or where could I get it?

Please, help me to use my two laptops to compute parallel.

Thanks

Aldo · September 18, 2017, 5:39pm

Excuse the insistence. Is there anyone who could help me run Dynare on two computers?

jpfeifer · September 19, 2017, 7:37am

For SMB, see https://en.wikipedia.org/wiki/Server_Message_Block
@rattoma Do you have any further insights?

rattoma · October 19, 2017, 6:40am

If you are in a ‘large’ network [like University] and both laptops are in the network, you just follow instructions, e.g. using the IP address or the ‘computer name’ of the ‘remote’ laptop. If you are at ‘home’ I would just set up a local/home network between the two laptops and, again set-up the remote laptop with the computer name or IP.

Aldo · October 22, 2017, 2:31am

Thanks a lot dear rattoma,
I am trying to run in a remote computer with this configuration file:

[cluster]
Name = vonNeumann
Members = n2

[node]
Name = n2
ComputerName = TOSHIBA
CPUnbr = 1
UserName = AREALOCAL
Password = 123
RemoteDrive = D
RemoteDirectory = dynare

But I get, this message:

Check on Local Variable ..... Ok!

Check on ComputerName Variable ..... Ok!


Check on UserName Variable ..... Ok!

Check on Password Variable ..... Ok!

Remote Directory does not exist or it is not reachable!

ErrorCode 5.


AnalyseComputationalEnvironment returned with Error Code: 5

Please, how could I fix this? It is required to create Working Group? and I have another question, what name could I put in name of cluster?
Thanks a lot

Aldo

jpfeifer · October 23, 2017, 7:36am

You can use any name for the cluster you like. If you define more than one cluster, you can tell Dynare at the command line via the parallel[=CLUSTER_NAME] option which one to use
I would recommend setting a debugging breakpoint in AnalyseComputationalEnvironment.m and try to see which command results in this error message. As a first step, you may want to add
disp(de2);
in the respective if (si2)-clauses to see the full error.

rattoma · October 24, 2017, 1:19pm

@jpfeifer thanks. right with debugging strategy.

jpfeifer · October 24, 2017, 1:38pm

@rattoma If we keep the current parallel setup, we should probably by default supply this debugging info in case of a crash.

rattoma · October 24, 2017, 4:37pm

indeed … we should do that

Aldo · November 1, 2017, 4:23pm

When I added:

disp(de2);

I get:

Check on Local Variable … Ok!
Check on ComputerName Variable … Ok!
Check on UserName Variable … Ok!
Check on Password Variable … Ok!

Remote Directory does not exist or it is not reachable!
Acceso denegado.

ErrorCode 5.

But I can create files, folders, modify files and delete files of my remote directory.

I saw AnalyseComputationalEnvironment.m and the error is generated in:

[si2 de2]=system([‘dir \’,DataInput(Node).ComputerName,’’,DataInput(Node).RemoteDrive,’$’,DataInput(Node).RemoteDirectory,’’,RemoteTmpFolder]);

but when I run in command window:

[s1 d2]= system([‘dir \’,‘Toshiba\d\dynare’])

I get this:

s1 =

d2 =

El volumen de la unidad \Toshiba\d es DATA
El n£mero de serie del volumen es: F858-19EB

Directorio de \Toshiba\d\dynare

01/11/2017 16:43 .
01/11/2017 16:43 …
01/11/2017 16:58 RemoteFolder
0 archivos 0 bytes
3 dirs 345.911.570.432 bytes libres

Please, what could it be happen, what could it be my error? Please.

jpfeifer · November 2, 2017, 11:42am

Have you checked whether

[‘dir \’,DataInput(Node).ComputerName,’’,DataInput(Node).RemoteDrive,’$’,DataInput(Node).RemoteDirectory,’’,RemoteTmpFolder]

really evaluates to

[‘dir \’,‘Toshiba\d\dynare’]

?

Aldo · November 2, 2017, 6:00pm

Thanks a lot dear jpfeifer,
I’ve evaluated:

['dir \\',options_.parallel.ComputerName,'\',options_.parallel.RemoteDrive,'$\',options_.parallel.RemoteDirectory,'\',options_.parallel_info.RemoteTmpFolder]

and I got:

dir \\TOSHIBA\d$\\dynare\RemoteFolder\2017-11-2-18h41m32s

That seems weird to me because the directory has $ symbol and folder 2017-11-2-18h41m32s I didn’t create. So I excluded it, and I run:

[si2 de2]=system(['dir \\',options_.parallel.ComputerName,'\',options_.parallel.RemoteDrive,'\',options_.parallel.RemoteDirectory])

then I got:

si2 =

     0


de2 =

 El volumen de la unidad \\TOSHIBA\d es DATA
 El n£mero de serie del volumen es: F858-19EB

 Directorio de \\TOSHIBA\d\dynare\RemoteFolder

01/11/2017  16:58    <DIR>          .
01/11/2017  16:58    <DIR>          ..
02/11/2017  18:27                 8 Nuevo documento de texto.txt
               1 archivos              8 bytes
               2 dirs  345.911.558.144 bytes libres

And of course when I run sentence with no modification, I got the result with error:
[si2 de2]=system(['dir \\',options_.parallel.ComputerName,'\',options_.parallel.RemoteDrive,'$\',options_.parallel.RemoteDirectory,'\',options_.parallel_info.RemoteTmpFolder])

si2 =

 1


de2 =

Acceso denegado.

jpfeifer · November 3, 2017, 8:29am

@rattoma Could it be that there is a bug here and there should be no dollar sign?

rattoma · November 3, 2017, 8:54am

Could you pls run

[si2 de2]=system(['dir \\',options_.parallel.ComputerName,'\', options_.parallel.RemoteDrive,'$\',options_.parallel.RemoteDirectory])

rattoma · November 3, 2017, 9:04am

In other words, the system command
dir \\Toshiba\d$\dynare
should work

without dollar, i.e.
dir \\Toshiba\d\dynare
should not work [unless Windows 10, which I do not have, has changed the syntax].

Then, the bug should be that we should not dir to the temp directory, which is not yet created, i.e. we should probably cut options_.parallel_info.RemoteTmpFolder from the dir command

Aldo · November 4, 2017, 3:08am

Thanks a lot dear rattoma and dear jpfeifer,

I run

[si2 de2]=system(['dir \\',options_.parallel.ComputerName,'\', options_.parallel.RemoteDrive,'$\',options_.parallel.RemoteDirectory])

and I get:

si2 =

     1


de2 =

Acceso denegado.

also I run:

>> dir \\Toshiba\d$\dynare
'\\Toshiba\d$\dynare' not found.
>> dir \\Toshiba\d\dynare

.             ..            RemoteFolder

I am reviewing in https://docs.microsoft.com/en-us/sysinternals/downloads/pstools:

Runs on:
Client: Windows Vista and higher
Server: Windows Server 2008 and higher
Nano Server: 2016 and higher
I am running in windows 7 with Matlab R2015a in my both computers.
When I cut the temp directory and the $ symbol from the dir command in AnalyseComputationalEnvironment.m, that is, when I replace with:

[si2 de2]=system(['dir \\',DataInput(Node).ComputerName,'\',DataInput(Node).RemoteDrive,'\',DataInput(Node).RemoteDirectory]);

and then I run dynare mymod parallel conffile='Config.txt' parallel_test I get:

Testing computer -> TOSHIBA <- ...
Check on Local Variable ..... Ok!
Check on ComputerName Variable ..... Ok!
Check on UserName Variable ..... Ok!
Check on Password Variable ..... Ok!
Check on RemoteDirectory Variable ..... Ok!
Check on RemoteDrive Variable ..... Ok!

Error using copyfile
Acceso denegado.

Error in dynareParallelSendFiles (line 113)
                    copyfile([NamFileInput{jfil,1},NamFileInput{jfil,2}],['\\',Parallel(indPC).ComputerName,'\',Parallel(indPC).RemoteDrive,'$\',Parallel(indPC).RemoteDirectory,'\',PRCDir,'\',NamFileInput{jfil,1}]);
                    
Error in AnalyseComputationalEnvironment (line 385)
        dynareParallelSendFiles('Tracing.m', RemoteTmpFolder,DataInput(Node));

Error in prometheo_p (line 1856)
ErrorCode = AnalyseComputationalEnvironment(options_.parallel, options_.parallel_info);

Error in dynare (line 223)
evalin('base',fname) ;

Please, what could I do to fix this?

Aldo · November 7, 2017, 1:04pm

Dear @rattoma could you give me some advice please?

rattoma · November 9, 2017, 9:48pm

what do you get when you type from matlab command line:

!dir \\Toshiba\d$\dynare

?

Aldo · November 10, 2017, 4:31pm

Dear @rattoma with this last sentence I get:

!dir \Toshiba\d$\dynare
Acceso denegado.
when I put dir \\Toshiba\d\dynare I get:

. Palalelizando Prometheo_2_1
… Prometheo_2 paralelizando.m
and when I put !dir \\Toshiba\d\dynare I get:

El n£mero de serie del volumen es: 14AE-F1E7

Directorio de \Toshiba\d\dynare

07/11/2017 05:52 p. m. .
07/11/2017 05:52 p. m. …
07/11/2017 06:27 p. m. Palalelizando
06/11/2017 11:49 a. m. 144 paralelizando.m
03/11/2017 10:05 p. m. Prometheo_2
01/11/2017 11:10 a. m. Prometheo_2_1
1 archivos 144 bytes
5 dirs 62,643,998,720 bytes libres

rattoma · November 13, 2017, 7:46am

OK. Although this is puzzling [I also have Windows 7 and I need as in all other tests tone in almost 10 yrs the $ in the system commands], it seems in your case you need to eliminate all $ from system commands of the parallel toolbox.
You may simply try that on your local copy of dynare.