Error message when clustering

[hpg-hub]

Hi all, we are currently modelling a nice SAOM on the LRZ RStudio Server Pro. It workes quite well, however when we try to use the parallel package enabling multiple processors, we get an error message which we could not resolve by now.

We have 40 cores available, thus we followed the advice in the help page of siena07. Our code looks as follows:
library(parallel)
(n.clus <- detectCores() -1)
niceResults <- siena07(niceModel, data=niceData, effects=niceEffects, batch=TRUE, useCluster=TRUE, nbrNodes=n.clus, initC=TRUE)

Unfortunately, we get the following error message:
Error in checkForRemoteErrors(lapply(cl, recvResult)) :
39 nodes produced errors; first error: package or namespace load failed for ‘RSiena’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
there is no package called ‘Matrix’

The same occurs when we use the c1 setup as cl <- makeCluster(n.clus) and [...] batch=TRUE, cl = cl.

The package Matrix has already been loaded, thus this seems not to be the "real" issue. Also reinstalling the packages did not help. I read in the RSiena_Manual that it might come to difficulties when running it on Linux, which I believe is the "back end" of the LRZ server. I could not find the answer to my question in the parallel-package documentation. Hence I am wondering whether someone of you could please help me out?

Thank you very much in advance!

[TomSnijders]

This seems to me an R problem, and not specific to RSiena.
Did you try with a lower number of clusters, e.g., 2 or 5?
By the way, I think using 39 clusters if your server has 40 cores might be unfriendly to other users, if this is a server that is also used by others.

[hpg-hub]

Thanks your your reply. Yes, I already tired with fewer cores but it didn't work either.

Do you have experience running RSiena on R in a Linux Server environment? Is there anything specific that needs to be taken into account when run on Linux?

Thanks!

PS: No worries about the users, that's not a problem. I think the LRZ uses hyperthreads (why 40 course should not be the maximum) and manages this with experience.

[hpg-hub]

Hi there, indeed it has been a problem with R itself. Problem solved...thanks.

[TomSnijders]

OK!