Re: Parallel (MPI/OpenMP) simulation

[Alejandro_Claro]

Hello Mathias,rnrnI checked the “”Makefile.inc”” and the definitions are:rnrn

Code:

rn## DEFINITIONS TO BE CHANGEDrnrnCXX := g++rn#CXX := icpc -D__aligned__=ignoredrn#CXX := mpiCCrnCXX := mpic++rnrnOPTIM := -O3 -WallrnDEBUG := -g -DOLB_DEBUGrnrnCXXFLAGS := $(OPTIM)rn#CXXFLAGS := $(DEBUG)rnrn# to enable std::shared_ptr in functor arithmetikrn# works in gcc 4.3 and later, source https://gcc.gnu.org/projects/cxx0x.htmlrnCXXFLAGS += -std=c++0xrn# works in gcc 4.7 and later (recommended)rn#CXXFLAGS += -std=c++11rnrn#CXXFLAGS += -fdiagnostics-color=autorn#CXXFLAGS += -std=gnu++14rnrnARPRG := arrn#ARPRG := xiar # mandatory for intel compilerrnrnLDFLAGS :=rnrnrnPARALLEL_MODE := OFFrnPARALLEL_MODE := MPIrn#PARALLEL_MODE := OMPrn#PARALLEL_MODE := HYBRIDrnrnMPIFLAGS :=rnOMPFLAGS := -fopenmprnrnBUILDTYPE := precompiledrn#BUILDTYPE := genericrn

rnrnI removed the executable from the Cavity2d-parallel and the Cylinder2d directories by hand. Then I made “”make clean””, “”make cleanbuild”” and “”make””. The compilation is working the same way as before:rnrnCavity2d-Parallel 4 Coresrn

Code:

rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 152.160srn[Timer] measured time (cpu): 133.202srn[Timer] average MLUPs : 2.100rn[Timer] average MLUPps: 2.100rn[Timer] ———————————————rn[LatticeStatistics] step=18304; t=14.3; uMax=0.1; avEnergy=0.000342504; avRho=1rn/*****************************************************************************/rn[LatticeStatistics] step=18688; t=14.6; uMax=0.1; avEnergy=0.000345858; avRho=1rn[Timer] step=18688; percent=97.3333; passedTime=154.379; remTime=4.22956; MLUPs=3.46913rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 154.859srn[Timer] measured time (cpu): 137.808srn[Timer] average MLUPs : 2.063rn[Timer] average MLUPps: 2.063rn[Timer] ———————————————rn[LatticeStatistics] step=18816; t=14.7; uMax=0.1; avEnergy=0.000346835; avRho=1rn/******************************************************************************/rn[LatticeStatistics] step=19072; t=14.9; uMax=0.1; avEnergy=0.000349094; avRho=1rn[Timer] step=19072; percent=99.3333; passedTime=155.8; remTime=1.04564; MLUPs=4.76521rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 156.165srn[Timer] measured time (cpu): 135.256srn[Timer] average MLUPs : 2.046rn[Timer] average MLUPps: 2.046rn[Timer] ———————————————rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 156.214srn[Timer] measured time (cpu): 136.896srn[Timer] average MLUPs : 2.045rn[Timer] average MLUPps: 2.045rn[Timer] ———————————————rn

rnrnCavity2d-Parallel 3 Coresrn

Code:

rn[LatticeStatistics] step=18176; t=14.2; uMax=0.1; avEnergy=0.000341447; avRho=1rn[Timer] step=18176; percent=94.6667; passedTime=119.872; remTime=6.75335; MLUPs=2.72733rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 120.176srn[Timer] measured time (cpu): 113.472srn[Timer] average MLUPs : 2.659rn[Timer] average MLUPps: 2.659rn[Timer] ———————————————rn/**************************************************************************************/rn[LatticeStatistics] step=19072; t=14.9; uMax=0.1; avEnergy=0.000349094; avRho=1rn[Timer] step=19072; percent=99.3333; passedTime=124.519; remTime=0.835698; MLUPs=3.49762rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 124.710srn[Timer] measured time (cpu): 118.398srn[Timer] average MLUPs : 2.562rn[Timer] average MLUPps: 2.562rn[Timer] ———————————————rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 124.978srn[Timer] measured time (cpu): 118.115srn[Timer] average MLUPs : 2.557rn[Timer] average MLUPps: 2.557rn[Timer] ———————————————rn

rnrnCylinder2d 4 Coresrn

Code:

rn[Timer] step=30800; percent=96.25; passedTime=470.8; remTime=18.3429; MLUPs=2.89811rn[LatticeStatistics] step=30800; t=15.4; uMax=0.0408193; avEnergy=0.000245803; avRho=1.00062rn[getResults] pressure1=0.134373; pressure2=0.0159914; pressureDrop=0.118382; drag=5.63413; lift=0.0122223rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 473.156srn[Timer] measured time (cpu): 362.941srn[Timer] average MLUPs : 2.489rn[Timer] average MLUPps: 2.489rn[Timer] ———————————————rn/******************************************************************************************************/rn[Timer] step=31400; percent=98.125; passedTime=478.282; remTime=9.13915; MLUPs=3.07357rn[LatticeStatistics] step=31400; t=15.7; uMax=0.0408199; avEnergy=0.000245815; avRho=1.00059rn[getResults] pressure1=0.132571; pressure2=0.0142374; pressureDrop=0.118333; drag=5.632; lift=0.0122383rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 479.666srn[Timer] measured time (cpu): 364.392srn[Timer] average MLUPs : 2.455rn[Timer] average MLUPps: 2.455rn[Timer] ———————————————rn/******************************************************************************************************/rn[Timer] step=31800; percent=99.375; passedTime=482.199; remTime=3.0327; MLUPs=3.8062rn[LatticeStatistics] step=31800; t=15.9; uMax=0.040816; avEnergy=0.000245588; avRho=1.00057rn[getResults] pressure1=0.13189; pressure2=0.0135893; pressureDrop=0.118301; drag=5.63043; lift=0.0122372rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 483.783srn[Timer] measured time (cpu): 363.782srn[Timer] average MLUPs : 2.435rn[Timer] average MLUPps: 2.435rn[Timer] ———————————————rn[Timer] rn[Timer] —————-Summary:Timer—————-rn[Timer] measured time (rt) : 483.808srn[Timer] measured time (cpu): 364.824srn[Timer] average MLUPs : 2.434rn[Timer] average MLUPps: 2.434rn[Timer] ———————————————rn

rnrnI also checked the mpi version installed in the machine:rn

Code:

rnPackage: Open MPI buildd@allspice Distributionrn Open MPI: 1.6.5rn Open MPI SVN revision: r28673rn Open MPI release date: Jun 26, 2013rn Open RTE: 1.6.5rn Open RTE SVN revision: r28673rn Open RTE release date: Jun 26, 2013rn OPAL: 1.6.5rn OPAL SVN revision: r28673rn OPAL release date: Jun 26, 2013rn MPI API: 2.1rn Ident string: 1.6.5rn Prefix: /usrrn Configured architecture: x86_64-pc-linux-gnurn Configure host: allspicern Configured by: builddrn Configured on: Sat Dec 28 23:38:31 UTC 2013rn Configure host: allspicern Built by: builddrn Built on: Sat Dec 28 23:41:47 UTC 2013rn Built host: allspicern C bindings: yesrn C++ bindings: yesrn Fortran77 bindings: yes (all)rn Fortran90 bindings: yesrn Fortran90 bindings size: smallrn C compiler: gccrn C compiler absolute: /usr/bin/gccrn C compiler family name: GNUrn C compiler version: 4.8.2rn C++ compiler: g++rn C++ compiler absolute: /usr/bin/g++rn Fortran77 compiler: gfortranrn Fortran77 compiler abs: /usr/bin/gfortranrn Fortran90 compiler: gfortranrn Fortran90 compiler abs: /usr/bin/gfortranrn C profiling: yesrn C++ profiling: yesrn Fortran77 profiling: yesrn Fortran90 profiling: yesrn C++ exceptions: norn Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no)rn Sparse Groups: norn Internal debug support: norn MPI interface warnings: norn MPI parameter check: runtimernMemory profiling support: nornMemory debugging support: norn libltdl support: yesrn Heterogeneous support: yesrn mpirun default –prefix: norn MPI I/O support: yesrn MPI_WTIME support: gettimeofdayrn Symbol vis. support: yesrn Host topology support: yesrn MPI extensions: affinity examplern FT Checkpoint support: yes (checkpoint thread: yes)rn VampirTrace support: norn MPI_MAX_PROCESSOR_NAME: 256rn MPI_MAX_ERROR_STRING: 256rn MPI_MAX_OBJECT_NAME: 64rn MPI_MAX_INFO_KEY: 36rn MPI_MAX_INFO_VAL: 256rn MPI_MAX_PORT_NAME: 1024rn MPI_MAX_DATAREP_STRING: 128rn MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6.5)rn MCA memory: linux (MCA v2.0, API v2.0, Component v1.6.5)rn MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.5)rn MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.6.5)rn MCA carto: file (MCA v2.0, API v2.0, Component v1.6.5)rn MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6.5)rn MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6.5)rn MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6.5)rn MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6.5)rn MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.5)rn MCA timer: linux (MCA v2.0, API v2.0, Component v1.6.5)rn MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6.5)rn MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6.5)rn MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6.5)rn MCA hwloc: external (MCA v2.0, API v2.0, Component v1.6.5)rn MCA crs: none (MCA v2.0, API v2.0, Component v1.6.5)rn MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6.5)rn MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6.5)rn MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6.5)rn MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6.5)rn MCA coll: basic (MCA v2.0, API v2.0, Component v1.6.5)rn MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6.5)rn MCA coll: inter (MCA v2.0, API v2.0, Component v1.6.5)rn MCA coll: self (MCA v2.0, API v2.0, Component v1.6.5)rn MCA coll: sm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA coll: sync (MCA v2.0, API v2.0, Component v1.6.5)rn MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6.5)rn MCA io: romio (MCA v2.0, API v2.0, Component v1.6.5)rn MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6.5)rn MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6.5)rn MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6.5)rn MCA pml: crcpw (MCA v2.0, API v2.0, Component v1.6.5)rn MCA pml: csum (MCA v2.0, API v2.0, Component v1.6.5)rn MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6.5)rn MCA pml: v (MCA v2.0, API v2.0, Component v1.6.5)rn MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6.5)rn MCA btl: ofud (MCA v2.0, API v2.0, Component v1.6.5)rn MCA btl: openib (MCA v2.0, API v2.0, Component v1.6.5)rn MCA btl: self (MCA v2.0, API v2.0, Component v1.6.5)rn MCA btl: sm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6.5)rn MCA topo: unity (MCA v2.0, API v2.0, Component v1.6.5)rn MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6.5)rn MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6.5)rn MCA crcp: bkmrk (MCA v2.0, API v2.0, Component v1.6.5)rn MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6.5)rn MCA iof: orted (MCA v2.0, API v2.0, Component v1.6.5)rn MCA iof: tool (MCA v2.0, API v2.0, Component v1.6.5)rn MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6.5)rn MCA odls: default (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ras: cm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ras: gridengine (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ras: tm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rmaps: load_balance (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rml: ftrm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA rml: oob (MCA v2.0, API v2.0, Component v1.6.5)rn MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6.5)rn MCA routed: cm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA routed: direct (MCA v2.0, API v2.0, Component v1.6.5)rn MCA routed: linear (MCA v2.0, API v2.0, Component v1.6.5)rn MCA routed: radix (MCA v2.0, API v2.0, Component v1.6.5)rn MCA routed: slave (MCA v2.0, API v2.0, Component v1.6.5)rn MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6.5)rn MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA plm: tm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA snapc: full (MCA v2.0, API v2.0, Component v1.6.5)rn MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6.5)rn MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: env (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: slave (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: tm (MCA v2.0, API v2.0, Component v1.6.5)rn MCA ess: tool (MCA v2.0, API v2.0, Component v1.6.5)rn MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6.5)rn MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6.5)rn MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6.5)rn MCA notifier: command (MCA v2.0, API v1.0, Component v1.6.5)rn MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6.5)rn

rnrn1. Do you have any idea of the possibles problems that are not letting me to send the simultion in parallel ?rnrnBest regards,rnrnAlejandro