MPI run — pvd file fault
OpenLB – Open Source Lattice Boltzmann Code › Forums › on OpenLB › Bug Reports › MPI run — pvd file fault
- This topic has 13 replies, 4 voices, and was last updated 7 years, 1 month ago by Kai.
-
AuthorPosts
-
April 19, 2017 at 2:51 am #1907steed188Participant
Hello,
When I use OpenMPI to parallelly run a case, I found that the pvd file, which should be the main interface for vtk file groups, has some faults ,leading to an error when read in ParaView. Part of the pvd file is like belows:<DataSet timestep=”0″ group=”” part=” 5″ file=”data/Cube<DataSet timestep=”0″ group=”” part=” 5″ file=”data/Cube<DataSet timestep=”0″ group=”” part=” 0″ file=”data/Cube<DataSet timestep=”0″ group=”” part=” 6″ file=”data/Cube_iT0000000iC00006.vti”/>
<DataSet timestep=”0″ group=”” part=” 2″ file=”data/Cube<DataSet timestep=”0″ group=”” part=” 4″ file=”data/Cube_iT0000000iC00004.vti”/>
<DataSet timestep=”0″ group=”” part=” 3″ file=”data/Cube_iT0000000iC00003.vti”/>
</Collection>
</VTKFile>
</Collection>
</VTKFile>
<DataSet timestep=”0″ group=”” part=” 5″ file=”data/Cube_iT0000000iC00005.vti”/>
<DataSet timestep=”0″ group=”” part=” 1″ file=”data/Cube_iT0000000iC00001.vti”/>
</Collection>
</VTKFile>
</Collection>
</VTKFile>
<DataSet timestep=”0″ group=”” part=” 6″ file=”data/Cube<DataSet timestep=”0″ gro<DataSet timestep=”0″ group=”” part=” 1″ file=”data/Cube<DataSet timestep=”0″ group=”” part=” 3″ file=”data/Cube_iT0000000iC00003.vti”/>
up=”” part=” 6″ file=”data/Cube_iT0000000iC00006.vti”/>
<DataSet timestep=”0″ group=”” part=” 5″ file=”data/Cube<DataSet timestep=”0″ group=”” part=” 2″ file=”data/Cube_iT0000000iC00002.vti”/>
</Collection>
</VTKFile>
</Collection>You can see that the order of the vti files is chaotic, even when I ran the example cases. It will only be normal if I ran the case sequential.And I am using OpenLB 1.0, in the platform of CentOS linux 7.3.1611 with OpenMPI 1.6.3.
your sincerely
steed188April 19, 2017 at 2:58 am #2595steed188ParticipantAnd I am using OpenLB 1.0, in the platform of CentOS linux 7.3.1611 with OpenMPI 1.6.3
April 19, 2017 at 12:12 pm #2597albert.minkModeratorDear Steed188,
I attached the sample pvd from cylinder3d. No matter if you you execute sequential or parallel, yout pvd file should look like this.
<?xml version=”1.0″?>
<VTKFile type=”Collection” version=”0.1″ byte_order=”LittleEndian”>
<Collection>
<DataSet timestep=”0″ group=”” part=”” file=”data/cylinder3d_iT0000000.vtm”/>
<DataSet timestep=”900″ group=”” part=”” file=”data/cylinder3d_iT0000900.vtm”/>
…
</Collection>
</VTKFile>By the way, can you open the pvd file with a web browser?
Regards,
AlbertApril 19, 2017 at 2:38 pm #2598steed188ParticipantDear albert.min-k,
Thank you so much for your focus on my question.
When I execute sequential, the pvd file is correct. But when I excute Parellel, the file is wrong. I seems like that the processors scrambled to write the vti files in the pvd file with out an order sequencing, leading the file fault.
And I can’t open the wrong pvd file with web browser, showing error.
Regards,
steed188April 19, 2017 at 2:57 pm #2600albert.minkModeratorYes, I agree. In your situation, several processes want to access the same file and then it crashes somehow.
So this means you might have a careful look on you MPI.
Can you post YOUR Makefile.inc in the OpenLB project?
April 19, 2017 at 4:23 pm #2601steed188ParticipantDear albert.min-k,
I’d love to. I agree with you that it may be my MPI’s fault. I have tried OpenLB1.1. The fault still existed.
Would you mind telling me how can I post the Makefile.inc in the OpenLB project?April 19, 2017 at 4:30 pm #2602albert.minkModeratorI see, the forum does not support any kind of attachement. That is not good.
By now, just copy the first 66 lines of the Makefile.inc in the thread.
Best regards,
AlbertApril 20, 2017 at 1:31 am #2607steed188ParticipantDear albert.min-k,
you mean the Makefile.inc file in the root folder of OpenLB? Listed below:# This file is part of the OpenLB library
#
# Copyright (C) 2007 Mathias Krause
# E-mail contact: info@openlb.net
# The most recent release of OpenLB can be downloaded at
# <http://www.openlb.net/>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public
# License along with this program; if not, write to the Free
# Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
# Boston, MA 02110-1301, USA.###########################################################################
###########################################################################
## DEFINITIONS TO BE CHANGEDCXX := g++
#CXX := icpc -D__aligned__=ignored
#CXX := mpiCC
#CXX := mpic++OPTIM := -O3 -Wall
DEBUG := -g -DOLB_DEBUGCXXFLAGS := $(OPTIM)
#CXXFLAGS := $(DEBUG)# to enable std::shared_ptr in functor arithmetik
# works in gcc 4.3 and later, source https://gcc.gnu.org/projects/cxx0x.html
CXXFLAGS += -std=c++0x
# works in gcc 4.7 and later (recommended)
#CXXFLAGS += -std=c++11#CXXFLAGS += -fdiagnostics-color=auto
#CXXFLAGS += -std=gnu++14ARPRG := ar
#ARPRG := xiar # mandatory for intel compilerLDFLAGS :=
PARALLEL_MODE := OFF
#PARALLEL_MODE := MPI
#PARALLEL_MODE := OMP
#PARALLEL_MODE := HYBRIDMPIFLAGS :=
OMPFLAGS := -fopenmpBUILDTYPE := precompiled
#BUILDTYPE := generic###########################################################################
## conditional settingsifeq ($(BUILDTYPE), precompiled)
CXXFLAGS := -DOLB_PRECOMPILED $(CXXFLAGS)
endififeq ($(PARALLEL_MODE), MPI)
CXXFLAGS := -DPARALLEL_MODE_MPI $(MPIFLAGS) $(CXXFLAGS)
endififeq ($(PARALLEL_MODE), OMP)
CXXFLAGS := -DPARALLEL_MODE_OMP $(OMPFLAGS) $(CXXFLAGS)
LDFLAGS := $(OMPFLAGS) $(LDFLAGS)
endififeq ($(PARALLEL_MODE), HYBRID)
CXXFLAGS := -DPARALLEL_MODE_OMP -DPARALLEL_MODE_MPI $(OMPFLAGS) $(MPIFLAGS) $(CXXFLAGS)
LDFLAGS := $(OMPFLAGS) $(LDFLAGS)
endif###########################################################################
## defines shellSHELL := /bin/sh
###########################################################################
## dependencies, object, library directory and library nameDEPENDDIR := build/$(BUILDTYPE)/dep
OBJDIR := build/$(BUILDTYPE)/obj
LIBDIR := build/$(BUILDTYPE)/lib
LIB := olb###########################################################################
## search directoriesSUBDIRS := src/boundary \
src/communication \
src/dynamics \
src/core \
src/geometry \
src/external/tinyxml \
src/functors \
src/functors/indicator \
src/io \
src/particles \
src/particles/forces \
src/particles/boundaries \
src/utilitiesEXAMPLEDIRS := examples/aorta3d \
examples/bifurcation3d/eulerEuler \
examples/bifurcation3d/eulerLagrange \
examples/bstep2d \
examples/bstep3d \
examples/cavity2d/sequential \
examples/cavity2d/parallel \
examples/cavity3d/sequential \
examples/cavity3d/parallel \
examples/cylinder2d \
examples/cylinder3d \
examples/multiComponent2d \
examples/multiComponent3d \
examples/nozzle3d \
examples/phaseSeparation2d \
examples/phaseSeparation3d \
examples/poiseuille2d/bgk \
examples/poiseuille2d/forced \
examples/poiseuille2d/mrt \
examples/thermal2d \
examples/thermal3d \
examples/venturi3dINCLUDEDIRS := src \
src/ \
src/externalBUILDTYPEDIRS := build/precompiled \
build/genericSUBDIRSH := $(foreach d,$(SUBDIRS),$(ROOT)/$(d))
IDIR := $(foreach d,$(INCLUDEDIRS),-I$(ROOT)/$(d))###########################################################################
###########################################################################April 20, 2017 at 9:29 am #2609albert.minkModeratorWell, this is the entire Makefile.inc 😉
Anyway, modify it according
CXX := mpic++
PARALLEL_MODE := MPIThen, run
make cleanbuild
make clean
make// Till now, you executed a sequential programm on several threads. With the above modifications the compiler will generate a parallel program.
Albert
May 15, 2017 at 5:13 am #2622steed188ParticipantI tried to modified the Makefile.inc like belowed
#CXX := g++
#CXX := icpc -D__aligned__=ignored
#CXX := mpiCC
CXX := mpic++
…….
#PARALLEL_MODE := OFF
PARALLEL_MODE := MPI
#PARALLEL_MODE := OMP
#PARALLEL_MODE := HYBRIDand made it again. But it didn’t work. The pvd file was still bad.
yours steed188
May 15, 2017 at 11:20 am #2623steed188ParticipantHi Albert,
I got the point. My OpenMPI’s version was 1.6.3. I changed it to version 1.10.0. And it works.
I don’t know why.
Thank you very much.
Yours steed188July 29, 2017 at 11:01 am #2679KaiMemberWith mpich-3.1.3, openLB 1.1r0 has the same problem when running in parallel.
“Error parsing XML in stream at line 25, column 7, byte index 4045797: not well-formed(invalid token)” …
The example is multiComponent2d.
July 31, 2017 at 6:45 pm #2681Markus MohrhardParticipantQuote:Quote from Kai on July 29, 2017, 11:01
With mpich-3.1.3, openLB 1.1r0 has the same problem when running in parallel.“Error parsing XML in stream at line 25, column 7, byte index 4045797: not well-formed(invalid token)” …
The example is multiComponent2d.
Hi Kai,
can you paste your Makefile.inc and make sure that you used a MPI frontend compiler, e.g. mpic++ or mpiCC and used the MPI mode.
In Makefile.inc the following two lines are important:
CXX := mpic++
PARALLEL_MODE := MPIAfter making sure that these are the only CXX and PARALLEL_MODE lines that are not prefixed with a # go to examples/multComponent2d and call:
make clean cleanbuild && make && mpirun -np 4 ./rayleighTaylor2d
This should make sure that the code is built with MPI support and that the code is run in parallel through MPI. If you still see the problem please paste the Makefile.inc file so that we can continue searching the problem.
Regards,
MarkusAugust 1, 2017 at 8:34 am #2683KaiMemberHi, Markus, I found the reason now, as posted in another board.
“There are more than one mpi installation in my machine. I used mpirun and mpic++ from different installations (maybe more things were messed up). Now it works fine.”
It was a mess/mistake from my side.
-
AuthorPosts
- You must be logged in to reply to this topic.