NSlattice.getLoadBalancer().size() is always 1 independent of blocks number

[jflorezgi]

Hello,

I am writing a function similar to the function created to compute the Nusselt number in squareCavity2d example: computeNusselt(SuperGeometry<T,2>& superGeometry,
SuperLattice<T, NSDESCRIPTOR>& NSlattice,
SuperLattice<T, TDESCRIPTOR>& ADlattice)

to add the value of q in every point of the wall through the entire wall is necessary look in each block of the superStructure. This is done with a loop inside the function:

for (int iC = 0; iC < NSlattice.getLoadBalancer().size(); iC++) {

where the member function NSlattice.getLoadBalancer().size() gives the number of blocks of the structure which I assume is the same number of blocks given by the mpirun -np # ./squareCavity2d instruction, but this function always is 1, as you can check. So the function is only calculating the value of Nusselt in the first block regardless of the number of blocks into which the superStructure is divided.

I appreciate a little clarification on this.

Best wishes

[Adrian]

The LoadBalancer::size method returns the rank-local number of cuboids, i.e. the number of blocks that each MPI process individually needs to process. The sum of the individual sizes is the total number of cuboids in the block decomposition.

For most situations it is sufficient to divide the problem into one block per process which is what you observe here.

[jflorezgi]

thanks for your response I understand what you’re saying, so I have a question.

I want to make a function that reviews all the cells in a horizontal plane at a certain height and, depending on the value of a specific functor in each cell, counts the number of cells that have the same value of the functor.

When I run the program with mpirun -np 1 ./…. and I look at the number of cells it is checking it gives me, for example:

nx = sLatticeNS.getBlock(iC).getNx() = 323 ny = sLatticeNS.getBlock(iC).getNy()= 415 and the number of cells that the program is checking is the multiplication of these two values, as I expected, but if I run the program with mpirun -np 2 ./…., the values are nx = sLatticeNS.getBlock(iC).getNx() = 323 ny = sLatticeNS.getBlock(iC).getNy()= 208 and the number of cells that the program is checking is the multiplication of these two values also, so depending on the number of blocks, the function only calculates the inverse (of the number of blocks) of the cells it should calculate.

the code of the function is:

void computeAirAngleFraction(SuperGeometry<T,3>& superGeometry,
UnitConverter<T, NSDESCRIPTOR> &converter,
SuperLattice<T,NSDESCRIPTOR>& sLatticeNS,
SuperLatticeAirAngleClass3D<T, NSDESCRIPTOR>& airAngleClass,
std::ofstream& fileAngleFr, T breathHeight)
{
OstreamManager clout( std::cout,”computeAirAngleFraction” );
clout << “Computing the Airflow Angle Fraction …” << std::endl;

int iZ = converter.getLatticeLength(breathHeight);

AnalyticalFfromSuperF3D<T> intpolateAirAngleClass( airAngleClass, true );

int material = 0;
//Write the macroscopic variables through the horizontal plane at breathHeight
T numerical[5] { };
T position[3] { };

for (int iC = 0; iC < sLatticeNS.getLoadBalancer().size(); iC++) {

for (int iY = 0; iY < sLatticeNS.getBlock(iC).getNy(); ++iY) {
for (int iX = 0; iX < sLatticeNS.getBlock(iC).getNx(); ++iX) {

material = superGeometry.getBlockGeometry(iC).getMaterial(iX,iY,iZ);
if (material == 1){
position[0] = (T)iX * converter.getPhysDeltaX();
position[1] = (T)iY * converter.getPhysDeltaX();
position[2] = breathHeight/*(T)iZ * converter.getPhysDeltaX()*/;

intpolateAirAngleClass(numerical, position);
//clout << numerical[0] << ” ” << position[0] << ” ” << position[1] << ” ” << position[2] << std::endl;

if (numerical[0] == 0.){
numerical[2]++;
numerical[4]++;
}
else if (numerical[0] == 1.){
numerical[3]++;
numerical[4]++;
}
else{
numerical[1]++;
numerical[4]++;
}
}
}
}
}

clout << “breathHeight = ” << breathHeight << ” ” << numerical[1] << ” ” << numerical[2] << ” ” << numerical[3] << ” ” << numerical[4] << ” ” << numerical[1] / numerical[4] << ” ” << numerical[2] / numerical[4] << ” ” << numerical[3] / numerical[4] << std::endl;

fileAngleFr << “breathHeight = ” << breathHeight << ” ” << numerical[1] << ” ” << numerical[2] << ” ” << numerical[3] << ” ” << numerical[4] << ” ” << numerical[1] / numerical[4] << ” ” << numerical[2] / numerical[4] << ” ” << numerical[3] / numerical[4] << std::endl;
}

I would appreciate if you can review the function and give me some idea about my question.

[Adrian]

Yes, this is exactly what is expected as one divides the problems into blocks. Each function instance (one per independent MPI process) only has access to its local data (by design, this is “the point of MPI”).

If you want to manually sum up and aggregate over the global simulation state in this way you will need to also call the MPI primitives (wrapped by OpenLB for convenience) on your own (e.g. MPI_Reduce).

The “OpenLB way” of implementing your function would be to wrap you per-cell criterion in a custom functor that returns either 0 or 1 depending on if the criteria is met. This functor can then be passed into one of the existing integral functors (e.g. we have BlockReduction3D2D for hyperplane reductions on top of which you would then call an integral functor with an appropriate norm in order to aggregate the values).

[jflorezgi]

Thank you Adrian, now I am accessing to my data through BlockReduction3D2D in any of planes that I need regardless of the number of blocks.

Best wishes.

Jonathan

[Author]

Posts