Reply To: Modifying field values for neighboring cells inside a postprocessor
Dear Adrian,
I believe I’ve identified the issue with the CCL implementation when MPI is enabled. Thank you for your insights so far in working toward a robust solution. Since you’ve already reviewed the code, you’re familiar with how cell IDs are used to assign bubble IDs and to navigate between cells in each kernel to access their bubble IDs.
I’ll do my best to clearly explain the problem. The CCL algorithm assumes that cell IDs follow a consistent order in each direction within a block, which works fine for a single block. However, when MPI is enabled and the domain is divided into multiple blocks, this consistency breaks down for neighboring cells located in different blocks. As a result, the bubble ID assignment becomes incorrect for blocks other than Block 0. In the worst-case scenario, this can lead to illegal memory access, especially since blocks may have an unequal number of cells, see the below picture.
When the orange cell in Block #2 is being processed, it accesses the bubble ID of the red cell, as the red cell is a valid neighbor. However, since the red cell belongs to a different block, its cell ID does not follow the block-local order of cell IDs in Block #2. When the orange cell adopts the bubble ID of the red cell, it may incorrectly reference a cell within Block #2, as indicated by the blue arrow. This can result in either assigning an incorrect bubble ID to the orange cell or, in the worst-case scenario, causing illegal memory access. This happens because Block #2 may not contain a cell with the cell ID from Block #0.
Is it possible to defer the automatic communication between blocks for a specific global data structure until it is explicitly triggered when needed?
Regards,
Danial