Domain Generalization with Anti-background Perturbation Consistency and Texture Reduction Ensemble Models for Hepatocyte Nucleus Segmentation

Hepatocyte nucleus segmentation in histopathology images is vital for diagnostics. However, varying slide background due to cutting and staining poses a great challenge for domain-agnostic segmentation, which limits model generalization. This paper first proposes an anti-background perturbation consistency (APC) loss to mitigate the influence of image background on model decisions and controlled background perturbations to enhance generalizability, while still maintaining feature consistency. Since convolutional neural networks (CNNs) often prioritize local texture over global shape which also limits generalization, we then introduce the concept of local self-information into the texture probability (TP) loss to reduce over-focus of CNN on local textures. To avoid model convergence to saddle points during training which yields varying outcomes and unstable performance, we finally conclude with a meta-learner which combines results from multiple models to improve stability and better decision-making. At the end, our developed APC coupled with the texture reduction ensemble model (TREM) effectively increases model generalizability across diverse data without fine-tuning model parameters. The code of this study are available at https://github.com/s07362022/ATE.