Pay Attention with Focus: A Novel Learning Scheme for Classification of Whole Slide Images

Deep learning methods such as convolutional neural networks (CNNs) are difficult to directly utilize analyze whole slide images (WSIs) due the large image dimensions. We overcome this limitation by proposing a novel two-stage approach. First, we extract set of representative patches (called mosaic) from WSI. Each patch mosaic is encoded feature vector using deep network. The extractor model fine-tuned hierarchical target labels WSIs, i.e., anatomic site and primary diagnosis. In second stage, patch-level features WSI used compute diagnosis probability through proposed Pay Attention with Focus scheme, an attention-weighted averaging predicted probabilities for all modulated trainable focal factor. Experimental results show that can be robust, effective classification WSIs.