0 avis
Topology-aware activation layer for neural network image segmentation
Archive ouverte
International audience. One of the recent developments in deep learning is the ability to train extremely deep residual neural networks, knowing that each residual block produces only marginal changes to the data. The accumulation of these changes result in the network's improved performance, analogous to a complex but trainable iterative algorithm. This intuition can be merged with the underlying theory of probabilistic graphical models in which these iterative algorithms are common and share the underlying probabilistic and information theoretic basis as deep learning. Prior models have been proposed with limitations on the number of iterations allowed in the solution algorithm due to the linear memory growth during the training process. This paper presents a structured activation layer which implements a conditional random field along with an arbitrary iteration message-passing marginal probability estimation algorithm which requires constant, rather than linear, memory with respect to the number of iterations. In this activation layer, the segmentation labels can be specified hierarchically, incorporating a level of abstract structure and some basic geometrical knowledge directly and easily into the network. Thus, this layer allows for the separation of abstract knowledge brought in by the network designer (in the form of the hierarchical structure) from probabilistic priors learned by the neural network. Preliminary results comparing this activation function to softmax and a similar non-hierarchical activation function indicate that it significantly improves performance in segmentation problems. © 2020 SPIE. All rights reserved.
Consulter en ligne
Suggestions
Du même auteur
