This is an overview and selection of recent research outputs. A more extensive presentation of previous research projects can be found here.

 

Localization of Anatomical Landmarks

 

We often require the robust and accurate localization of anatomical landmarks as a pre-processing step in medical image analysis, e.g. to roughly extract relevant structures for further processing (classifying disease, regressing age, …).

Integrating spatial configuration into heatmap regression based CNNs for landmark localization

In many medical image analysis applications, only a limited amount of training data is available due to the costs of image acquisition and the large manual annotation effort required from experts. Training recent state-of-the-art machine learning methods like convolutional neural networks (CNNs) from small datasets is a challenging task. In this work on anatomical landmark localization, we propose a CNN architecture that learns to split the localization task into two simpler sub-problems, reducing the overall need for large training datasets. Our fully convolutional SpatialConfiguration-Net (SCN) learns this simplification due to multiplying the heatmap predictions of its two components and by training the network in an end-to-end manner.

Payer C, Stern D, Bischof H, Urschler M: Published in Medical Image Analysis 54:207-219, 2019. DOI (open access)

 

Integrating geometric configuration and appearance information into a unified framework for anatomical landmark localization

In approaches for automatic localization of multiple anatomical landmarks, disambiguation of locally similar structures as obtained by locally accurate candidate generation is often performed by solely including high level knowledge about geometric landmark configuration. In our novel localization approach, we propose to combine both image appearance information and geometric landmark configuration into a unified random forest framework integrated into an optimization procedure that iteratively refines joint landmark predictions by using the coordinate descent algorithm. Depending on how strong multiple landmarks are correlated in a specific localization task, this integration has the benefit that it remains flexible in deciding whether appearance information or the geometric configuration of multiple landmarks is the stronger cue for solving a localization problem both accurately and robustly.

Urschler M, Ebner T, Stern D: Published in Medical Image Analysis 43:23-36, 2018. DOI (open access), BAA Web X-Ray Landmark annotations

 

Segmentation and Detection

 

To derive quantitative details on particular organs of interest, or their substructures, like e.g. its volume, automatic segmentation of structures is required, sometimes preceded by a coarse detection step.

 

Coarse to Fine Vertebrae Localization and Segmentation with SpatialConfiguration-Net and U-Net

Localization and segmentation of vertebral bodies from spine CT volumes are crucial for pathological diagnosis, surgical planning, and postoperative assessment. However, fully automatic analysis of spine CT volumes is difficult due to the anatomical variation of pathologies, noise caused by screws and implants, and the large range of different field-of-views. We propose a fully automatic coarse to fine approach for vertebrae localization and segmentation based on fully convolutional CNNs.

Payer C, Stern D, Bischof H, Urschler M: Published in Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020). DOI, PDF

 

Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks

Differently to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same object class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time, which is highly relevant, e.g., in biomedical applications involving cell growth and migration. Our network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal information, e.g., from microscopy videos. Moreover, we train our network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos, even in the presence of dynamic structural changes due to mitosis of cells.

Payer C, Stern D, Bischof H, Urschler M: Published in Medical Image Analysis 57:106-119, 2019. DOI (open access)

 

Matwo-CapsNet: A Multi-Label Semantic Segmentation Capsules Network

CNNs have been largely adopted by the computer vision community due to their efficacy and versatility. Introduced by Sabour et al. to circumvent some limitations of CNNs, capsules replace scalars with vectors to encode appearance feature representation, allowing better preservation of spatial relationships between whole objects and its parts. In this work, we introduce several improvements to the capsules framework, allowing it to be applied for multi-label semantic segmentation.

Bonheur S, Stern D, Payer C, Pienn M, Olschewski H, Urschler M: Presented at International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2019, Shenzhen, China. DOI

 

Vertebrae Localization and Segmentation with SpatialConfiguration-Net and U-Net

This technical report introduces our proposed pipeline for fully automatic vertebrae localization and segmentation in CT volumes for the VerSe 2019 Large Scale Vertebrae Segmentation Challenge. The challenge consists of two tasks, where the first one is to localize and label the centers of the individual vertebrae, and the second one is vertebrae segmentation.

Payer C, Stern D, Bischof H, Urschler M: Presented at MICCAI VerSe Grand Challenge 2019, Shenzhen, China. Ext Abstract PDF, Presentation PDF

 

Generative Adversarial Networks to Synthetically Augment Data for Deep Learning based Image Segmentation

In recent years, deep learning based methods achieved state-of-the-art performance in many computer vision tasks. However, these methods are typically supervised, and require large amounts of annotated data to train. Acquisition of annotated data can be a costly endeavor, especially for methods requiring pixel-wise annotations such as image segmentation. In this work we evaluate if GAN-based data augmentation using state-of-the-art methods, such as the Wasserstein GAN with gradient penalty, is a viable strategy for data augmentation in the context of small datasets.

Neff T, Payer C, Stern D, Urschler M: Published in Proceedings of the OAGM Workshop 2018, Hall/Tyrol, Austria, pp. 22-29, 2017. DOI

 

Multi-Label Whole Heart Segmentation Using CNNs and Anatomical Label Configurations

We propose a pipeline of two fully convolutional networks for automatic multi-label whole heart segmentation from CT and MRI volumes. Atfirst, a convolutional neural network (CNN) localizes the center of the bounding box around all heart structures, such that the subsequent segmentation CNN can focus on this region. Trained in an end-to-end manner, the segmentation CNN transforms intermediate label predictions to positions of other labels. Thus, the network learns from the relative positions among labels and focuses on anatomically feasible configurations.

Payer C, Stern D, Bischof H, Urschler M: In Proceedings of MICCAI STACOM Workshop 2017, Quebec City, Canada, 2017. DOI (Winner of the MM-WHS Challenge and receiver of the Best Paper Award)

 

Segmentation and Classification of Colon Glands with Deep Convolutional Neural Networks and Total Variation Regularization

Segmentation of histopathology sections is an ubiquitous requirement in digital pathology and due to the large variability of biological tissue, machine learning techniques have shown superior performance over standard image processing methods.

Kainz P, Pfeiffer M, Urschler M: Published in PeerJ 5:e3874, 2017. DOI

 

You Should Use Regression to Detect Cells

Automated cell detection in histopathology images is a hard problem due to the large variance of cell shape and appearance. We show that cells can be detected reliably in images by predicting, for each pixel location, a monotonous function of the distance to the center of the closest cell. Cell centers can then be identified by extracting local extremums of the predicted values.

Kainz P, Urschler M, Wohlhart P, Schulter S, Lepetit V: Presented at International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2015, Munich, Germany. DOI

 

Automatic Intervertebral Disc Localization and Segmentation in 3D MR Images Based on Regression Forests and Active Contours

We introduce a fully automatic localization and segmentation pipeline for three-dimensional intervertebral discs (IVDs), consisting of a regression-based prediction of vertebral bodies and IVD positions as well as a 3D geodesic active contour segmentation delineating the IVDs.

Urschler M, Hammernik K, Ebner T, Stern D: Presented at MICCAI Workshop Computational Spine Imaging (CSI) 2015, Munich, Germany. DOI

 

Forensic Age Estimation

 

In recent years we have developed an automated method for predicting chronological age of children and adolescents in the age range between 13 and 24 years based on MRI data of their hand, wisdom teeth and clavicle bones.

 

Automatic Age Estimation and Majority Age Classification from Multi-factorial MRI Data

Age estimation from radiologic data is an important topic both in clinical medicine as well as in forensic applications, where it is used to assess unknown chronological age or to discriminate minors from adults. In this work, we propose an automatic multi-factorial age estimation method based on MRI data of hand, clavicle and teeth to extend the maximal age range from up to 19 years to up to 25 years. Fusing age-relevant information from all three anatomical sites, our method utilizes a deep convolutional neural network that is trained on a dataset of 322 subjects in the age range between 13 and 25 years, to achieve a mean absolute prediction error in regressing chronological age of 1.01+/-0.74 years.

Stern D, Payer C, Giuliani N, Urschler M: Published in IEEE Journal of Biomedical and Health Informatics, 23(4):1392-1403, 2019. DOI (open access)

 

Automated age estimation from MRI volumes of the hand

Highly relevant for both clinical and legal medicine applications, the established radiological methods for estimating unknown age in children and adolescents are based on visual examination of bone ossification in X-ray images of the hand. We developed a fully automatic age estimation methods from 3D MRI scans of the hand, in order to simultaneously overcome the problems of the radiological methods including (1) exposure to ionizing radiation, (2) necessity to define new, MRI specific staging systems, and (3) subjective influence of the examiner. Trained on a large dataset of 328 MR images including ground truth bone age, we obtain a mean absolute error of 0.37 ±0.51 years for the age range of the subjects ≤18 years, i.e. where bone ossification has not yet saturated.

Stern D, Payer C, Urschler M: Published in Medical Image Analysis, 58:101538, 2019. DOI (open access)

 

Thoracic Image Analysis

 

The morphological analysis of thoracic CT images has high potential for disease prediction related to the lung. We perform research on vessel segmentation and artery vein separation in pulmonary hypertension together with the Ludwig Boltzmann Institute for Lung Vascular Research in Graz, Austria.

 

Pulmonary Lobe Segmentation in CT Images using Alpha-Expansion

Fully-automatic lung lobe segmentation in pathological lungs is still a challenging task. A new approach for automatic lung lobe segmentation is presented based on airways, vessels, fissures and prior knowledge on lobar shape. The anatomical information and prior knowledge are combined into an energy equation, which is minimized via graph cuts to yield an optimal segmentation. The algorithm is quantitatively validated on an in-house dataset of 25 scans and on the LObe and Lung Analysis 2011 (LOLA11) dataset, which contains a range of different challenging lungs (total of 55) with respect to lobe segmentation.

Giuliani N, Payer C, Pienn M, Olschewski H, Urschler M: In Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) – Volume 4: VISAPP, Funchal – Madeira, Portugal, pages 387-394, Jan 2018. DOI

 

Automated Integer Programming Based Separation of Arteries and Veins from Thoracic CT Images

Automated computer-aided analysis of lung vessels has shown to yield promising results for non-invasive diagnosis of lung diseases. To detect vascular changes which affect pulmonary arteries and veins differently, both compartments need to be identified. We present a novel, fully automatic method that separates arteries and veins in thoracic computed tomography images, by combining local as well as global properties of pulmonary vessels.

Payer C, Pienn M, Balint Z, Shekhovtsov A, Talakic E, Nagy E, Olschewski A, Olschewski H, Urschler M: Published in Medical Image Analysis 34: 109-122, 2016. DOI