Skip to content


PICTURE project pipelines are published on our GitLab page.

Our automatic segmentation networks, and XNAT tools are currently publicly available.

PICTURE scientific publications


Hugh Pemberton & Jiaming Wu et al.
Scientific Reports13, 18911

This study tests the generalisability of three Brain Tumor Segmentation (BraTS) challenge models using a multi-center dataset of varying image quality and incomplete MRI datasets. In this retrospective study, DeepMedic, no-new-Unet (nn-Unet), and NVIDIA-net (nv-Net) were trained and tested using manual segmentations from preoperative MRI of glioblastoma (GBM) and low-grade gliomas (LGG) from the BraTS 2021 dataset (1251 in total), in addition to 275 GBM and 205 LGG acquired clinically across 12 hospitals worldwide. Data was split into 80% training, 5% validation, and 15% internal test data. An additional external test-set of 158 GBM and 69 LGG was used to assess generalisability to other hospitals’ data. All models’ median Dice similarity coefficient (DSC) for both test sets were within, or higher than, previously reported human inter-rater agreement (range of 0.74–0.85). For both test sets, nn-Unet achieved the highest DSC (internal = 0.86, external = 0.93) and the lowest Hausdorff distances (10.07, 13.87 mm, respectively) for all tumor classes (p < 0.001). By applying Sparsified training, missing MRI sequences did not statistically affect the performance. nn-Unet achieves accurate segmentations in clinical settings even in the presence of incomplete MRI datasets. This facilitates future clinical adoption of automated glioma segmentation, which could help inform treatment planning and glioma monitoring.

Ragnhild Holden Helland & Alexandros Ferles et al.
Scientific Reports13, 18897

Extent of resection after surgery is one of the main prognostic factors for patients diagnosed with glioblastoma. To achieve this, accurate segmentation and classification of residual tumor from post-operative MR images is essential. The current standard method for estimating it is subject to high inter- and intra-rater variability, and an automated method for segmentation of residual tumor in early post-operative MRI could lead to a more accurate estimation of extent of resection. In this study, two state-of-the-art neural network architectures for pre-operative segmentation were trained for the task. The models were extensively validated on a multicenter dataset with nearly 1000 patients, from 12 hospitals in Europe and the United States. The best performance achieved was a 61% Dice score, and the best classification performance was about 80% balanced accuracy, with a demonstrated ability to generalize across hospitals. In addition, the segmentation performance of the best models was on par with human expert raters. The predicted segmentations can be used to accurately classify the patients into those with residual tumor, and those with gross total resection.


David Bouget et al.
Cancers13(18), 4674

Neurosurgical decisions for patients with glioblastoma depend on visual inspection of a preoperative MR scan to determine the tumor characteristics. To avoid subjective estimates and manual tumor delineation, automatic methods and standard reporting are necessary. We compared and extensively assessed the performances of two deep learning architectures on the task of automatic tumor segmentation. A total of 1887 patients from 14 institutions, manually delineated by a human rater, were compared to automated segmentations generated by neural networks. The automated segmentations were in excellent agreement with the manual segmentations, and external validity, as well as generalizability were demonstrated. Together with automatic tumor feature computation and standardized reporting, our Glioblastoma Surgery Imaging Reporting And Data System (GSI-RADS) exhibited the potential for more accurate data-driven clinical decisions. The trained models and software are open-source and open-access, enabling comparisons among surgical cohorts, multicenter trials, and patient registries.

Domenique Müller et al.
Journal of Neurosurgery136(1), 45-55

The aim of glioblastoma surgery is to maximize the extent of resection while preserving functional integrity. Standards are lacking for surgical decision-making, and previous studies indicate treatment variations. These shortcomings reflect the need to evaluate larger populations from different care teams. In this study, the authors used probability maps to quantify and compare surgical decision-making throughout the brain by 12 neurosurgical teams for patients with glioblastoma.

Ivar Kommers & David Bouget , et al.
 202113(12), 2854

Neurosurgical decisions for patients with glioblastoma depend on tumor characteristics in the preoperative MR scan. Currently, this is based on subjective estimates or manual tumor delineation in the absence of a standard for reporting. We compared tumor features of 1596 patients from 13 institutions extracted from manual segmentations by a human rater and from automated segmentations generated by a machine learning model. The automated segmentations were in excellent agreement with manual segmentations and are practically equivalent regarding tumor features that are potentially relevant for neurosurgical purposes. Standard reports can be generated by open access software, enabling comparison between surgical cohorts, multicenter trials, and patient registries.

Domenique Müller et al.
Neuro-oncology advances3(1)

The impact of time-to-surgery on clinical outcome for patients with glioblastoma has not been determined. Any delay in treatment is perceived as detrimental, but guidelines do not specify acceptable timings. In this study, we relate the time to glioblastoma surgery with the extent of resection and residual tumor volume, performance change, and survival, and we explore the identification of patients for urgent surgery.

Ivar Kommers et al.
Journal of neuro-oncology152(2), 289-298

For decisions on glioblastoma surgery, the risk of complications and decline in performance is decisive. In this study, we determine the rate of complications and performance decline after resections and biopsies in a national quality registry, their risk factors and the risk-standardized variation between institutions.


Roelant Eijgelaar & Martin Visser, et al.
Radiology: Artificial Intelligence, 2(5), e190103

Robust deep learning–based segmentation of glioblastoma on routine clinical data can be achieved using a large heterogeneous training dataset or using sparsified training on a combination of public and site-specific data.

Martin Visser, et al.
Frontiers in neuroscience14, 585

To summarize the distribution of glioma location within a patient population, registration of individual MR images to anatomical reference space is required. In this study, we quantified the accuracy of MR image registration to anatomical reference space with linear and non-linear transformations using estimated tumor targets of glioblastoma and lower-grade glioma, and anatomical landmarks at pre- and post-operative time-points using six commonly used registration packages (FSL, SPM5, DARTEL, ANTs, Elastix, and NiftyReg).

Domenique Müller et al.
Journal of Neurosurgery134(4), 1091-1101

Decisions in glioblastoma surgery are often guided by presumed eloquence of the tumor location. The authors introduce the “expected residual tumor volume” (eRV) and the “expected resectability index” (eRI) based on previous decisions aggregated in resection probability maps. The diagnostic accuracy of eRV and eRI to predict biopsy decisions, resectability, functional outcome, and survival was determined.


Roelant Eijgelaar, et al.
PLoS One14(9), e0222939

During resections of brain tumors, neurosurgeons have to weigh the risk between residual tumor and damage to brain functions. Different perspectives on these risks result in practice variation. We present statistical methods to localize differences in extent of resection between institutions which should enable to reveal brain regions affected by such practice variation.

Esther Habets, et al.
Journal of Neuro-Oncology, 144(3), 573-582

Patients with diffuse glioma often experience neurocognitive impairment already prior to surgery. Pertinent information on whether damage to a specific brain region due to tumor activity results in neurocognitive impairment or not, is relevant in clinical decision-making, and at the same time renders unique information on brain lesion location and functioning relationships. To examine the impact of tumor location on preoperative neurocognitive functioning (NCF), we performed MRI based lesion-symptom mapping.

Eef Hendriks, et al.
Neurosurgery85(2), E304-E313

Preoperative interpretation of resectability of diffuse nonenhancing glioma is primarily based on individual surgical expertise. In this study, we compare the accuracy and precision between observed resections and preoperative estimates of neurosurgeons and a resection probability map (RPM). We hypothesize that the RPM estimates is as good as senior neurosurgeons.

Martin Visser, et al.
NeuroImage: Clinical22, 101727

Tumor segmentation of glioma on MRI is a technique to monitor, quantify and report disease progression. Manual MRI segmentation is the gold standard but very labor intensive. At present the quality of this gold standard is not known for different stages of the disease, and prior work has mainly focused on treatment-naive glioblastoma. In this paper we studied the inter-rater agreement of manual MRI segmentation of glioblastoma and WHO grade II-III glioma for novices and experts at three stages of disease. We also studied the impact of inter-observer variation on extent of resection and growth rate.

Domenique Müller et al.
JCO clinical cancer informatics2, 1-12

The aim of glioblastoma surgery is to maximize the extent of resection while preserving functional integrity, which depends on the location within the brain. A standard to compare these decisions is lacking. We present a volumetric voxel-wise method for direct comparison between two multidisciplinary teams of glioblastoma surgery decisions throughout the brain.


Roelant Eijgelaar, et al.
Journal of Neuro-oncology139(3), 591-598

Detection of glioblastoma progression is important for clinical decision-making on cessation or initiation of therapy, for enrollment in clinical trials, and for response measurement in time and location. The RANO-criteria are considered standard for the timing of progression. To evaluate local treatment, we aim to find the most accurate progression location. We determined the differences in progression free survival (PFS) and in tumor volumes at progression (Vprog) by three definitions of progression.

Eef Hendriks, et al.
Human brain mapping39(5), 2064-2074

Patients with a diffuse glioma may experience cognitive decline or improvement upon resective surgery. To examine the impact of glioma location, cognitive alteration after glioma surgery was quantified and related to voxel-based resection probability maps.


Philip de Witt Hamer, et al.
PloS one8(9), e73353

Intraoperative brain stimulation mapping reduces permanent postoperative deficits and extends tumor removal in resective surgery for glioma patients. Successful functional mapping is assumed to depend on the surgical team’s expertise. In this study, glioma resection results are quantified and compared using a novel approach, so-called resection probability maps (RPM), exemplified by a surgical team comparison, here with long and short experience in mapping.

PhD Theses

Domenique Müller

This thesis set out to answer the question whether probability maps can be used to evaluate surgical decision-making in glioblastoma.

Roelant Eijgelaar.

This thesis focuses on incorporating brain location information in the evaluation of glioblastoma treatment, by addressing the feasibility and showing the value of large-scale data collection and localized treatment evaluation for glioblastoma patients using routine clinical data.

Martin Visser

The aim of this thesis was to assess the accuracy of image segmentation and registration, since these are necessary tools for generating a probability map.

Do you want to know more about the
project and the research team?

Get in touch with us and leave a message. We will get back to you as soon as possible.
Or do you just want to support us? Feel free to ask about the possibilities.