Task
The segmentation of gray matter, white matter, cerebrospinal fluid, and other structures on multi-sequence brain MR images with and without (large) pathologies.
Background
Many algorithms for segmenting brain structures in MRI scans have been proposed over the years. Especially in such a well-established research area, there is a tremendous need for fair comparison of these methods with respect to accuracy and robustness. Although there is an increasing awareness of the importance of comparing different algorithms on the same data, many methods are still compared to previous versions of the same type of algorithm on privately held data. This complicates the choice for a certain brain segmentation method among a wide variety of available methods.
This challenge aims to directly compare automated brain segmentation methods. The output will be a ranking of techniques that robustly and accurately segment brain structure on MR brain images, both with and without pathology. We welcome both multi- and single-sequence (i.e. T1-weighted only) approaches.
Method
Participants will containerize their algorithms with Docker and submit these to the organizers. Detailed instructions and easy-to-follow examples are provided and, if needed, the organizers will help with containerization. The organizers will run the techniques on the test data. This guarantees that the test data remains secret and cannot be included in the training procedure of the techniques. This workflow has proven successful for previous MICCAI challenges.
Results
The organizers will evaluate all methods submitted before the deadline, according to the evaluation criteria. The results will be presented during the challenge session at the MICCAI, on 16 September 2018. During this challenge session, participating teams should present their method by means of an oral pitch presentation and a poster.
Journal publication
Results of the challenge will be summarized in a journal paper, to be submitted after the MICCAI. All participating teams who submitted before the challenge deadline and presented their method at the challenge session at MICCAI will be included in the paper. Each team is allowed two co-authorships on the paper.
Future submissions
The challenge will remain open after the conference deadline. New methods can be submitted and join the challenge. Results as presented at the MICCAI will be frozen and presented separately on this website. Next, there will be an up-to-date ranking available on this website. The container / Docker infrastructure guarantees standardized testing in the coming years, and is supported by the UMC Utrecht. The main organizer and the UMC Utrecht agree to commit themselves to future support of this challenge.
Terms of participation
The Grand Challenge on MR Brain Segmentation is organized in the spirit of cooperative scientific progress. We do not claim any ownership or right to the methods, but we require anyone to respect the rules below. The following rules apply to those who register a team and/or download the data:
- The downloaded data sets, associated reference standard, or any data derived from these data sets, may not be given or redistributed under any circumstances to persons not belonging to the registered team.
- All information entered when registering a team, including the name of the contact person, the affiliation (institute, organization or company the team’s contact person works for) and the e-mail address must be complete and correct. Anonymous or incomplete registration is not allowed. If you wish to submit anonymously, for example because you want to submit your results to a journal or conference that requires anonymous submission, please contact the organizers first.
- The data provided may only be used for preparing an entry to be submitted to this challenge. The data may not be used for other purposes in scientific studies and may not be used to train or develop other algorithms, including but not limited to algorithms used in commercial products, without prior participation in the challenge and approval by the organizers.
- Results of your submission will only be published on the website when a document describing the method is provided.
- If a commercial system is evaluated no method description is necessary, but the system has to be publicly available and the exact name and version number have to be provided.
- The organizers of the challenge will check the method description before your results will be published on the website.
- If the results of algorithms in this challenge are to be used in scientific publications (e.g. journal publications, conference papers, technical reports, presentations at conferences and meetings) you must make an appropriate citation. Currently, this citation will refer to the this website (https://mrbrains18.isi.uu.nl/), and later to the publication that will describe the results of the Grand Challenge on MR Brain Segmentation.
- Evaluation of registration results uploaded to this website will be made publicly available on this website (Results section), and by submitting results, you grant us permission to publish our evaluation. Participating teams maintain full ownership and rights to their method.
- Teams must notify the organizers of this challenge about any publication that is (partly) based on the results data published on this website, in order for us to maintain a list of publications associated with the challenge.