Fully Automated Radiotherapy Treatment Planning Challenge Banner
Join

Fully Automated Radiotherapy Treatment Planning Challenge

Motivation

Some aspects of radiotherapy treatment delivery, such as acquiring the imaging and positioning the patient require human intervention, but a large part of the planning is computer-based and is well placed for automation. As the radiation oncology pushes towards adaptive, even daily online adaptive, treatments, Automation of this planning is becoming more and more important. Already, many systems automated the planning processes to a greater or lesser extent. This challenge seeks to take a snapshot and address the question "How close are we to fully automated radiotherapy treatment planning?".

Workshop

Please note this challenge will form part of the ESTRO Physics workshop. Participants should request to participate in this workshop, mentioning their intent to participate in the challenge, by 3 July 2023. You may still participate in the online challenge without attending the workshop, by workshop attendance is strong encouraged, since the greatest benefit comes from discussing the methods and results as a community.

Task

Participants will be provided with simulation CTs for ten prostate cancer patients, together with a treatment intent/prescription (in a machine readable format). The cases will be a mix of prostate only and prostate + nodes. Participants are asked to generate a treatment plan in an as-automated-as-possible way, including contouring and plan generation. No manual intervention on contouring or planning should be performed, but manual steps to transfer data between systems are permitted if required. Freedom is given to participants with respect to the "treatment machine" the plan is designed for. However, it is expected that all participants produce a plan that is deliverable in clinically reasonable time frame.

Participants are asked to submit the contours, treatment plan, and dose volume, all in DICOM format (RTSTRUCT, RTPLAN, RTDOSE) for each case. Participants will also be asked to complete a questionnaire about their method(s) and the degree of automation achieved.

Following the web-based challenge, a live contest will be conducted at the ESTRO 2023 Physics workshop, enabling participants to demonstrate their systems and discuss the challenge and its results

Dataset

Since the aim of the challenge is to evaluate the current state-of-the-art to generate plans automatically to a prescription, rather than to mimic a particular training set, no training data will be provide. This reflects a use-case of deploying the tool/process as an "off-the-shelf" technology in a new clinic.

Examples of each type of case will be provided to enable participants to verify they can load the input data, and test that they can successfully upload results in the correct format. However, no "ground-truth" will be provided for these cases.

Evaluation

The primary evaluation will be based achieving planning objectives and constraints. See the Scoring page for full details. Additional measures (such as contour added path length) will be reported by not scored.

If it proves practical, a secondary winner will be selected as the “experts favourite” based on a clinical assessment by a a panel of experts. The panel will be asked to visually assess the plans to determine if the plans are clinically acceptable and to choose their favourites.

Scoring Code

The scoring code has been made available on Github. The "ground truth" is not available with the code. The code  is so that participants can understand the details to the scoring (and point out any bugs they perceive).  Apologies of the lack of commenting/documentation!

References

Below are a selection of general review references as a starting point for those interested in this challenge;