American Association of Physicists in Medicine Radiation Therapy Committee Task Group 53: Quality assurance for clinical radiotherapy treatment planning

In recent years, the sophistication and complexity of clinical treatment planning and treatment planning systems has increased significantly, particularly including three-dimensional (3D) treatment planning systems, and the use of conformal treatment planning and delivery techniques. This has led to the need for a comprehensive set of quality assurance (QA) guidelines that can be applied to clinical treatment planning. This document is the report of Task Group 53 of the Radiation Therapy Committee of the American Association of Physicists in Medicine. The purpose of this report is to guide and assist the clinical medical physicist in developing and implementing a comprehensive but viable program of quality assurance for modern radiotherapy treatment planning. The scope of the QA needs for treatment planning is quite broad, encompassing image-based definition of patient anatomy, 3D beam descriptions for complex beams including multileaf collimator apertures, 3D dose calculation algorithms, and complex plan evaluation tools including dose volume histograms. The Task Group recommends an organizational framework for the task of creating a QA program which is individualized to the needs of each institution and addresses the issues of acceptance testing, commissioning the planning system and planning process, routine quality assurance, and ongoing QA of the planning process. This report, while not prescribing specific QA tests, provides the framework and guidance to allow radiation oncology physicists to design comprehensive and practical treatment planning QA programs for their clinics.

In recent years, the sophistication and complexity of clinical treatment planning and treatment planning systems has increased significantly, particularly including three-dimensional ͑3D͒ treatment planning systems, and the use of conformal treatment planning and delivery techniques. This has led to the need for a comprehensive set of quality assurance ͑QA͒ guidelines that can be applied to clinical treatment planning. This document is the report of Task Group 53 of the Radiation Therapy Committee of the American Association of Physicists in Medicine. The purpose of this report is to guide and assist the clinical medical physicist in developing and implementing a comprehensive but viable program of quality assurance for modern radiotherapy treatment planning. The scope of the QA needs for treatment planning is quite broad, encompassing image-based definition of patient anatomy, 3D beam descriptions for complex beams including multileaf collimator apertures, 3D dose calculation algorithms, and complex plan evaluation tools including dose volume histograms. The Task Group recommends an organizational framework for the task of creating a QA program which is individualized to the needs of each institution and addresses the issues of acceptance testing, commissioning the planning system and planning process, routine quality assurance, and ongoing QA of the planning process. This report, while not prescribing specific QA tests, provides the framework and guidance to allow radiation oncology physicists to design comprehensive and practical treatment planning QA programs for their clinics. © 1998 American Association of Physicists in Medicine. ͓S0094-2405͑98͒03410-5͔ Key words: treatment planning, quality assurance, 3D treatment planning

PREFACE
This document is the report of Task Group 53 of the Radiation Therapy Committee of the American Association of Physicists in Medicine ͑AAPM͒. The purpose of this report is to guide and assist the radiation oncology physicist in developing and implementing a comprehensive but viable program of quality assurance for radiotherapy treatment planning. This report is the first guidance on the topic of treatment planning quality assurance ͑QA͒ from the AAPM, although there are several related reports, 1 including the recent report from Task Group 40 on Comprehensive QA for Radiation Oncology. 2 Further expansion of AAPM recommendations regarding treatment planning quality assurance is likely after the radiation oncology community accumulates some experience with the approach recommended in this report.
In recent years, the increased complexity of the treatment planning process required to support such procedures as conformal radiotherapy has led to the need for a comprehensive set of quality assurance guidelines that can be applied to treatment planning systems that support this complex process. This Task Group has been charged by the AAPM to prepare this report recommending the scope and content of necessary quality assurance procedures and the frequency of tests, from acceptance testing, characterization and commissioning to routine quality assurance of clinical system use.
These procedures will be tailored to the complexity and functionality of the treatment planning procedures used clinically. This report provides the overall framework within which individualized quality assurance programs may be designed and implemented.
This report on treatment planning quality assurance attempts to aid the radiation oncology physicist in creating a quality assurance program for the clinical use of treatment planning in the physicist's department. In general, except for recommendations summarized in one appendix, this report does not discuss quality assurance activities that should be carried out by vendors or other providers of treatment planning systems. The numerous important quality assurance tasks associated with the design, software engineering, testing, validation, packaging, marketing, and other preparation of a commercial treatment planning system for safe use are beyond the scope of the current task group. This document considers only the responsibility of the radiation oncology physicist in establishing and maintaining a quality assurance program for the clinical use of radiotherapy treatment planning.
The report also concentrates on quality assurance for the treatment planning process, and not just QA or commissioning of the treatment planning system. Although a treatment planning system ͑software and hardware͒ may be tested extensively, a QA program for treatment planning must also consider how the treatment planning system is used as well as how it interacts with the treatment planning process. Therefore, creation of a treatment planning process that incorporates self-consistency and procedural checks is a major component of a quality assurance program for treatment planning.
In order to successfully implement an appropriate quality assurance program for treatment planning, adequate resources must be allocated. The radiation oncology physicist must be afforded adequate time to ascertain the extent and complexity of the treatment planning needs of the radiation oncology clinic, and based upon this information, the physicist must design and implement an appropriate quality assurance program. For a treatment planning process of a given complexity, the quality assurance requirements in a small radiation oncology facility should be no less than those in a large, academic medical center.
The report begins with a summary intended for radiation oncology administrators ͑Part A͒. Part B is directed to the radiation oncology physicist, and comprises the bulk of the report. Part B begins with an introduction which delineates the scope of the task, introduces some definitions and terms, and establishes targets for the accuracy of treatment planning results. Chapter 2 describes specifications and acceptance testing for the treatment planning system. The most extensive part of the report is contained in Chaps. 3 and 4, which describe commissioning of the nondosimetric and dosimetric parts of the planning system, respectively. Routine testing of the treatment planning system is described in Chap. 5. Chapter 6 discusses ways to apply QA to the entire planning process, while Chap. 7 lists computer-system management activities which are an important part of the treatment planning quality assurance process. Finally, the last chapter summarizes some of the important recommendations of the task group. Appendix 1 contains some recommendations and comments about both vendor and user responsibilities. Appendix 2 contains examples of some nondosimetric test procedures, to give the reader an idea of how to design and implement test procedures. Appendices 3, 4, and 5 give examples of dose calculation commissioning tests for photon beams, electron beams, and brachytherapy, respectively.
Terminology used in this report will be similar to that used in other AAPM task group reports: • Shall or must are used when the activity is required by various regulatory agencies. • Recommend is used when the task group expects that the procedure should normally be followed as described. However, there may prove to be instances where other issues, techniques or priorities could force the modification of the recommendation of the task group. • Should is used when it is expected that local analysis of the situation may change the way a particular activity is performed. This report recommends the institution of a comprehensive quality assurance program for treatment planning in each radiation oncology clinic. As will be seen, this encompasses a large amount of work, requiring the attention particularly of the radiation oncology physicist, but also including dosimetrists/treatment planners, radiation oncologists, radiation therapists and, if available, computer support staff. Particularly at this time of downsizing and major restructuring of the way the practice of clinical medicine works, it is very important for hospital administrators and providers of medical care reimbursement to understand the critical nature of appropriate quality assurance for a procedure that is such an important part of the way high quality radiotherapy is performed. If compromises must be made in the interest of cost reduction, these compromises should be made initially in establishing the complexity and efficiency of the treatment planning process in the clinic. Once a particular type of process has been established, then it is imperative for the safety and well-being of the patient that an appropriate quality assurance program be implemented to support that process. In this report, we have tried to balance the need to be cost effective and efficient with the need for high quality care. As the recommendations of this task group are used throughout the community, it will be important for radiation oncology physicists to improve their quality assurance tools and programs, so that the quality of treatments can be improved while also keeping the costs as low as feasible. The goal of radiotherapy treatment of cancer is to cure or locally control the disease while minimizing complications in normal tissues. The process of treatment planning, inasmuch as it determines the detailed technique used for a patient's radiation treatments, is instrumental in accomplishing that goal. The term ''treatment planning'' has sometimes been narrowly interpreted as a process primarily concerned with dosimetry procedures such as the generation of computerized dose distributions and the calculation of treatment times or monitor unit settings.

OUTLINE
In actuality, treatment planning is a much broader process than just performing dose calculations: it encompasses all of the steps involved in planning a patient's treatment.
• The initial step in the treatment planning process is patient positioning and immobilization, during which an optimum patient position for treatment is determined and immobilization devices necessary to maintain the patient in that position during treatment are constructed.
• Next, the size, extent, and location of the patient's tumor ͑target volume͒, and its relationship with normal organs and external surface anatomy must be determined. This step, often referred to as localization or simulation, requires special equipment ͑e.g., simulators, CT simulators or CT scanners, other imaging studies͒ for imaging the tumor and normal organs and obtaining the shape of the external patient surface. In some ͑but not all͒ cases, the treatment fields are designed or ''simulated'' during this step. Other information must also be incorporated into the planning process, including prior radiation therapy, concurrent chemotherapy, and other radiosensitive conditions. • Only at the completion of these first two procedures can traditional ''treatment planning'' or ''dose planning'' begin. This step in the treatment planning process is performed using a computerized radiation treatment planning system ͑RTP system͒. The RTP system is comprised of computer software, at least one computer workstation which includes a graphical display, input devices for entering patient and treatment machine information, and output devices for obtaining hardcopy printouts for patient treatment and records. The patient anatomical information and any treatment field information obtained during localization and simulation are entered into the RTP system, field design is performed as necessary, the dose distribution within the patient is calculated and optimized by the treatment planner, and the final plan is evaluated by a radiation oncology physicist and approved by the radiation oncologist. • The last step in the treatment planning process, plan verification, involves checking the accuracy of the planned treatment prior to treatment delivery. During this step, the patient may return to the department for additional procedures including a ''plan verification'' simulation or ''setup'' ͑treatment simulation on the treatment machine͒. Additional radiographic images may be taken and treatment information may be transferred from the planning system to other computer systems ͑such as a record and verify system or treatment delivery system͒ so that the plan may be delivered to the patient by the treatment machine.
It should be apparent from this description that the treatment planning process, in its entirety, is a complex series of interwoven procedures involving the efforts of many departmental personnel.
The complexity and sophistication of treatment planning and treatment planning systems has increased tremendously during the past decade. In addition to the software features found in traditional RTP systems, sophisticated options such as three-dimensional ͑3D͒ and beam's eye view ͑BEV͒ displays, digitally reconstructed radiographs ͑DRRs͒, threedimensional dose computations and display, and plan evaluation tools such as dose volume histograms ͑DVHs͒ have become standard on the newest systems. Furthermore, the complexity of the treatment planning process may increase with more complex treatments. For example, electronic portal imaging, multileaf collimators, and computer controlled treatment delivery are all treatment options which offer the potential of improving patient care and treatment delivery efficiency, but also require increased personnel efforts for commissioning and quality assurance at both the treatment delivery and treatment planning levels.
The International Commission on Radiation Units and Measurements 3 recommends that radiation dose be delivered to within 5% of the prescribed dose. This requires that the uncertainty in each individual step in the treatment process ͑including treatment planning͒ be significantly less than the quoted 5%, and is a worthy goal. Unlike small errors in treatment delivery which usually occur on a daily basis and are often random in nature, uncertainties or errors introduced during the treatment planning process are much more likely to be systematic and constant over the entire course of treatment. Therefore, they harbor a huge potential for adversely affecting tumor control and/or normal tissue complications. The need for stringent QA requirements to minimize the possibility of systematic errors-so the ICRU recommendations can be met-is obvious.
While specific goals of a treatment planning QA program include meeting the ICRU dose delivery standards and addressing specific QA issues related to the increased complexity and sophistication in planning and treatment delivery systems, the overall aim should be to improve the care of patients treated with radiation. To meet the goals of a QA program, adequate equipment including treatment and imaging units, computerized treatment planning systems, and radiation measuring devices such as computerized data acquisition systems and phantoms are necessary, along with adequate staffing of all the specialties, including radiation oncologists, radiation oncology physicists, medical radiation dosimetrists, and radiation therapists. It is important to realistically assess the staffing required for a QA program, particularly when new, sophisticated systems are introduced into a department. Clearly, increased treatment planning complexity calls for more, not less, staffing to ensure the systems are used safely and that the complex QA procedures can be practically completed. We therefore concur with the recommendation of the AAPM Radiation Therapy Committee Task Group 40 2 that radiation facilities should be staffed at least at the levels described in the ''Blue Book,'' the Report of the InterSociety Council for Radiation Oncology, 4 since this report does not directly consider the requirements of more modern 3D planning systems.
As discussed in the TG 40 report, 2 the QA program for a radiation oncology department should originate from the departmental QA committee, and the QA program designed for treatment planning should be subject to review and approval by that committee. It is the opinion of this task group, however, that QA for the treatment planning process and for the treatment planning system is primarily the responsibility of the radiation oncology physicist. Nevertheless, the support of other departmental members will be crucial to the success of the program. The responsibilities of various members of the department with regard to comprehensive radiation oncology QA have been outlined by Task Group 40. These recommendations are reproduced below with additional emphasis on the role of each group in treatment planning QA.
Radiation oncologist. Radiation oncologists are solely re-sponsible for crucial aspects of the treatment planning process including the dose prescription, localization of the patient's tumor and the related target volumes, any dosimetric or normal tissue dose constraints, as well as final approval of the treatment plan. They should be certified by one of the recognized boards ͑the American Board of Radiology or its equivalent͒ and hold an appropriate state license, where applicable.
Radiation oncology physicist. The radiation oncology physicist shall be primarily responsible for the design and implementation of the QA program for treatment planning. The physicist generates the treatment machine data necessary for input into the planning system, and directs and reviews all computerized dosimetry planning for patients. Moreover, the radiation oncology physicist determines the local QA program for treatment planning, including the tests to be performed, tolerances, and frequency of the tests. The physicist shall also understand and appropriately respond to discrepancies or problems uncovered by that QA program. We recommend that the radiation oncology physicist be certified in Radiation Oncology Physics by the American Board of Radiology or American Board of Medical Physics ͑or the Canadian College of Physicists in Medicine, if applicable͒ and hold an appropriate state license, where applicable.
Radiation therapist. The radiation therapist is often involved in or responsible for several aspects of the treatment planning process, most notably patient positioning and immobilization, simulation or localization, and plan verification. The radiation therapist should be able to detect equipment deviations or malfunctions, understand the safe operating limits of the equipment, and be able to judge when errors in treatment planning may have occurred, due to equipment, patient-related problems, or human mistakes. We recommend that the radiation therapist have credentials in Radiation Therapy Technology as defined by the American Registry of Radiologic Technologists or possess suitable equivalent qualifications, and hold an unrestricted state license in radiation therapy technology, where applicable.
Medical radiation dosimetrist. The medical dosimetrist is responsible for patient data acquisition, radiation treatment design, and manual and computer-assisted calculations of radiation dose distributions. In consultation with the radiation oncology physicist and radiation oncologist, the dosimetrist generates and documents the chosen treatment plan for each patient. The final plan is reviewed by the radiation oncology physicist and approved by the radiation oncologist. The dosimetrist may also assist the radiation oncology physicist with various aspects of the treatment planning QA program. We recommend that medical dosimetrists be certified by the Medical Dosimetry Certification Board, or at least possess the credentials for board eligibility, if possible.
In summary, it is important to understand that the treatment planning process involves multiple complex steps performed by many people throughout the department. The QA program for treatment planning must therefore focus on the process as a whole and assess the cumulative effects of uncertainties throughout the process. It is also important to reiterate that the complexity of the treatment planning process is increasing, making it imperative that a strong QA program be designed and that the appropriate equipment, personnel, and time be available to implement it. QA for treatment planning has clinical, physical, and administrative components, and its successful implementation requires the teamwork of many personnel.

PART B: QUALITY ASSURANCE FOR CLINICAL RADIOTHERAPY TREATMENT PLANNING
Chapter 1: Introduction

Introduction
Radiotherapy treatment planning ͑RTP͒ has long been an important part of the radiotherapy treatment process, so assuring that the treatment planning process is being performed correctly is thus an important responsibility of the radiation oncology physicist. In recent years, as three-dimensional ͑3D͒ and image-based treatment planning has begun to be practiced in numerous clinics, the need for a comprehensive program for treatment planning quality assurance ͑QA͒ has become even more clear. An AAPM task group ͑TG 40͒ has recently published an overall approach to QA for the therapy process, 2 but this work includes only a very general discussion of treatment planning QA issues. In this report, we propose a methodology to be used by radiation oncology physicists to create the appropriate QA program for the treatment planning systems and processes used in their clinics. Although this QA program will vary widely between different clinics, use of this report should allow each clinic to concentrate its QA efforts on those areas of most importance.

General definitions and aims
The radiotherapy treatment planning process is defined to be the process used to determine the number, orientation, type, and characteristics of the radiation beams ͑or brachytherapy sources͒ used to deliver a large dose of radiation to a patient in order to control or cure a cancerous tumor or other problem. Most often, treatment planning is performed with the assistance of a computerized treatment planning system that helps the treatment planner and physician define the target volume, determine beam directions and shapes, calculate the associated dose distribution, and evaluate that dose distribution. The RTP system consists of a software package, its hardware platform, and associated peripheral devices. Diagnostic tests ͑imaging, x rays, other laboratory tests͒, clinical impressions, and other information are also incorporated into the planning process, either qualitatively or quantitatively ͑an example is the creation of a model of the patient's anatomy based on information from CT scans͒. The treatment planning process includes a wide spectrum of tasks, from an evaluation of the need for imaging studies up to an analysis of the accuracy of daily treatments. This broad definition of the treatment planning process will be described further below ͑see Sec. 1.5͒.
The aim of this report on Quality Assurance for Clinical Radiotherapy Treatment Planning is to describe in detail those issues that should be considered when a QA program is designed. There can be a very large difference in treatment planning capabilities and their clinical utilization among different clinics. Therefore, this report will not define a standard QA program which should be applied by each clinic. Rather, the radiation oncology physicist in each clinic should review this report, use its guidelines to determine those issues that are of most importance, and then concentrate the RTP QA program on those issues. An example framework for each clinic's QA program can be found here, but the specific details of the program should be determined individually.

Scope
In earlier decades, the scope of the decisions made inside the treatment planning system, and inside the treatment planning process, generally involved only dose calculation results and related issues such as wedge selection. Much of the planning was done by the physician-often in the simulator-where the number of beams, beam directions, field sizes, field shaping, and related issues were all determined. Quality assurance work performed in this environment naturally concentrated on dose calculation-related issues. [5][6][7][8][9][10]1 Now, however, with the continuing expansion of 3D planning capabilities in many centers, a huge increase in the magnitude and complexity of treatment decisions that are made inside the RTP system has occurred. With full 3D planning, decisions about the area to be treated, importance of normal tissue doses, beam directions and energy, field sizes, beam aperture, and most other aspects of how to treat the patient are usually made during treatment planning by some combination of the treatment planner, physician, and physicist. The scope of the RTP QA program must therefore be increased significantly. Therefore, this report encompasses QA for the entire treatment planning process, and not just the limited dose calculation and display parts of planning.
In recent years there have been a number of attempts to broaden the scope of QA efforts in treatment planning. [11][12][13][14][15][16][17] The report by Van Dyk et al., 18 containing recommendations for commissioning and QA of treatment planning computers from the Ontario Cancer Institute and Ontario Cancer Treatment and Research Foundation, is a very valuable description of an approach to RTP QA that should be reviewed carefully by all radiation oncology physicists involved in treatment planning. However, that report did not deal with many of the issues that have become important with the increased availability and use of image-based 3D planning systems.
In this report, a comprehensive approach to the design of a quality assurance program for the radiation treatment planning process will be described. QA issues to be addressed include: • Acceptance testing and specifications for acquisition of a RTP system ͑Chap. 2͒. • Testing, documentation, and characterization of the nondosimetric aspects of planning ͑Chap. 3͒.
• Measurement, testing, and verification of the dosimetric aspects of the planning system ͑Chap. 4͒. • Routine QA testing ͑Chap. 5͒. • QA of the clinical use of treatment planning throughout the entire planning and treatment processes ͑Chap. 6͒. • Computer systems management as part of the QA program ͑Chap. 7͒. • Vendor and user responsibilities in the areas of software quality assurance and vendor support. Although a very important part of the report, this discussion is included in Appendix 1 since it deals with the interactions between vendor and user, rather than the direct activities that are the main part of the QA program.

Initial recommendations (how to use this report)
A small number of recommendations are listed in Table  1-1 to help readers read and use this report effectively.

The treatment planning process
As described in Sec. 1.2, the treatment planning process consists of all the activities associated with determining how the radiation treatments will be carried out. Table 1-2 lists a general model of the treatment planning process. This model is not intended to include all institution-specific details, but it does include most major aspects.

Sources of uncertainties
Treatment planning involves numerous uncertainties, all of which can affect the accuracy with which planning and treatment are done. From a QA standpoint, one should estimate each uncertainty and then determine the expected re- 1. This report is not a prescriptive listing of all that must be done to perform adequate RTP QA. The report is intended to give a comprehensive summary of issues which should be considered when creating the RTP QA program for an institution. No one institution will need to perform all of the work discussed in this report.
2. The Task Group recommends that users of a particular commercial treatment planning system should band together, with or without the assistance of the vendor of that system, to help each other create and perform the comprehensive QA which is required for that particular planning system. It is unlikely that any one institution can perform all the quality assurance, by itself, that is appropriate for a complex commercial planning system. 3. It is critical that each institution name one radiation oncology physicist to be the ''responsible physicist'' for treatment planning in that institution. This position includes overall responsibility for the implementation, quality assurance, and clinical use of treatment planning in the institution, and is the most appropriate point of contact for vendor RTP support or other people involved in treatment planning outside of the institution. 4. Treatment planning system vendors have important quality assurance and testing requirements ͑see Appendix 1͒, but this report deals only with the kinds of work which should be performed by the radiation oncology physicists in order to assure the appropriate use of treatment planning in their institution. 5. Although this report includes discussion of many issues which are relevant only to RTP systems which are so-called ''3-D RTP systems'', institutions with less sophisticated and complex RTP systems should also make use of the report. Some issues discussed here may be trivially or simply handled inside those simpler planning systems, but even so, those issues are still present somewhere in the planning process, either explicitly or implicitly in the way the system is used. In either event, the process of treatment planning should be analyzed in the same way, and the quality assurance program should be appropriately modified to handle the situation.

Patient Positioning and Immobilization
• Establish patient reference marks/patient coordinate system.

Image Acquisition and Input
• Acquire and input CT, MR, and other imaging information into the planning system.

Anatomy Definition
• Define and display contours and surfaces for normal and critical structures. • Geometrically register all input data ͑CT, MR͒, including registration with initial simulation contours, films, patient position, etc. • Define target contours, generate 3-D target surface using surface expansion, import target information from multiple imaging modalities. • Generate electron density representation from CT or from assigned bulk density information.

Beam/Source Technique
• Determine beam or source arrangements.
• Determine beam or source weighting.

Dose Calculations
• Select dose calculation algorithm and methodology, calculation grid and window, etc. • Perform dose calculations. • Set relative and absolute dose normalizations. • Input the dose prescription.

Plan Evaluation
• Generate 2-D and 3-D dose displays.
• Use automated optimization tools.

Plan Implementation
• Align ͑register͒ the real patient with the plan ͑often performed at a plan verification simulation͒. • Calculate Monitor Units or implant duration. • Generate hardcopy output. • Transfer plan into record and verify system. • Transfer plan to treatment machine.

Plan Review
• Perform overall review of all aspects of plan before implementation.
sultant uncertainty in the calculated dose distribution. Some of the sources of uncertainty in the RTP process are listed below: • Patient localization. Patient motion, including organ motion, during CT scanning, simulation, treatment, and other associated procedures adds to the uncertainty in location of the patient, target, and/or critical normal structures with respect to the radiation beams. • Imaging. Problems in transfer, conversion, or use of imaging data can lead to increased geometrical uncertainties in the relationship of the beams to the anatomy. Use of more than one imaging modality increases this problem due to the need to geometrically register the image sets with each other. Additional uncertainty is caused by geometrical distortions ͓magnetic resonance ͑MR͔͒ and/or lack of resolution ͓positron-emission tomography ͑PET͒, single photon emission computed tomography ͑SPECT͔͒. • Definition of the anatomy. Inaccuracy in definition of the anatomical model of the patient may be one of the largest sources of uncertainty in the entire RTP process. Each of the steps involved ͑drawing contours, meshing contours into a 3D object description, creating surface and volumetric displays͒ include a geometrical uncertainty. Furthermore, the delineation of tumor and target volumes by the physician is very dependent on the physician, and differences between physicians or between different sessions with the same physician have been demonstrated. 19,20 • Establishment of beam geometry. The accuracy of the treatment planning beam geometry depends on the resolution and tolerance of each machine parameter, and on the frequency and magnitude of setup errors made during daily treatments. Error rates on the order of 1% have been described. 21 Computerized record and verify ͑R/V͒ systems and multileaf collimators ͑MLCs͒ may reduce some of these errors, but may substitute more systematic errors for the random errors which they help prevent. • Dose calculation. Sources of uncertainty include the accuracy of the original measured data, consistency of machine output, resolution and sensitivity of the measuring instruments, quality of the data analysis, transfer of the data into the RTP system, and the way those data are used. Uncertainties associated with calculation algorithms arise from poor modeling of the physical situation, lack of appropriate supporting physics, inappropriate approximations, use of calculational grids that are too large, poor parametrizations, and other limitations of either the basic algorithm or its use. • Dose display and plan evaluation. Uncertainties in dose display depend mostly on how accurate the representation of the dose distribution is, but are also related to how clearly the information is presented. Dose volume histograms ͑DVHs͒ are sensitive to anatomical definition, the methods used for representing the anatomical objects, the resolution and extent of the dose calculation grid, the resolution and methodology behind the formation of the DVH, and how the DVH is presented. If tools like normal tissue complication probability ͑NTCP͒ and tumor control probability ͑TCP͒ are used, then the reliability and clinical relevance of those models must be considered, as well as the limited clinical data which are available to help parametrize the models. • Plan implementation. Errors include transcription errors in writing the plan into a patient ͑paper or electronic͒ chart and misconceptions of the treatment therapists when faced with poor documentation of some aspect of patient or plan setup.

Required and/or desired tolerances and accuracy
Determining the required or achievable accuracy for treatment planning is a very difficult aspect of the creation of a RTP QA program. Here, we will not provide a table of recommended values, since it is clear that what is achievable with one kind of planning system may be quite unachievable with another. It is the responsibility of the radiation oncology physicist to determine ͑1͒ the accuracy of the institution's particular RTP system for a range of clinical situations; and ͑2͒ how that expectation of accuracy must be modified to account for any particular clinical situation, the kinds of treatment plans that are created, and other aspects of the local situation.
For illustration, we present two example sets of expectations for the accuracy of various parts of two different treatment planning systems spanning the range of sophistication found in RTP systems: • ''Traditional.'' This is the prototypical ''twodimensional'' ͑2D͒ planning system, which uses only manual contour input ͑no CT data͒, allows only axial beams, does not model blocks or compensators, and only contains a 2D model for calculating the dose distribution from a beam. • ''3D.'' This is a fully 3D system, which models all the capabilities of normal treatment machines and contains a modern 3D pencil beam electron dose calculation model and a modern 3D photon beam dose calculation algorithm that take into account 3D scatter, the 3D shape of inhomogeneities, and other effects. Table 1-3 gives the range of accuracies that are probably achievable with these two kinds of planning systems.
Chapter 2: Acceptance tests for treatment planning systems 2.1. Acceptance testing QA testing is sometimes confused with acceptance testing. In this report, we use the term acceptance testing as follows: an acceptance test is performed to confirm that the RTP system performs according to its specifications. If there is little rigor in the specifications of the RTP system, then there will be little need or ability to design an acceptance test. This highlights the need for rigorous and careful design of the specifications for acquisition of an RTP system if one wants to ͑1͒ know how the RTP system should perform in various situations; and ͑2͒ be able to design and perform a formal acceptance test to verify that the system works as specified. Beam location resolution 0.5 cm Ͻ0.1 cm Traditional system may force beam center to be on axial calculation plane or CT slice. 3-D system allows any specified isocenter coordinates.
Collimator setting 0.5 cm 0.1 cm Resolution of jaw positions typically 1 mm, although traditional system will usually specify field width and length with resolution of 0.5 cm at best.
Aperture definition 0.3 cm or more 0.1 cm Block shape not modeled in prototype traditional system, but may be entered with digitizer for some types of systems. 3-D system may use computer-generated aperture.
Collimation and aperture display up to many cm 0.1 cm Traditional system may not display aperture shape and may not display divergence effects.
Gantry angle 1 deg Ͻ1 deg Resolution of gantry angle typically 0.1 deg in 3-D systems.

Determination of specifications
A detailed discussion on the creation of specifications for a modern 3D RTP system is a large task and beyond the scope of this report. However, a few brief comments are included here.
Specifications must be reasonable constraints that are quantifiable and testable or measurable. For example, it is meaningless to write a specification requiring 2% accuracy in dose calculations. This is much too broad a statement.
Where? Under what circumstances? With what input beam data? In addition, satisfaction of specifications usually should not be dependent on clinic-specific beam data since a vendor typically cannot test or verify the quality of an individual clinic's data.
Items suitable for specification can be divided into three broad categories: • Computer hardware: This includes the CPU and all the peripheral devices that are part of the RTP system, such as the display monitor͑s͒, printer, plotter, tape drive, etc. • Software features and functions: Many software feature specifications will be of the yes/no or exists/does-notexist type, rather than quantitative. • Benchmark tests: Performance on benchmark tests indicates the accuracy of the dose calculation algorithm under very specific circumstances with specific beam data. Calculation times can also be measured. If the radiation oncology physicist chooses to write specifications for the purchase of a new RTP system, rather than just selecting a particular system, then the needs and requirements of that particular clinic must first be carefully assessed. This includes evaluation of the manner in which the treatment planning system will be used. All aspects of the treatment planning process should be considered, not just the dose calculation abilities. What functionality and capabilities are required? What types of input are needed? What types of input will be used? What level of performance ͑on which benchmark tests͒ is desirable? These requirements then need to be translated into specifications that can be quantitatively stated and tested. The specifications document itself should clearly define each item and the desired specification, as illustrated in Table 2-1. After the physicist has determined the ideal specifications, the physicist will need to negotiate with the vendor to settle upon a final set of specifications.

Acceptance testing procedure
Specifications should be written with particular tests already in mind. It is important to make sure that the procedure actually tests the feature to be tested and is capable of determining whether the specification is satisfied or not. Thought should be put into the exact procedures and the order of the tests in order to minimize the total work necessary and to correlate optimally with other acceptance tests as well as with QA and commissioning tests. A procedures document should then be written that clearly describes the individual procedures in detail. The procedures to be used must be agreed to by both the user and the vendor.
The acceptance testing should be carried out on the system after it has been installed in the clinic but before it is used clinically. Tests of the hardware and the software features should be performed by the user. Significant time may be required to perform detailed benchmark testing of dose calculation or other algorithm accuracy, so it should be determined at the time of the definition of the acceptance test procedure whether these tests are to be performed by the user or the vendor. If these tests are performed by the vendor, the user may want to repeat some or all of the tests to verify the results.
Results from the acceptance testing should be carefully documented, along with any variation from the defined procedures, and kept as long as the treatment planning system is used in the department.

Chapter 3: Nondosimetric Commissioning
The modern RTP process includes many aspects not directly related to dose calculations. Therefore, the RTP QA program must also handle these important nondosimetric issues. Most of the general topics the QA procedure should cover are discussed below, although all possible nondosimetric issues are not listed.
The long list of issues in this section may appear to apply only to complex 3D planning systems. However, these issues should also be considered for 2D systems, although many of the issues raised may condense to testing a few simple features of the system. Conversely, this list may be incomplete for workers who have advanced systems or those who have developed specialized techniques. The aim of this section is to provide a framework that will help radiation oncology physicists design QA programs appropriate for their clinical planning techniques and systems. Considering the huge amount of work that would be required to thoroughly test each of the features listed here, it is reasonable to expect that only those RTP system features that will be used clinically should be tested initially. However, one should be aware that some of these features may be important to understand, even if no explicit use of the feature is intended, due to exploration, evolution of planning techniques, or design of the system. The terms ''confirm'' and ''verify'' are used throughout this section as testing of various capabilities or features are The NCI ECWG electron dataset will be used for a series of dose calculation verification checks of the accuracy of the 3-D electron pencil beam dose calculation which is included in the system.
1. The vendor shall demonstrate that the dose calculations for open field electron beams with applicator sizes 6ϫ6 and 15ϫ15 cm, at 100 and 110 SSD, will agree with the ECWG measured data within Ϯ3% in the central 80% of the projected field size, and that the 10, 20, 50, 80 and 90% isodose lines ͑relative to 100% at dmax on the central axis of the beam͒ will be within 2 mm of the respective measured isodose lines.
2. The vendor ... discussed: Note that the methods used to perform and document this task may be very dependent on the treatment planning system and/or features being considered.

Introduction
This chapter is perhaps the most complex chapter in the report. To a physicist familiar only with older treatment planning systems that support only straightforward twodimensional treatment planning, the terminology and tasks developed in this chapter may seem unfamiliar, for it is in the nondosimetric issues that much of the complexity of modern treatment planning systems is manifest. The quality assurance testing of the nondosimetric aspects of the treatment planning process that is recommended in this report follows the actual clinical treatment planning process, as summarized in Table 1-2, and this table can provide a helpful guide through this chapter. The first part of the chapter deals with acquisition of patient information, starting with patient positioning and immobilization, image acquisition, and conversion of the image information into a suitable anatomical model of the patient. The chapter continues with a discussion of acquisition of beam information, including beam geom-etry, definition of field aperture, identification and description of beam modifiers, and identification of treatment machine, modality, and energy. The next part of the chapter addresses operational aspects of the dose calculations, including selection of dose algorithm and heterogeneity corrections. Evaluation of treatment plans is addressed next, including issues related to dose display and dose-volume histograms. The next part of the chapter looks at plan documentation, implementation, verification, and transferring plan information from the treatment planning system to the treatment machine and the patient record. The chapter then addresses nondosimetric quality assurance issues in brachytherapy including source definition, source geometry, source display, and dose calculations. The chapter concludes with a description of integrated ''start-to-finish'' tests used to perform a final check on the systematic behavior of the treatment planning process.

Patient positioning and immobilization
Patient immobilization and positioning are an important part of the planning process, since many planning decisions are based on data from these procedures. Hardcopy output Print out all hardcopy documentation for a given series of plans, and confirm that all textual and graphical information is output correctly.
3.2.1. Immobilization. The purpose of patient immobilization is to help position the patient in a reproducible manner ͑consistent with the technical goals of the treatment͒ and to help the patient remain motionless during treatment. Immobilization techniques may be as simple as positioning the arms in a particular fashion or as complicated as the use of an invasive stereotactic device. The quality of the immobilization affects the reproducibility with which the patient is positioned for each of the procedures involved in the planning/delivery process, and may affect the accuracy of treatments. The use of particular immobilization devices may change image quality and/or monitor unit calculations, so these effects should be investigated prior to clinical use. Note that few immobilization devices actually keep the patient immobile, so motion and positioning errors often continue to be a concern even with use of such a device.
3.2.2. Positioning and simulation. The next step in the planning process involves localizing the volume to be treated. This includes defining the positions of the patient, tumor, target, and normal structures. Traditionally, this procedure has been accomplished with the simulator using orthogonal radiographs, a manual contour, and laser marks which establish an initial isocenter. However, with the development of image-based RTP systems and ''virtual'' simulation, localization procedures involving CT images are now often used.
No matter how it is obtained, the patient position information must be acquired accurately and then transferred accurately into the RTP system for further planning and analysis. Similar accuracy requirements hold for beam geometry and other information obtained during simulation. Simulators, CT scanners, and ''virtual'' simulators should therefore be subject to a rigorous QA program that includes both mechanical and image quality tests. For example, for simulators and CT/MR scanners, the geometrical accuracy of all beam and couch parameters, laser alignment systems, and gradicules should be assessed. QA for simulators has been the topic of a number of publications 2,22,23 and the reader is referred to those reports. QA for CT scanners is discussed in Sec. 3.3.1. QA for CT-simulation software is covered in the present report, as CT-simulation software corresponds primarily to the geometric aspects of a treatment planning system.

Image acquisition
A set of ''images'' used to define the patient anatomy can be as simple as a manual contour and a pair of orthogonal simulator films, or as complex as cross-sectional image sets from several different modalities. Images can be obtained from many sources including planar radiography ͑film or digital͒, computed tomography ͑CT͒, magnetic resonance ͑MR͒, positron emission tomography ͑PET͒, single photon emission computed tomography ͑SPECT͒, and ultrasound ͑US͒. Although most of these imaging sources are used for visualizing anatomy or physiology, there are also other reasons for their use. For example, CT often is used to generate a 3D map of the patient electron density, necessary for accurate dose calculations.
The manner in which these imaging data are acquired may have dramatic effects later in the planning process, particularly if the data are not acquired correctly. QA of image acquisition must ensure that images have been obtained in an optimal way, and that their transfer into the RTP system, and use therein, has been performed accurately.
3.3.1. Imaging parameters. Numerous imaging system parameters can affect how the image data are used. For example, incorrect setting or reading of image parameters such as pixel size, slice thickness, CT number scale, and orientation coding can cause the RTP system to make incorrect use of the data. Furthermore, lack of understanding of partial volume effects in cross-sectional images may cause incorrect identification of anatomical or other information from the images. Control of the imaging parameters at acquisition is therefore an important part of the QA process that applies to each patient.
For correct use of imaging information, this report recommends developing standard protocols for image acquisition, optimized for each disease site. These protocols should be used routinely and should be confirmed by routine inspection of clinical procedures. These protocols should include the following information: • the extent of the patient that is to be scanned, • the position of the patient as well as any immobilization devices, • location and type of radio-opaque markers used on patient surface as coordinate system reference, • scan parameters such as slice spacing and thickness, • breathing instructions for patients scanned in abdomen and/or chest, • the policy on the use of contrast agents ͑for CT, MR, and other modalities͒.
QA and commissioning of the simulator ͑see Sec. 3.2.2͒ and other imaging devices such as CT or MR should be performed according to relevant AAPM task group recommendations 2,24,25 and other useful work. 26 3.3.2. Artifacts and distortion in image acquisition systems. All imaging systems are susceptible to artifacts and/or geometrical distortions, thus information from the image may need to be modified or interpreted before it can be used. Examples abound, such as streaking in CT images near highdensity anatomic structures such as teeth and fillings, modification of the derived tissue densities when CT contrast is used, distortion in MR images ͑e.g., near interfaces of changes in magnetic susceptibility such as the tissue/air interface which causes distortion in external fiducial markers͒, or the general systematic variations in image value ͑Hounsfield units͒ at different locations in the imaging volume. Imaging protocols should therefore try to minimize artifacts, allow easy identification of an artifact when it does occur, and allow for correction of the image data. Geometrical distortions and inaccuracies in various imaging modalities have been discussed in the literature for CT 17 and MRI. 20 Because these artifacts are dependent on the particular situation and are not due to software-specific problems, the QA procedures to deal with these issues are part of the clinical planning process. Although detailed discussion of these issues is beyond the scope of this report, the user should be aware of the possibility of the kinds of artifacts listed in Table 3-1, as well as how to resolve, circumvent, or compensate for the problems they cause.

Anatomical description
The anatomical model or description of the patient is one of the most critical issues in RTP, and the introduction of 3D planning has greatly expanded our knowledge of the anatomy of each individual patient. As we all know, very precise knowledge of the dose distribution will do little good if we have incorrectly identified the tumor, target or normal tissues. Therefore, a significant effort should be spent on QA of the anatomical description. Since much of the testing associated with the anatomical description of the patient is dependent on the details of the RTP system used, this section concentrates on delineating the issues that should be considered and why they are important, rather than describing specific tests in detail.
3.4.1. Image conversion and input. In recent years, imaging information obtained from CT has become the basis of our anatomical model. Other image information, such as digitized radiographs or images from other imaging modalities, may also be incorporated. Typically, each of these images is transferred from a vendor-specific computer system, usually with a vendor-specific image file format and/or transfer media or network, to the RTP system. Test issues are listed in Table 3-2. Many of these tests can be performed using scans of phantoms with various configurations of the imaging device. In Appendix 1, The Task Group recommends that all vendors of image acquisition systems and RTP systems make available the standard DICOM image format for image input/output, 33 so that the number of image conversion methods is reduced to this one universal format. Note that the dataset registration process which is necessary if one uses more than one set of images ͑dataset͒ is discussed in Sec. 3.4.5.

Anatomical structures.
In older 2D RTP, the only anatomical information available was one or more contours of different structures taken on one or a few slices, so the description of anatomical structures was quite simple. Little QA was required other than confirmation that the drawing device ͑digitizer or other such device͒ accurately input the desired coordinates for a particular contour. In a 3D planning system, however, the anatomical model used for the patient is much more complex, requiring a much more complete set MR distortion Distortion in geometric accuracy of MR images, dependent typically on magnetic field homogeneity, changes in magnetic susceptibility at interfaces, and other effects. May lead to incorrect geometrical positioning of imaging information.

Paramagnetic sources
Local distortions in MR images.

Topic Tests Reasons
Image geometry Document and verify parameters used to determine geometric description of each image ͑e.g., number of pixels, pixel size, slice thickness͒.
Vendor and scanner-specific file formats and conventions can cause very specific geometrical errors when converted for RTP system.
Geometric location and orientation of the scan Document and verify parameters used to determine geometric location of each image, particularly left-right and head-foot orientations.
Vendor and scanner-specific file formats and conventions can cause very specific geometrical errors when converted for RTP system.

Text information
Verify that all text information is correctly transferred. Incorrect name or scan sequence identification could cause misuse or misinterpretation of the scans.
Imaging data Verify accuracy of grayscale values, particularly for conversion of CT number to electron density.
Wrong grayscale data may cause incorrect identification of anatomy or incorrect density corrections.

Image unwarping ͑removing distortions͒
Test all features, including the documentation tools which assure that the original and modified images are correctly identified within the system.
Methodologies which modify imaging information may leave incorrect data in place.
of test procedures. The basic contours of the 2D system have been superseded by a hierarchy of objects including points, contours, slices, 3D structures, 3D surface descriptions, and even multiple datasets of self-consistent volumetric descriptions, as summarized in Table 3-3.

3D Structures:
One of the major differences between 2D and 3D RTP systems is how anatomical structures are described. In 2D, most structures are defined by 2D contours on one or a few axial slices, and contours are generally not related from one slice to the next. In 3D, a 3D structure is created for each anatomical object. This structure is often defined by a series of contours drawn on multiple slices of some image dataset ͑for example, CT͒, and the contours for a particular structure are all related. A 3D RTP system may require many different procedures to check the 3D anatomical structure description functionality, as listed in Table 3-4. 3.4.2.2. Contours: Anatomical structures can be entered into the RTP system by a variety of methods, but the most typical method is to create contours on a series of slices through the patient, and then to create the 3D structure from the serial contours. QA tests for contour definition are considered in Table 3-5.

Points and lines:
The display and geometrical definition of points and lines defined inside the system must accurately reflect the geometrical location of the image on which they are defined. If multiple datasets are allowed, then the point and line definitions must be checked in all image sets and coordinate systems.

Density representation.
In most image-based planning systems, the CT data are used not only for positional information about the anatomy, but also to define the relative electron density ͑number of electrons per unit volume͒ distribution throughout the patient model. This information is used for density-corrected dose calculations. Table 3-6 discusses issues related to the density description.
The actual performance of density-corrected dose calculations, and the specific use of the relative electron density information, are part of the dosimetric QA and are discussed in Chap. 4.

Bolus and editing the 3D density distribution:
Bolus may be used in treatment planning in at least three different ways: • Definition of external bolus on the surface of the patient. • Modification of the CT-based electron densities in a certain region of the patient ͑e.g., to edit out the effects of contrast material͒. • Introduction of bolus material into sinuses or other body cavities. • In each of the three implementations, the bolus may affect the rest of the RTP system in a different way. Bolus test issues are listed in Table 3-7.

Image use and display.
The various ways image information is used and displayed should be considered in the RTP QA program, as in Table 3-8. 3.4.5. Dataset registration. One of the more powerful advances associated with the use of 3D planning has been the ability to quantitatively use imaging information from various different imaging modalities such as CT, MR, PET, SPECT, ultrasound, and radiographic imaging. In order to use this information, the planning system must contain tools which make it possible to quantitatively register the data from one imaging modality with similar data obtained from another modality. Checks of the dataset registration and multiple dataset functionality involve general commissioning tests as well as development of routine procedural checks to make sure the information is used correctly for each particular case.
Dataset registration and the use of multiple datasets in RTP, as well as in other fields, is a large and complex area, and detailed discussion of methods or QA of dataset registration are beyond the scope of work of this task group. The task group recommends that AAPM form another task group specifically charged to develop a report on use and quality assurance of dataset registration techniques. Readers should consult the relevant registration literature [27][28][29][30][31]20,32,79 for further guidance. Voxel description A set of 3-D voxels used to describe a particular 3-D structure.
Surface description A surface mesh that defines the boundary of a 3-D structure.
Contours 2-D outlines, usually created on a slice or image plane. These outlines are typically used to generate the 3-D anatomical structure description.
Reference lines Straight or curved line segments used to mark special anatomy or other features relevant to the treatment plan.

Points
Points defined in 3-D, often used as markers.
Density description A description of the electron density of a structure. Either defined as a bulk ͑or assigned͒ value or derived from CT data.

Region-of-interest ͑ROI͒ description
A voxel or surface description of each 3-D structure of interest. Used for calculation of dose volume histograms and other kinds of statistics.
Dataset A geometrically self-consistent set of data ͑e.g., a set of CT scans obtained in one acquisition͒.

Beams
The next major section of a normal planning system incorporates modeling of, and interactions with, the radiation beams. Numerous aspects of the beam definition and use functionality are critical items to be checked by the QA program. Table 3-9 lists some of the parameters required to create the specification of a beam. Clearly, it is essential to understand, document, and test the behavior of all beam parameters as beams are created, edited, saved, and used throughout the planning process. Understanding how these parameters are used and

Relative electron density definition
Verify that correct definition for relative electron density ͑r.e. density͒ is used: • Assigned bulk density which sets specified r.e. density everywhere inside structure. • R.e. densities derived from CT number ͑see density tests in 3.4.3, Table 3-6͒.
Relative electron densities used during dose calculations depend on the choice of method for definition of r.e. density and on its correct implementation.
Display characteristics Check color, type of rendering, and type of contours to be drawn when displaying structure.
Display errors can cause planning errors due to misinterpretations.

Auto-segmentation parameters
Check parameters for autocontouring and other types of autostructure definition for each structure.
Incorrect parameters can lead to incorrect structure definition. Parameters are likely to be defined separately for each structure.

Structure created from contours
Resolve issues such as: • Can non-axial contours be used?
• Is number of contour points limited?
• What is the response to sharp corners in contours?
• What happens with missing contours?
• Is regular spacing required between contours?
• Does algorithm handle bifurcated structures?
This is the most common way to define 3-D structures. Errors in functionality, use or interpretation could lead to systematic errors in treatment planning for a large number of patients.

Structure constructed by expansion or contraction from another structure
Resolve issues such as: • What are the limits of the expansion algorithm? Planning target volumes ͑PTVs͒ are often defined by expansion from the clinical target volume ͑CTV͒. 66 Errors in the expansion could cause errors in target definition.
Structure constructed from non-axial contours • Test should include same tests as for creation of structures from axial contours but should be performed separately for all contour orientations. • Verify bookkeeping for source of structure definition.
Numerous independent difficulties can arise dependent on the underlying 3-dimensionality of the data structures and design of the code. 72,80 ''Capping'' ͑how end of structure is based on contours͒ • Verify that all methods of capping are performed correctly and 3-D implications are understood. • Document default capping for different structures. • Establish clinical protocols for each 3-D anatomical structure.
Capping can affect dose calculation results, target volume shapes, BEV display and DRR generation, effects of lung densities and other important parts of the plan.
Structure definition • Verify basic surface generation functionality using simple contours. See example test in Appendix 2. • Run test case͑s͒ for situations in which the exact formulation of the surface mesh has been calculated by hand. • Verify surface generation functionality for extreme cases ͑e.g., sharply pointed contours, unclosed contours͒. Tests will depend on algorithm.
These tests should convince the user that the algorithm generally works correctly. Geometrical accuracy of the digitization device can be quite user-dependent. Many digitization systems suffer from position-dependent distortions. Digitizer behavior can also be time-dependent.

Contouring on 2-D images
Verify: • The accuracy of the contour display with respect to the image display. • The 3-D location of the contour in the coordinate system͑s͒ in which the planning system calculates dose. • The response of the contouring algorithm to extreme situations ͑e.g., too many points entered, looped contour, Ͼ1 distinct closed contours created͒. • The identification of each contour and its associated 3-D structure.
Contouring on CT images is the basis of most 3-D planning. Errors in contour coordinates or display can lead to incorrect anatomy being used for planning. Contour accuracy may be dependent on image type or orientation.
Tests may include: • Contouring structures on a scanned phantom and comparing contours to the known dimensions of the phantom's structures. • Contouring structures on a grayscale phantom constructed in software. This eliminates any image acquisition and pixel averaging errors. • A subset of tests should be performed for each type of image, and for each slice orientation ͑sagittal, coronal, axial, oblique͒, since the contouring features and/or use of the contours may not be independent of these parameters.
Autotracking contours • Verify proper response of the tracking algorithm for various situations ͑e.g., different grayscale gradients, different image types, markers, contrast, image artifacts͒. • Tests may involve scanned phantoms or simulated grayscale phantoms as described above. Partial volume effects probably are most easily sorted out using images which model the effects of slice thickness changes on the grayscale values.
• The gradient range used to identify the threshold to be autotracked can affect the size and location of the contour.
• Misunderstandings of partial volume effects may lead to improper contours.

Bifurcated structures
Resolve issues such as: • Can the system maintain more than one contour per slice for a particular structure? • Does it form the 3-D structure correctly? Check 3-D surfaces visually and check DVHs.
The algorithm for creating bifurcated structures may affect the calculation of volumes of these structures.

Contours on projection images ͑DRRs, BEVs͒
• Check that points defined on projection images define lines through the 3-D data. • Check that contours drawn on projection images are projected correctly when viewed in full 3-D displays. • Check intersection of such contours with various axial, sagittal, and coronal slices.
Incorrect handling of contours on projection images can lead to misinterpretation of plan displays.

Contours on CT scannograms
Same tests as for projection images. CT scannograms have significant divergence in the axial direction but typically negligible divergence in the sagittal direction.

Extracting contours from surfaces
Determine the general limitations and functionality of the implementation: • Can contours be cut onto a slice of arbitrary orientation?
• Are enough points used to accurately define the contour?
• Does an extracted contour overwrite the original drawn contour? • What happens for complex structures which result in multiple independent contours on a single slice?
Contour extraction onto axial and non-axial images or reconstructions provides one of the best ways to quantitatively check the 3-D description of anatomical structures.
when they can be modified is an important and difficult part of the QA program design. Table 3-10 lists some parameters which describe a MLC in the RTP system. If some of these parameters are missing from the description, there may be limitations in how the system can model a particular MLC.
In order to assure that the RTP system faithfully reproduces the desired beam configuration, numerous issues must be verified, as listed in Table 3-11. 3.5.2. Machine description, limits and readouts. As modern planning systems use more and more of the capabilities of the treatment machine, an increasingly sophisticated description of the limits of those capabilities for each par-ticular machine must be a part of the beam technique module of the planning system. Complex systems may make use of: • numerous energies/modalities and/or specialized modes, • individual jaw and MLC leaf motion limits, • number, type, and orientation of wedges, • naming conventions, • machine angle conventions, limitations, and resolution of readouts for each motion, • speed of motions, if available, • the entire geometric shape of the treatment machine.

Topic Tests Reasons
Relative electron density representation • Verify that the system creates the correct relative electron density representation. See example test in Appendix 2. • Verify that the representation is maintained correctly when contours and/or images are modified.
Incorrect relative electron density information may result in incorrect dose calculations.
CT number conversion Verify that the CT number ͑image grayscale value͒ to Hounsfield number to relative electron density conversion are performed correctly. The conversion may be scanner dependent.
Incorrect conversion can cause incorrect result for density-corrected calculations.

Editing
Verify the proper operation of functions used to edit the relative electron density.
Image grayscale might be altered due to the presence of contrast or image artifacts, leading to incorrect derived relative electron densities.

Measurement tools
Verify display tools used to measure relative electron density. Incorrect information may lead to errors in planning.

Topic Tests Reasons
Electron density within bolus Verify that the density in the bolused region is set to the assigned value. Particularly check use of bolus to edit a CT image.
Incorrect density will lead to incorrect density-corrected dose calculations.
Density measurement tools Verify that tools read the correct density values within the bolus.
Error reading density values makes verification of correct behavior difficult.
Automated bolus design Verify that: • Bolus is designed correctly.
• Bolus information is correctly exported for manufacture and physical bolus is correctly made.
Incorrect behavior will lead to wrong design or implementation of bolus.
Beam assignment Confirm whether bolus is associated with a single beam or with the entire plan.
Could lead to incorrect calculation results.

Dose calculation
Verify that the bolus is accounted for in the dose calculation.
Could lead to incorrect calculation results.
Monitor unit calculation Confirm the proper method to calculate monitor units when bolus is used.
Possible incorrect MU calculation or patient set-up.
Output and graphic displays • Verify that bolus is displayed properly in all displays and hardcopy output. • Verify that bolus is properly documented within the plan and in the hardcopy output.
Possible incorrect bolus setup or use during treatment.
This task group recommends the adoption of the IEC 1217 conventions 34 for specifying gantry angle, collimator angle, table angle, wedge orientation, multileaf collimator leaf specification, and patient orientation. However, until these standards are universally used, it is necessary that the user be aware of both the convention used by his/her treatment machine and that used by the RTP system. If possible, the planning system should be configured to agree with the treatment machine. If this is not possible, the user must determine and document transformation of planning system parameters to machine settings. Testing is suggested in Table  3-12. 3.5.3. Geometric accuracy. The location and orientation of each beam in a plan must correspond to the real situation. The correctness of the translation of the planning system beam coordinates into those coordinates used to setup the fields on the actual patient must be continuously monitored, since it depends not only on software but on the treatment planning and treatment delivery procedures used in the clinic.
Further geometric checks of accuracy are listed below: • The geometric resolution and accuracy for each parameter must be assessed using the coordinate values contained inside the file which contains the beam description as well as with graphical displays of the information inside the RTP system. Use of CT information which describes material which will not be present during dose delivery will cause dose distribution to not be representative of the real dose distribution.
Geometrical accuracy of slices associated with images Verify accuracy of the geometrical location of the slices with respect to the rest of the patient anatomy.
Inaccuracies in geometry can lead to errors in the 3-D visualization and in planning.
Region-of-interest analysis Verify mean, minimum, and maximum CT number inside a region of interest ͑in a slice and in a volume͒ for a range of situations.
CT numbers and electron densities are important when evaluating the accuracy of the dose calculation results.
Positional measurements Verify point coordinates, distances, and angles in each coordinate system for each display type.
Measurements are often used for important planning and evaluation functions such as placing beams and identifying anatomical markers.

3-D object rendering
Confirm color and other rendering functions. Incorrect rendering may misrepresent the geometrical situation.
Multiple window display use Verify that each panel of a multiple window display is kept current as the planning session proceeds.
Inconsistencies could lead to incorrect planning decisions. 3.5.4. Field shape design. Field apertures can be created using rectangular collimators, shaped focused blocks, irregularly shaped electron cutouts, and multileaf collimators, and can be entered into a RTP system using several different methods. All methods of field shape entry should be checked. Field shape design issues are described in Table  3-13. 3.5.4.1. Manual aperture entry: Field shape can be manually entered in several ways, e.g., by digitizing block shapes drawn on simulator films, drawing with the mouse on a BEV display 35 or using keyboard or mouse to move the leaves of

Leaf width
Leaf travel ͑min, max͒, field size min and max.

Number of leaves
Overlap between leaves ͑the tongue and groove design of most MLC systems affect this parameter͒.
Distance over midline that can be traveled by a leaf Maximum extension between leaves.
Movement of the leaf carriage Interdigitation of leaves allowed or disallowed.
Leaf transmission Leaf readout resolution.
Minimum gap between opposing leaves Jaw algorithm ͑how the jaw positions are required to relate to the MLC shape͒.
Leaf labels Leaf end design ͑curved versus focused͒.
Leaf editing capabilities Design of side of leaves.
Dynamic leaf motion ͑DMLC͒ capability Leaf synchronization for DLMC Lack of agreement between readout information in RTP system and machine leads to systematic machine treatment errors.

Beam technique tools
Verify correct functionality of tools such as those to move isocenters or set SSDs.
Incorrect functioning of these features will lead to internal mistakes in planning.

Wedges
Verify that wedge characterizations such as coding, directions, field size limitations, and availability are correct.
This can lead to incorrect wedge use in plan or during treatment.

Compensators
Verify correct use and display. Incorrect use during treatment may cause important dosimetric errors. a MLC. Testing of manual aperture entry is described in Table 3-14. 3.5.4.2. Automatic aperture definition: Automatic shape creation algorithms are often used to design block and MLC shapes. 36,35,37 A more complex testing procedure may be necessary for this function, since these algorithms often include use of 3D projections of the selected 3D surface͑s͒ onto the BEV plane, followed by an automatic routine which generates the correct aperture shape. These algorithms can be sensitive to details of the anatomical or beam aperture representations, and should be carefully checked over a series of different situations.

Special MLC features:
In addition to the issues discussed above, there are some special considerations for MLC-defined apertures. The exact correspondence of the MLC leaf position with the desired and recorded positions General system conventions Verify that the planning system conventions agree with system documentation and are used consistently throughout the system.
Problems can cause systematic treatment errors.
Internal consistency Examine the machine settings and 2-D and 3-D displayed orientation of the beam for a variety of gantry, collimator, and target angles. Confirm that the displayed orientations agree with the parameter specifications and with calculated dose distributions. For example, the user should confirm that the beam diverges in the direction away from the gantry, and that the hot spot for a wedged field appears under the toe of the wedge.
Problems here will cause systematic planning system errors.

Readouts
Verify that the planning system parameters ͑transformed as necessary͒ agree with the actual machine settings required to obtain the desired treatment configuration. This can be done by configuring the treatment machine according to the planning system specifications and comparing to the planning system displays, especially a 3-D room view display.
Errors may cause very isolated but systematic treatment errors.

Test frequency
Verify the accuracy of this information at the commissioning of the RTP system and at each major software update.
Systematic errors might be missed at new releases unless checks are made.
Multi-user environment Establish a procedure to ensure consistent beam information in multi-user and network environments.
Users might interfere with each other's plans, or access to the machine database, or other similar problems.  Table 3-15. 3.5.6. Beam and aperture display. Modern 3D planning systems make use of various types of displays and anatomical representations to aid the treatment planner in designing and evaluating a beam configuration. It is thus important to avoid misconceptions of the relationship of the beams and anatomy by verifying the accuracy of these representations, as described in Table 3-16.
Checks of the beam-anatomy projections can be based on calculations of how various anatomical objects should be projected, or they can be confirmed with film and the radiotherapy simulator using a phantom. The calculation approach should be used at least once to confirm the accuracy of the Film magnification factors Confirm that film magnification is correct for film digitization entry.
Incorrect block shape could be used in plan.
Special drawing aids Check geometrical accuracy of aids such as a circular cursor with definable radius.
Could lead to incorrect margins during aperture design.

Number of points in aperture definition
Evaluate the effects of any limitation on number of defining points.
Could lead to incorrect aperture shape.
Editing apertures Evaluate how the algorithm handles aperture editing. Could lead to incorrect aperture shape.
Defining apertures on BEV/DRR displays Confirm geometry, particularly the distance from the source at which the displayed ''BEV plane'' is located.
This could lead to incorrect interpretation of planned aperture.

3-D projections
Confirm correct 3-D projections of anatomical information including contours, structures, and 3-D points into BEV/DRR displays.
Might lead to incorrect aperture design or choice of beam direction.

Topic Tests Reasons
Orientation and angle specifications Confirm that wedge orientation and angle specifications are consistent throughout the planning system, including the hardcopy output. If possible, they should agree with treatment machine conventions.
Wedge labeling or orientation conventions which do not agree with the RTP system can lead to confusion in plans and treatment.
2-D display Check display of wedges in different 2-D planes ͑parallel, orthogonal, oblique͒ for different beam directions, collimator rotations, and wedge orientations.
Visual orientation checks are most effective way to prevent wrong wedge orientation in plan or treatment.

3-D display
Check display of wedges in room view 3-D displays for situations as described above.
Incorrect wedge orientation leads to large dose differences.

Orientation and field size limitations
Verify that wedge orientations and field sizes not allowed by the treatment machine are not allowed in the planning system. These limits might be defined separately for each beam energy, so they should be tested for each energy/wedge combination.
May lead to plans which cannot be delivered.
Autowedges ͑wedges inside the head of the machine͒ Confirm that the division of a field into fractional open and wedged fields agrees in the RTP system and on the treatment machine.
Could lead to incorrect dose distribution or monitor units.
Dynamic wedge 69 Verify that the implementation in the RTP system has the same capabilities, limitations, orientations, and naming conventions as on the treatment machine.
Incorrect use of dynamic wedge possible.
system, but simulator-based checks may be appropriate for routine checks that can be combined with other RTP QA tests.

Compensators.
Compensators can be designed either within the RTP system or by some independent system. In either case, the accuracy of the input of compensator information such as size, shape, thickness variation, and associated beam must be confirmed. Display and specification of compensators can be checked much like that for wedges, blocks, and other beam modifiers. Automated transmission of compensator information to a compensator maker must also be checked. Calculational accuracy is assessed in Chap. 4.

Operational aspects of dose calculations
The dose calculation is often thought of as the heart of the treatment planning process; however, it may be better to consider it as just one of the many different aspects of planning. Quality assurance of dose calculations includes more than confirming that the algorithm works correctly or that the calculated doses agree with the measured ones. Many parameters must be defined before calculations can be performed, either explicitly by the user or by default by the system, and these parameters influence the resulting dose distributions.
The scope of checks of the operational aspects of the calculation methodology which are required can be quite dependent on the sophistication of the RTP system implementation. However, even if not all of the details below are handled explicitly by the RTP system, each institution should consider the relevance of each issue, since somewhere within the planning process most of these issues are being handled, either explicitly or implicitly.
3.6.1. Methodology and algorithm use. Table 3-17 gives a list of issues that should be investigated as part of the RTP QA program.
3.6.2. Density corrections. The accuracy of the density corrections which are part of most dose calculation algorithms will be discussed in the next chapter. However, a number of operational issues related to inhomogeneity corrections are part of this discussion on the mechanics of dose calculations ͑Table 3-18͒. 3.7.1. Dose display. Analysis of displays of the dose distribution, particularly in association with the anatomical data, is one of the major ways that physicians and planners make decisions about how the treatment plan should be optimized. A series of issues is listed in Table 3-19. For all tests, it is important for the user to be aware that correctness of dose refers to agreement of the display with calculated, not measured, dose. Agreement between calculations and measurements is discussed in Chap. 4.

Plan evaluation
Tests should be performed first for single beams, then for one or more simple multiple field configurations. Similarly, brachytherapy tests should be performed first with a single source, then with multiple sources. The user should be aware that RTP systems often calculate point doses independently from 2D and 3D dose distributions, therefore these methods Incorrect divergence leads to selection of wrong field sizes or aperture shape.
Non-axial divergence Test intersection of divergent beam and aperture edges with sagittal, coronal, and oblique slices. For systems that are not fully 3-D, there may be 2-D limitations in the projections which must be taken into account.
Incorrect divergence leads to selection of wrong field sizes or aperture shape, especially if 3-D effects are not completely understood.
BEV/DRR displays • Verify projection of contours/structures defined on axial slices into BEV-type displays. Compare with the grayscale images for DRR displays. This is most easily done with a simple phantom containing only a few internal structures. • Verify projection of divergent beam and aperture edges.
• Check at several different SSDs and projection distances.
Incorrect projections lead to selection of wrong aperture shape, especially if 3-D effects are not completely understood.

3-D displays
• Verify that apertures defined on 2-D planes are correctly projected in 3-D. • Verify that the relationships between structure and beam and aperture edges are correct. • 2-D limitations of the system must be considered ͑e.g., a 2-D system may not correctly display divergence in the third direction͒.
Incorrect projections lead to selection of wrong aperture shape, especially if 3-D effects are not completely understood.
Patient and beam labels • Verify patient orientation with respect to beam and orientation annotations. • Verify correctness of orientations and annotations for machine position views or icons associated with 2-D or 3-D displays.
Incorrect labeling can mislead treatment therapists or physicians.
may not exactly agree. Any differences should be documented. 3.7.2. Dose volume histograms. The use of dose volume histograms ͑DVHs͒ is an important part of modern treatment planning. Care must be taken when designing tests for this function, since the simple dosimetric and anatomic models which would be easy to use are often prone to various grid alignment type errors. 39 Issues to be tested or checked are listed in Table 3-20. 3.7.3. Use of NTCP/TCP and other tools. Modern planning systems sometimes include calculations based on nor-mal tissue complication probability ͑NTCP͒ and tumor control probability ͑TCP͒ models to aid in evaluation of competing treatment plans. If these capabilities are used for clinical planning, it is essential that they be included in the QA program. Note that many of the parameters of NTCP and TCP models, and in fact the models themselves, are not wellknown, and may be the subject of significant controversy. The verification checks used for NTCP/TCP calculation functions should ͑1͒ verify the correct implementation of the model; and ͑2͒ verify values of the parameters which the physicians and physicists expect to use. It is also desirable to verify that the clinical ''predictions'' of the model are in agreement with the expectations of the physicians interpreting those values, but this is clearly an area in which the physician's clinical judgment cannot be ignored.

Composite plans.
In some planning systems, it is possible to add ͑and/or subtract͒ 15 dose distributions from different plans in order to create a composite dose distribution which represents the entire treatment course for the patient. This ''composite plan'' may often be the plan which is evaluated for dose, complication probability, etc. In addition to checking all the input data for these composite plans, other issues include: • Dose prescription input for each component plan. • Availability of fractionation ͑bio-effect͒ corrections. Calculation validity logic Evaluate system rules for recalculation of dose distribution when changes are made in anatomy, beam definitions, beam weights, or normalization. Often, only the affected beam͑s͒ will be recalculated.
Incorrect logic will either 1͒ waste valuable time and resources; or 2͒ leave an invalid dose calculation for incorrect interpretation.

Dose calculation algorithm selection
Verify that default algorithm selections are appropriate, and that the selected algorithm is the one actually used.
If more than one algorithm is available, most likely the different algorithms are intended for specific purposes.

TABLE 3-18. Density Correction Issues
If the density corrections are turned on or off, this should force a new dose calculation with or without the corrections, respectively.
Some RTP systems allow the use of either a CT-based density distribution or one based on assignment of bulk densities. In each case, the user must confirm that the correct density distribution is used and that it is appropriately documented in the plan datafiles and hardcopy output.
The CT number to Hounsfield Unit conversion is machine and vendor dependent and can also be dependent on the CT calibrations. These conversions should be the subject of routine checks. 17 Proper functioning of tools which display relative electron density at a point should be verified.
• Interpolation of individual plan dose distributions onto a common grid. • Handling of plans with different dose units ͑e.g., % vs. daily dose vs. total dose vs. dose rate͒. • Accuracy of the addition/subtraction.

Hardcopy output
RTP system hardcopy output may include text information, plots of 2D dose distributions on arbitrarily oriented planes, DVHs, BEV, and DRR displays, and 3D displays of anatomy, beams, and dose. These various types of hardcopies are used to implement and/or document the treatment plan, so the accuracy of this information is critical. Table 3-21 lists the minimal information that should appear on the various types of output, and therefore should be confirmed in various situations. In addition, all output should contain the patient name and ID, the treatment plan ID, and a plan version number or time/date stamp.

Plan implementation and verification
Once a treatment plan has been completed and approved, the plan must be implemented. Implementation includes transfer of planning system treatment parameters to actual treatment unit settings; fabrication of blocks, compensators, and bolus from planning system information; proper use and positioning/orientation of beam modifiers; and proper positioning of patient. Since much if not all of this information is obtained via the planning system hardcopy output, testing of plan implementation should be carried out after verification of the hardcopy output from the RTP system ͑see Sec. 3.8͒.
3.9.1. Coordinate systems and scale conventions. Potential problems arise when the nomenclature and conventions used by the RTP system are not the same as those used by the department and/or by the treatment unit ͑see also Sec. 3.5.2͒. Some of the problem areas are listed in Table 3-22. The RTP QA program must check and document the way each parameter is represented ͑names, units, scaling, resolu-tion͒ in the RTP system and how it should be transferred to the physical treatment machine.
3.9.2. Data transfer. Numerous potential problems can develop during the transfer of treatment planning information from the RTP system to the paper chart, treatment machine, record/verify ͑R/V͒ system, or anywhere else. The issues listed in Table 3-23 must be considered as part of the QA for the planning process.
Correct transfer of parameters should be verified using a set of test plans varying from simple ͑e.g., single axial field͒ to complex ͑e.g., multiple non-coplanar and oblique fields͒. These plans should make use of all the methods used by the RTP system to indicate treatment machine information, location of treatment fields, correct phantom/patient information, correct collimator, table, and gantry settings, extended treatment distance techniques, and use and orientation of beam modifiers such as wedges, bolus, blocks, and compensators. For each test case, the user should implement the plan on the treatment unit using a phantom and then verify that the implementation is correct using visual inspection and portal films or images. 3.9.3. Portal image verification. 3D planning systems may contain the ability to import portal and simulator images and to register or at least compare those images with RTP system images such as BEV displays and/or DRRs. Some of the QA associated with this part of the process is described in the TG 40 report on a comprehensive QA program for radiotherapy. 2 QA for these features should address ͑at least͒ the issues listed in Table 3-24.

Brachytherapy issues
Many brachytherapy issues have been discussed in two recent publications: the NCI-funded Interstitial Collaborative Working Group report, 40 and the recent AAPM Task Group 43 report, 41 however, neither of these reports describe all of the QA which should be incorporated in a QA program for brachytherapy RTP. Many of these issues can be handled in parallel to those which address external beam RTP. However, we specifically describe some of the more important QA issues below.
• Brachytherapy source arrangements consist of individual sources, but they are often grouped as strings, trajectories, or applicators. One should confirm that parameter changes which should affect an entire group of sources are correctly made. • During commissioning, and also in later checks, each property or attribute described for each source in the source library should be verified ͑see Appendix 5͒. • Input, display, and plan optimization and evaluation test issues which are relatively specific to brachytherapy planning are listed in Table 3-25, and are further discussed in Appendix 5. • Clinical ''system'' tests or benchmark tests which confirm the entire process used for brachytherapy planning in each clinic are recommended by the Task Group to be performed for each basic kind of brachytherapy procedure ͑interstitial breast implants, GYN cesium applications, etc.͒. Further specifics for brachytherapy planning QA are included in Sec. 4.7 and Appendix 5.

Chapter 4: Dose calculation commissioning
Historically, most treatment planning quality assurance has been primarily concerned with dosimetric issues, particularly dose calculation verification. Most users of treatment planning systems, realizing the importance of dose calculations, have performed some tests of their systems to verify the agreement between calculated and measured doses. Furthermore, most published reports have concentrated exclusively on verification of 2D dose calculations, 5-10 although the recent work by Van Dyk 18 contains many other specific recommendations for dosimetric QA.
However, none of these studies addresses in detail the issues and techniques which must be applied to commissioning dose calculations in a modern treatment planning system. In this chapter, we present one consistent approach to the commissioning of dose calculations for treatment planning. Other organizations and methods are of course possible, but this approach is flexible and adaptable to a wide range of dose calculation and treatment planning situations.   Plan information transfer by hand into a paper chart or record/verify system is prone to significant transcription error rates. 70 Blocks and compensators are made using information from the planning system. The physical blocks and compensators should be verified for correct size, shape, and placement in the treatment field. Verification should be performed for simple and complex shapes of modifiers associated with orthogonal and oblique fields.
MLC shape information is often transferred to ͑or from͒ the treatment machine from the planning system. 63,68,38 This is clearly a critical quality assurance issue, and must be carefully verified and routinely checked.
Several QA considerations for automatic transfer of the complete set of plan information from the RTP system to the treatment machine or to its record/verify system have been discussed in detail in recent papers on a Computer-Controlled Radiotherapy System. 64,65,73

TABLE 3-24. Portal Image Verification Issues
Importing portal or simulator images directly from digital imagers or through the use of a laser digitizer system.
Image registration capabilities which allow geometrical registration of a particular portal or simulator image with the coordinate systems used for planning. The quality of the registration is often user-dependent, therefore QA procedures should be built into the clinical process to confirm the registration quality for each registration.
Image enhancement tools, since a number of these functions can actually change the way the image and/or registration are used elsewhere in the planning process.
Bookkeeping which ties various images to the appropriate plans and/or fields inside the RTP system must be confirmed.

Introduction
Several different terms ͑and issues͒ which figure prominently in the commissioning of dose calculations for RTP are defined below: • Input data checks. Most RTP systems require some input data. One of the most basic checks required in a dosimetric QA program is verification that the RTP system accurately reproduces the input data. • Algorithm verification. The purpose of algorithm verification testing is to demonstrate that the calculation algorithm is working correctly, 16 not to determine how well the algorithm predicts the physical situation. Calculational results may not agree well with measured data, but if the model on which the algorithm is based is inadequate, this is to be expected. Algorithm verification requires detailed knowledge of the dose calculation algorithm and its implementation, and may easily be beyond the testing capabilities of individual radiation oncology physicists. • Calculation verification. Calculation verification tests compare calculated and measured doses for the user's beam͑s͒ over a range of expected or representative clinical situations. These comparisons reflect the overall agreement ͑or disagreement͒ between the dose calculations from the RTP system, as handled by the user, and the data, as measured by the user. Disagreements revealed in these types of tests are not necessarily related to the software or the calculation algorithm, but may simply reflect anomalies in the system use and/or measured data. • Applicability and limits of the dose calculation algorithm. Some of the most important checks that can be performed on a dose calculation algorithm are those that investigate the limits of applicability of the algorithm. The user must understand the limitations of each algorithm so that dose calculations for clinical situations which press ''the edge of the envelope'' for that algorithm are either avoided or appropriately interpreted. These tests may be more extreme than is expected in clinical use. • Dose verification over the range of clinical usage. These checks are similar to the algorithm limitation checks described above, except that in this case the clinical limits of usefulness of the actual calculations are determined. Evaluation of the clinical situations for which the model is and is not adequate is necessary. With very complex 3D dose calculation algorithms which consider 3D inhomogeneities, conformal field shapes, intensity modulation, and various other complex dosimetric issues, there is a very large range of clinical usage that must be investigated.
The radiation oncology physicist should be aware of several basic dosimetric QA facts: • Most dose calculation verification tests traditionally in- Incorrect functioning of optimization and evaluation tools can result in sub-optimal or incorrect treatment. volve comparison of calculated doses with measured data for a range of clinical situations. As treatment planning in the institution becomes more sophisticated, the range of dosimetric testing should expand and will eventually become quite extensive. Identifying the various effects or situations to be tested, and defining the limits over which each effect will be tested, will help the physicist organize the testing. • Calculation verification tests generally fall into two categories: ͑1͒ comparisons involving simple water phantom-type geometries, which are usually easy to interpret; and ͑2͒ comparisons involving complex geometries ͑often with anthropomorphic phantoms͒ in clinically realistic situations, which are difficult to interpret, since uncertainties in measurements, errors in input data, parameter fitting, algorithm coding and/or design, calculation grid effects, and various other uncertainties are all incorporated into the results. Although these complex tests are critical for evaluating the overall system precision for particular calculations, their usefulness in explaining discrepancies is limited. • Often, in an attempt to minimize effort, some of the tests and measured data are used repeatedly to test multiple aspects of the planning system. When this is done, the tests should be designed to be as independent as possible, so that the appropriate analysis and actions are taken when necessary. • The comparison of calculation results and measurements is not a competition. The task of performing the measurements and parameter determination and calculation verification testing should begin by assuming that there are likely to be many errors and inconsistencies uncovered, and that these will have to be resolved by the whole team in an open, cooperative fashion.
The three following recommendations stress the importance of dosimetric QA to all radiation oncology physicists, physicians, administrators, and dosimetrists who are involved with treatment planning systems: ͑1͒ The verification of external beam and brachytherapy dose calculations for clinical use is a very important part of RTP system commissioning. A comprehensive series of test cases must be planned, measured, calculated, compared, analyzed, and evaluated before any dose calculations are used clinically. ͑2͒ The particular test cases designed as part of the commissioning and QA programs for any particular institution depend on the RTP system involved, the way the system is ͑or will be͒ used clinically, and many other clinic and system-dependent factors. While most basic testing will be similar, optimizing the test procedure for each clinic is essential if the QA program is to be effective yet achievable in a modern sophisticated radiation oncology department. ͑3͒ Tools such as precise water phantom scanning systems, calibrated film digitizers, TLD readers, redundant detector systems, measurement phantom systems ͑including anthropomorphic phantoms͒ must be readily available to perform quality assurance. The effort required for this QA testing increases dramatically if the appropriate tools are difficult or impossible to access, so these systems normally must be maintained on-site at each clinic. A QA program for the test tools must be instituted for the QA tools to be effective.

Measurement of self-consistent dataset
Measurement of a self-consistent dataset is a fundamental part of commissioning and QA for a treatment planning system. A measured dataset is used initially as system input for modeling the institution's treatment beams and subsequently in calculation verification tests. For 3D dose calculation algorithms in particular, the basic data should be measured in a manner that adequately describes all of the dosimetric attributes of the beams or sources.
4.2.1. Self-consistency. The requirements for measured data at each institution will depend primarily on the needs of the RTP system for beam modeling and system QA. As a minimum, most systems require depth dose and beam profiles at one or more depths in one or more planes through the central axis for multiple open field sizes, as well as data for fields modified with wedges or other devices. Many systems will require more. In addition to the data necessary for beam modeling, data must also be acquired for calculation verification tests.
It is of primary importance to generate a self-consistent dataset. This means, for example, that all of the depth dose curves, axial and sagittal plane profiles, coronal plane profiles and/or 2D dose distributions and any other data, for a particular experiment, are all consistent with each other, and can be combined into one self-consistent dose distribution for that experiment. This can typically be achieved by acquiring a set of relative measurements which are then interrelated by a small subset of either relative or absolute measurements. 42 Recommendations for methods to assure dataset self-consistency are listed in Table 4-1.

. Methods for Obtaining a Self-Consistent Dataset
Design the measurements so that the data required to tie all the various separate measurements together are obtained during the same measurement session.
Make measurements over the shortest time span possible consistent with obtaining representative dose measurements.
Use the same equipment and procedures for all similar measurements.
Relate measurements made with different measurement methods to each other. Ideally, some of the measurements should be repeated with an independent, preferably different type, dosimeter.
Use a reference chamber to account for output fluctuations when making measurements with a scanning ionization chamber.
Periodically repeat base measurements, such as the dose at 10 cm depth for a 10ϫ10 cm 2 field, to monitor the consistency of the machine output and the measuring system. Note that this may involve use of temperature equilibrated water and/or monitoring the barometric pressure, in certain situations.

4.2.2.
Data analysis, handling, and storage. As discussed above, the measured data ͑depth dose curves, profiles, 2D distributions, etc.͒ must be coalesced into a single selfconsistent dataset. This involves careful data handling, analysis, and renormalization, much of which may be performed with the RTP system: • Postprocessing. All measurements must be converted to dose, either relative or absolute. • Smoothing. Raw data often should be smoothed to remove artifacts of the measurement technique. Care must be taken to ensure that the smoothing is not done too aggressively, smoothing out real dose variations. • Renormalization. All data ͑depth doses, profiles, etc.͒ should be renormalized to make the dataset selfconsistent. • The task group recommends that vendors of RTP systems provide sophisticated data input, storage, analysis, renormalization, display and other capabilities inside their RTP systems 15,43 to help physicists utilize the measured data.

Data input into the RTP system
All treatment planning systems require the entry of data associated with specific treatment machine beams and brachytherapy sources. The data required are specified by the vendor of the system and can vary substantially depending on the type of dose calculation algorithm used by the system.
The task group strongly recommends the following: • Vendors should specify the data required by their system in the system documentation and make this information available to users before purchase of the system. • Only data that has been measured on the specific treatment machine being commissioned into the RTP system should be used, unless it is known that the treatment units in question have exactly the same characteristics. Other beam data or ''representative'' data provided by an accelerator vendor ͑or by others͒ should never be used for dose calculation verification testing. Generic dose distribution data, such as depth-doses and profiles, are only useful for self-consistent checks of the software. • A data log book for documenting data acquisition, data handling, renormalization and/or data smoothing procedures used in preparation and analysis of the beam data should be maintained. The source of the data, the date that the measurements were done and the person or persons involved in the measurements should be logged. The log book should be maintained for the lifetime of the treatment planning system.

General considerations.
The kinds of data input into any particular RTP system for dose calculation, and the methods used for that input, are quite varied. For each particular situation, therefore, the following issues should be addressed by the user: • A clear understanding of the data required by the system is necessary before purchase. Often, the physicist will need to request this information specifically, be-cause it is not always clearly indicated in the manufacturer's prepurchase information. Knowledge of the beam data requirements will allow an accurate assessment of the amount of new beam data needed. • A complete review of the currently available data should be performed. The existing beam data may have been obtained several years earlier, may not be in the correct format, may not be documented adequately, or may be irrelevant to the new RTP system. • The data required by the system may have to be renormalized or reformatted, necessitating modification of the measured data before it can be used. • If monitor unit settings will be generated by the RTP system, then the monitor unit calculation algorithm and methodology should be compared to the present system used in the department. Any differences between the methods must be thoroughly understood and resolved before the new system is used. • At least one complete set of photon beam, electron beam, and brachytherapy source data should be available for entry when the system is installed. Vendor training can then include data entry and beam parameter fitting processes. • Additional beam data ͑more than is required by the RTP system͒ will always be needed. These data should be carefully prepared and handled as part of the verification dataset.

Computer transfer of data from a computercontrolled water phantom.
Direct transfer of data from a computer-controlled water phantom system ͑WPS͒ to the RTP system is the most common method of inputting data into the RTP system. The task group recommends that vendors provide information on the required data and/or file structures to users and WPS vendors, so that direct data transfer is available from each WPS to each RTP system. Data transfer issues which must be considered by the physicist are listed in Table 4-2.

Manual data entry.
If computer-based data transfer is not possible, manual entry of the data into the RTP system may be necessary. This is usually accomplished using both the keyboard and the digitizer tablet. For manual data entry, the following should be considered: • Digitizer accuracy should be tested before data entry begins. This testing should include determination of the inherent accuracy with which data can be entered using the digitizer. Significant data entry errors, particularly in low dose regions, may result because of digitizer inaccuracies. • Special attention should be paid to the digitization of data plotted on nonstandard scales. • Keyboard entry of data should be checked carefully, particularly for typographical errors.

Verification of input data.
After data are input into the RTP system, the user must verify that the data were input correctly.
• 2D algorithms are usually based directly on input data.
Data entry can be verified by generating dose distributions for the field sizes used for input data and comparing with the input data. • Many 3D dose calculation algorithms, such as convolution algorithms, 44,45 are much more complex and not directly based on input data. For these types of algorithms, much of the input data is not directly related to any measured dose distributions, but rather to machineindependent calculation results. 46 In any event, all input data should be verified, preferably independently by two people, and all discrepancies must be resolved, or at least well-characterized and understood, since they will affect all further comparisons between calculations and measured data.

Dose calculation algorithm parameter determination
For many systems, once the beam data are input into the RTP system, beam parameters that fit the beam model to the measured data must be determined. The beam model parameters that are selected will directly affect the accuracy of the dose calculations and must be determined with great care. Although the details of the parameter determination process are highly system dependent and beyond the scope of this report, documentation of the results of this process is an important issue addressed below. The user should: • Review any beam model data files or similar data used by the calculation algorithm and verify that the final parameters are correct.
• Document the dose calculations, fits and other checks that were used during the process of parameter determination and the results of those activities. • Summarize data sources, methods used for parameter determination, the presumed accuracy or sensitivity of the parameters, and any other salient information. This information should be stored in the RTP system log.

Methods for dosimetric comparison and verification
Dose calculation verification tests compare calculated and measured dose distributions. The standard method of comparison for 2D dose distributions consists of overlaying hardcopy plots of measured and calculated doses in the form of cross-beam profiles, depth doses, or isodose distributions. For quantitative comparisons of entire 3D dose distributions, more sophisticated techniques, such as those listed in Table  4-3, are also needed to perform the analysis.
To use these tools, the RTP system must be able to handle 1D, 2D, and 3D measured dose distributions. Although this kind of functionality has been demonstrated, 15,43 it is not yet

. Water Phantom System Data Issues
Data exchange compatibility between the WPS and the RTP system should be determined prior to purchase. Often the WPS or RTP system vendor will provide exchange software.
File naming/labeling conventions should be decided before data is taken or transferred. Files should be uniquely identified on both systems. Documentation for each WPS data file should include: • filename in the WPS • filename in the RTP system, if different • date of measurement • machine parameters such as beam energy, field size and shape, gantry/collimator angle, beam modifiers • phantom setup, including any special features ͑e.g., an air inhomogeneity͒ • 3-D coordinate system of the WPS and its relationship to the beam coordinate system • scan parameters such as scan direction, scan mode, depth/location of scan Records should be kept in the data log book in addition to information stored within the WPS.
The data exchange link should be initially tested with a small test data sample. Verify that format modifications are made correctly and that no substantive changes are made to the measured dose values.

FDD and TPR tables of differences
Tables of the differences between calculated and measured FDD ͑fractional depth dose͒ or TPR ͑tissue phantom ratio͒ values as a function of field size and depth are useful for analyzing overall data agreement. 14,78 Statistics calculated using the difference table are also useful.

2-D isodose lines
In addition to isodose curves overlaid on axial planes, overlays on sagittal and coronal planes and 3-D axonometric displays 14 are useful for 3-D dose comparisons.

Colorwash dose displays
Colorwash display can aid in visualizing dose differences between calculations and measurements. Some systems allow interactive colorwash display of dose ranges on planar or axonometric displays.

Dose difference displays
Graphical display of dose difference distributions in 1, 2, or 3 dimensions, generated by subtracting measured and calculated dose distributions, can be useful for highlighting small differences in the distributions. 14

DVH analysis
Results of the dose comparison throughout the 3-D volume of interest can be summarized by making a histogram ͑DVH in 3-D͒ of the dose difference distribution. 15,71 Distance maps A distance map showing the distance between particular isodose lines in the measured and calculated distributions is particularly useful in high gradient regions. 81 available in many commercial RTP systems. The task group recommends that vendors include all of these types of extensive data analysis and display features in their RTP systems. 4.6.1. Introduction. There are a number of different ͑but valid͒ approaches to designing and organizing the experiments and calculation verification checks to be used during commissioning of a particular calculation algorithm or individual beam parametrization. In this section, one approach is outlined. We recommend that the radiation oncology physicist analyze the clinical needs, dose calculation algorithms, treatment machines, and treatment techniques specific to his/ her clinic and then modify this outline to fit that particular situation.

External beam calculation verification
Each kind of calculation test should be clearly identified as an input check, algorithm test, or calculation verification check. In some situations, one or more tests may be used to satisfy multiple needs. For example, it is possible for one particular test to be analyzed from two different standpoints: ͑1͒ whether or not the algorithm is working correctly; and ͑2͒ whether or not the result is clinically acceptable.
For each test, the radiation oncology physicist should know how well the calculations are expected to work. This is important so that decisions can be made about whether the agreement ͑1͒ is the best that can be expected; ͑2͒ can be improved; or ͑3͒ indicates the existence of a problem. This determination depends on knowledge of the physics of the algorithm and its implementation, knowledge of the user's parametrization and use of the model, and knowledge of the accuracy of the data against which the calculations are compared.
4.6.2. Required and/or achievable accuracy. The dosimetric accuracy required or achievable for treatment planning purposes has been the subject of much discussion. Cunningham 47 and others have indicated that an overall accuracy of 5% in dose delivery may be a good goal on radiobiological grounds. He concludes that an accuracy of 2.5% may be achievable in beam calibration, 3%-4% may be possible in relative dose calculations and perhaps 3%-4% in treatment delivery, resulting in between 5% and 6% overall accuracy. The Canadian group led by Van Dyk spent a great deal of effort to determine ''Criteria for Acceptability'' for a whole series of dosimetric situations. 18 Their suggestions are quite useful when applied to the situations considered in their report and may be a good guide for the user. However, each planning system, institution, and dosimetric situation will have its own requirements, capabilities, and limitations. There is an extremely wide range of accuracies of which various calculation algorithms are capable, and it is important that the user determine the accuracy which can be expected in his/her particular implementation and situation.
In this report, we propose a method for characterization of the accuracy of a dose calculation method similar to that used by Van Dyk et al. 18 For analysis of agreement between calculations and measurements, the dose distribution due to a beam is broken up into several regions, illustrated in Fig.  4-1: • The inner beam ͑central high-dose portion of the beam͒ • The penumbral region ͑0.5 cm inside and outside each beam/block edge͒ • The outside region ͑outside the penumbra͒ • The buildup region ͑from the surface to dmax, both inside and outside the beam͒ • The central axis • Absolute dose at the beam normalization point These regions should be analyzed separately, so that reasonable characterization of the agreement between calculations and data can be performed without combining the regions of large dose gradients with those which have small gradients. Table 4-4 illustrates the suggested analysis and includes examples of acceptability criteria. These criteria are only an example of the kinds of variations in dose calculation agreement with measurements that might be expected for a sophisticated dose calculation algorithm. For each situation, the accuracy of any particular algorithm or dataset may affect these expectations. The radiation oncology physicist in each institution must evaluate the expectations for each situation and determine the criteria to which the particular beam and algorithm will be compared. The criteria shown as examples in Table 4-4 are based on the collective expectations of the members of the task group and are not to be used as goals or requirements for any particular situation.

Photon calculation verification experiments.
A general photon calculation test plan is described in detail in Appendix 3. This plan consists of a series of tests that range from basic checks of depth dose curves to much more sophisticated dose calculation situations including heavily blocked fields and inhomogeneous phantoms. The radiation oncology physicist should evaluate the importance of each class of tests and prioritize the verification checks so that the clinically most important checks are performed first. This listing is intended to act as an example, rather than a prescription, for the testing that should be performed. beam dose calculation algorithm. A subset of these measurements is also required for initial commissioning of each particular electron beam.

Brachytherapy calculation verification
Brachytherapy dose calculation verification should be approached with many of the same concerns as that for external beam calculations. Here, however, the situation is often more straightforward than for external beams. Reasons include: • Standard sources with universal characteristics are used. • Most dosimetric parametrizations are obtained from the literature, rather than individual measurements. • Calculation algorithms are often quite simple. • Often, more than one calculation model is available to the user. Great care must be exercised to determine the correct coefficients for use in these models, as they are source type dependent. • Some calculation complexities ͑e.g., the effects of bone and air inhomogeneities or of applicator shielding͒ are typically ignored. Note, however, that when these effects are ignored, the user must understand the implications of those approximations. A number of published reports contain a large amount of useful information relevant to forming a QA program for brachytherapy treatment planning, and we recommend the review and consideration of the following references: • The NCI-funded Interstitial Collaborative Working Group report. 40  • Various other published books and articles on brachytherapy QA. [49][50][51][52][53] Appendix 5 includes examples of a number of brachytherapy test procedures, including tests of dose calculations and source localization methods. The task group recommends that a dose calculation verification test should be performed for each type of brachytherapy source used, and that each method of source localization also be checked.

Absolute dose output and plan normalization
How each treatment plan is normalized is one of the most critical parts of a treatment planning system, since it determines how the monitor units should be calculated, which in turn determines the actual doses delivered to the patient. Study of all the different methods of plan normalization which are available in the RTP system is critical to confirm that ͑1͒ they work as expected; and ͑2͒ the treatment delivery system in the department uses them correctly. This section deals primarily with external beam planning, while parallel issues in brachytherapy such as dose specification criteria and dwell time normalization are addressed in Appendix 5.
4.8.1. General guidelines for QA for normalization and MU calculation. The first and most basic recommendation of the task group on this subject is the following: A complete check of the entire treatment plan normalization and monitor unit calculation process must be performed for a series of different kinds of plans. Each plan should be normalized in a number of different ways, and for each method, the user should utilize the available methods to calculate the monitor units required to treat the plan. The different methods should then be compared to assure that (1) the correct monitor units and doses are always achieved; and (2) the results of the different methods are the same (within tolerance).  Study of the normalization/MU calculation process should be performed in each clinic. Clinics should of course attempt to standardize this process, to minimize the complexity and possibility for misinterpretation of input data or results. The radiation oncology physicist should attempt to ensure that the process will perform as expected for any likely combination of situations, even in the face of deliberate errors or misuse of the system functions. A careful analysis of the possible hazards associated with this aspect of the system should be performed at each institution, since plan prescription, normalization, and monitor unit calculation methods vary quite a bit from institution to institution. A detailed knowledge of the design, methodologies, algorithms, and safety checks which are part of the RTP system design is required. The task group recommends that vendors provide enough information so that the user can carry out such an analysis of the normalization/MU calculation process.
The task group also recommends that vendors incorporate into the RTP system design automated checks of geometric and dosimetric information to be performed during beam and plan normalization. 13 Such checks can detect not only software errors but also incorrect system use and errors in judgment in choice of normalization points and/or methods. Error or warning messages generated by the system can help users avoid inappropriate or incorrect normalization situations that might lead to incorrect treatment.

Verification of the steps in the process.
In order to determine the monitor units required to give a prescribed dose to a particular treatment plan, various steps in the planning process are involved, including: • The relative beam weights are set as part of the plan technique. • The overall relative plan normalization method is chosen for the treatment plan. • The total dose and fractionation are prescribed by the physician. • A particular prescription point or isodose level is chosen by the physician. • Monitor units are calculated so that the prescribed dose is delivered. Each step in this process should be carefully studied and appropriate testing carried out.
Relative beam weights. In order to add the doses from several beams together, some method of determining the relative beam weight of each beam is used in each RTP system. This relative weight may be the dose defined at the beam normalization point, the relative number of MU for the field, or may be related to the energy fluence. Typically, the RTP system calculates the relative dose to be delivered to the normalization point ͑beam norm-pt͒ chosen for each beam ͑in older systems, this point may be at d max on the central axis for each beam, or it may be the isocenter for an isocentric plan͒. In more complex systems, the beam normalization point may be different for each beam, since d max or isocenter may not always be appropriate. After the point is identified, some relative dose ͑called the beam weight͒ is delivered to this point for each beam, and then individual beam dose distributions are summed to yield the dose distribution for the plan. Table 4-5 lists some beam weight issues to be checked. Comparable questions must be asked for any of the methods used for beam weights inside the RTP system.
Overall relative plan normalization. After the relative dose distribution is obtained, most RTP systems allow the normalization of the entire distribution to give a specified dose at some defined point ͑the plan normalization point, or plan norm-pt͒. The value at the plan norm-pt might be in terms of relative dose, absolute dose for one fraction, or dose for the entire treatment. Testing issues are listed in Table  4-6.
Isodose level chosen for dose prescription. A common use of plan normalization features is to normalize the plan to 100% at the isocenter of the plan, and then to choose a mini- mum isodose line ͑isodose surface͒ that encloses the planning target volume ͑PTV͒, and to use this isodose level as the prescription dose. It is critical that this part of the prescription process be included in any monitor unit calculation methods. Alternately, the same result can be accomplished by increasing the dose at the plan normalization point ͑e.g., to 105%͒ so that the 100% isodose level covers the PTV.
Calculation of monitor units (MU) to deliver prescribed dose for a plan. QA for the calculation of monitor units for a particular plan is of course very dependent on the methods used inside the RTP system and any external MU calculation program or techniques, if used. It is here that all of the questions about exactly how the planning system calculates and displays dose to the beam normalization points and the plan normalization point become most important. The MU calculation methodology must be completely tied to the methods of normalization used inside the RTP system, or incorrect doses delivered to the patient will result.

Clinical verification
A reasonable final check on the systematic behavior of the RTP system and the RTP process includes a series of clinical tests. These tests should be designed to check most of the important functions involved in planning through performance of the entire planning process, including dose prescription and the final dose distribution and monitor unit setting calculations. Commissioning data and/or special measurements made in appropriate phantoms can be used to verify the dose and MU results. Test cases with graded levels of complexity can be selected, for example: • Square manual contour with several blocked fields. • Tangential breast plan with manual contour. • CT-based plan for phantom with density connections. • 3D CT-based plan for phantom involving nonaxial and noncoplanar fields with conformal blocking.
Brachytherapy planning should be tested similarly, with plans involving single and multiple source configurations and different source strength specification and source localization methodologies. Such a series of clinical tests can also be used for routine testing, dosimetric checks, and review of actual RTP activities ͑when appropriate͒.

Chapter 5: Periodic quality assurance testing
In this chapter we discuss testing that should be performed periodically at specified time intervals. This testing is not associated with commissioning the new RTP system or accepting or commissioning a new version of the software. Just as the AAPM Task Group 40 report 2 suggests time intervals for many different parts of the QA process in a radiotherapy department, this task group also suggests various tests that should be performed at certain intervals. The periodic QA needs of the RTP process should be considered before the initial system commissioning because it may prove possible to use a subset of the commissioning tests for these routine tests, avoiding the need to repeat the test design process.
All components of the RTP system and RTP process need to be considered when developing the QA program, although some may not require much periodic testing. For a software device such as a RTP system, one must be concerned about data files, integrity of the software executables, failures or problems in hardware peripherals and general system configuration, as well as the process that uses the software. The main aims of a routine periodic QA program for the RTP system include the following: • Confirm the integrity and security of the RTP data files that contain the external beam and brachytherapy information used in dose and monitor unit calculations. • Verify the correct functioning and accuracy of peripheral devices used for data input, including the digitizer tablet, CT, MR, video digitizer, simulator control system and devices for obtaining mechanical simulator contours. One must separately consider the devices themselves and the networks, tape drives, software, transfer programs, and other components which are involved in the transfer of the information from the device to the RTP system. • Check the integrity of the actual RTP system software. • Confirm the function and accuracy of output devices and software, including printers, plotters, automated transfer processes, connections to computer-controlled block cutters and/or compensator makers, etc. Ongoing QA for several additional aspects of the RTP process has been described in the 1991 ACMP symposium on Quality Assurance in Radiotherapy Physics. 12,17 Recently, two articles 2, 18 have made similar recommendations regarding the frequency of routine reliability testing. Using this information as a basis, this task group recommends periodic testing of various parts of the RTP system as specified in Table 5-1. Commercial manufacturers often make their own recommendations regarding ongoing QA of their planning systems. Each radiation oncology physicist

. MU/Normalization Process Issues
Each permutation in types of beam normalization, plan normalization, isodose level prescription and MU calculation must be verified.
A series of standard clinical protocol cases should be planned, and MUs calculated for each field. The doses actually delivered by these fields and plans can then be verified independently, either by measurement or through use of standard MU calculation data. Note that hand MU calculation methods are likely to be less accurate than modern RTP system dose calculations and so cannot necessarily be used as the gold standard for complex cases.
For these standard cases, as many permutations of the normalization/MU calculation process should be used as possible, and derived MUs should be compared. These results should be analyzed not only to detect errors or misinterpretation, but also to obtain the approximate accuracy of the different methods which could be used in the same case.
If possible, the RTP system computations should be checked using a hand dose calculation method ͑although as noted above, this will not always be possible͒.
should review all the recommendations and develop a program of periodic testing that will match the planning system characteristics and its user base. The frequency of testing of each specific feature of the RTP system should depend on how that feature is used in the clinic and how critical that feature is from a safety point of view.
One of the recommendations above involves recommissioning checks for each beam as required after major repairs, tuning, or other changes to beam parameters or machine. One possible recommissioning protocol is shown in Table  5-2. The amount of work involved can vary from a few hours to many days work, per beam, for a complex 3D dose calculation algorithm. Note that the annual QA of each treatment machine, recommended by AAPM Task Groups 40 2 and 45, 54 should be performed in conjunction with the QA for the RTP system use of that machine in order to minimize the amount of new work which is necessary.
The more different treatment machines and beams there are ͑involved in the QA program͒, the more time will be required for QA testing, so a systematic review spaced over the entire year should be considered. Careful prioritization of issues specific to each clinic is critical, otherwise the ongoing QA work on the system will become quite time intensive and difficult to fund or accomplish.
A series of reviews and training sessions is recommended to be included as part of the periodic QA program, as listed in Table 5-3.

Chapter 6: QA as part of the daily planning process
Even after all quality assurance tests for the RTP system and process have been developed, there is still a major segment of the QA process to be considered. Experience with complex treatment planning and its associated QA has led to the conclusion that the most important part of the QA program is neither the dosimetric or nondosimetric tests; it is the design and implementation of a clinical planning/delivery process that incorporates QA elements to comprehensively check all aspects of the planning and delivery for each patient and each plan.
There are several reasons for carefully designing the planning/delivery process to include QA checks: Monthly CT data input into RTP system Review the CT data within the planning system for geometrical accuracy, CT number consistency ͑also dependent on the QA and use of the scanner͒, and derived electron density.

Problem review
Review all RTP problems ͑both for RTP system and clinical treatment planning͒ and prioritize problems to be resolved.

Review of RTP system
Review current configuration and status of all RTP system software, hardware, and data files.

Annual
Dose calculations Annual checks. Review acceptability of agreement between measured and calculated doses for each beam/source. Data and I/O devices Review functioning and accuracy of digitizer tablet, video/laser digitizer, CT input, MR input, printers, plotters, and other imaging output devices. Critical software tools Review BEV/DRR generation and plot accuracy, CT geometry, density conversions, DVH calculations, other critical tools, machine-specific conversions, data files, and other critical data.

Variable
Beam parameterization Checks and/or recommissioning may be required due to machine changes or problems. Software changes, including operating system Checks and/or recommissioning may be required due to changes in the RTP software, any support/additional software such as image transfer software, or the operating system.
• A modern planning system may be the result of 30-50 person-years of work and may consist of as many as a million lines of code. It is well known in the software engineering world that even well-designed and implemented software systems still usually contain at least one software error in every 100-1000 lines of code. 16 Therefore, there will always be software errors, some of which will be significant in certain clinical situations. • Modern RTP systems contain complex data structures and algorithms and offer a great deal of flexibility. It is impossible to perform exhaustive testing on any one section of such a system, let alone the entire system. Therefore, other QA tools must be used to help assure the correct behavior of the system. • The entire treatment planning/delivery process involves a complex series of procedures and decisions. Ongoing QA for the process can ensure that the user makes correct decisions and uses the planning software correctly. • Since the optimal way to use patient information or to design a treatment plan for a particular patient may not be obvious, many variations of standard planning procedures may be used. In fact, new techniques that have never before been considered will likely appear during the planning process. Continual QA of the planning process will help confirm the reasonability of these new developments and flag those plans that may require additional verification checks before implementation. Several examples of ways to incorporate quality assurance into the daily treatment planning process are listed below in Table 6-1. Some of these reviews are also recommended by AAPM Task Group 40. 2

Chapter 7: System management and security
In earlier years, the RTP system was a stand-alone computer system, sometimes based on proprietary hardware. Now, however, most modern planning systems consist of standard computer hardware systems and system software, standard peripheral equipment, and the RTP software. The RTP system can be a complex system consisting of networked or clustered graphics workstations, servers, and peripheral devices, all of which require sophisticated system management to keep the system operating. Correct management of these systems must be a part of any overall QA program for treatment planning.
Machine and patient data stored on the RTP system computer should be considered to have the same status regarding maintenance and security as this data when it is stored in a logbook or patient's chart. For the physicist in charge of Make the dataset used for the RTP system recommissioning as similar as possible to the dataset that is remeasured as part of the annual linear accelerator recommissioning.
Store these standard data, related treatment plans and other necessary information together to minimize the time spent hunting for data, creating new test cases, etc.
Use a checksum program or other software analysis tool to confirm the constancy of the data, datafiles, and other related information used in recommissioning tests.
Verify new tables of TPR, TMR, or FDD data at the standard SSD.
Verify the phantom scatter factor, collimator scatter factor, wedge and tray factors, and any other factors which contribute to monitor unit calculations performed inside the RTP system. Verify off axis beam profiles for open and wedged fields.
Use a standard set of square and rectangular field sizes to reproduce isodose curves.
Verify a subset of FDD, profile and isodose curve data at two other clinically relevant SSDs.
Calculate the dose for standard square and shaped fields using irregular field entry methods ͑if different than the normal mode of operation͒.
Verify the dose distribution from blocked fields for several standard block shapes for each energy.
Verify the dose distribution from MLC-shaped fields for several standard shapes for each energy.
Verify the standard SSD depth dose and output factors for each electron energy and applicator.
Verify the dose profiles and isodose curves for a standard set of applicator sizes ͑small, medium, and large͒ for each electron energy at the standard SSD.
Verify the dose distribution from shaped electron fields for several shape/energy combinations.
Review the results from density correction algorithms for each photon and electron energy, if using an energy dependent density correction algorithm.
Review CT-based and bulk density calculations for a selection of energies and anatomical models.
Perform a series of procedural checks of monitor unit calculations based on treatment plans.

Topic Description
Staff training Each clinic should develop a procedure for training ͑and re-training͒ staff in the use of its specific RTP system and process.
Clinical plan review A formal review of clinical plans should be developed, with a specific set of parameters to be reviewed. A planning library with examples of planned treatments can be useful when questions arise regarding particular plans.
Error review A formal review of any errors found should be presented to representatives of the entire staff.
QA program review Documentation of the quality assurance program and the continued efforts to improve the planning process should be part of the institution's efforts to reduce patient treatment errors.
treatment planning to ensure that such information is secure and adequately maintained, care must be taken so that the data are not corrupted, lost, or used inappropriately. The following guidelines and responsibilities can aid the physicist in developing a set of procedures for management and security of the RTP system.

Management personnel
Overall management of treatment planning includes two distinct areas of responsibility: ͑1͒ overall responsibility for all aspects of treatment planning and the RTP system; and ͑2͒ technical responsibility for the hardware and software of the RTP system. Typically, but not necessarily, these responsibilities may be handled by two different people.
7.1.1. Responsible physicist. The ''responsible physicist'' or ''treatment planning system manager'' should be a radiation oncology physicist with a large amount of experience in the field of treatment planning. This individual is responsible for the overall maintenance, use, and security of the planning system. Decisions about release of new versions, quality assurance testing needs, commissioning, clinical use, and the resolution of planning problems are all made by the planning system manager. This person also supervises the activities of the computer systems manager when they affect the planning system and computer͑s͒.
7.1.2. Computer systems manager. The management of modern computer systems, such as those used for RTP, requires experienced computer systems management person-

Test Reasons
Multiplanar reconstructed images • Inconsistencies in the image dataset will produce inconsistencies or artifacts in the reconstructed images. • Beam orientation and patient anatomy which is difficult to visualize on axial images is often easily seen on non-axial planes. • Contours cut from 3-D structures onto reconstructed images may show inconsistencies or problems in 1͒ the original axial contours; 2͒ the 3-D structure; 3͒ the way the structure was identified on different imaging studies.

3-D surface displays
Surface displays help verify that component 2-D contours are consistent and realistic.

Dataset registration review
The responsible physician should review the accuracy of registration of multiple datasets and the transfer of information such as tumor or critical normal structure delineation between datasets.
Target definition checks Projection of a CT-defined target volume onto BEV images, which are then compared to simulator films, can help physicians and staff check target location, patient positioning, and beam orientation.

Point dose calculations
Hand calculations of dose to the prescription point and/or normalization point help verify correct delivery of dose to the patient.

Plan visualization and documentation techniques
Plots in appropriate non-axial planes can be used to show beam, wedge, and block orientations for non-axial beams, electron cutout accuracy.

Treatment plan review
The physician and a second treatment planner/physicist should review the plan, including all treatment parameters, before implementation.
Monitor unit review Monitor unit calculations should be reviewed by a second physicist, preferably before treatment starts, but certainly before the third fraction or 10% of the dose has been delivered.
SSD Checks SSD to the central axis of each treatment field should be measured during simulation and periodically during treatment and compared to that used in the treatment plan.
External beam plan implementation review • The physicist or therapist should confirm before the first treatment that all treatment parameters were transferred correctly from plan to patient chart and/or record and verify system. • Periodic port films or port images help verify the correct positioning of the patient and correct orientation of the blocks. • Consider feasibility of treatment plan ͑re: interference or collision of machine gantry with table and/or patient and/or immobilization devices͒.
Brachytherapy plan implementation review • The physicist or therapist should confirm before the brachytherapy sources are placed into the patient that all source and plan information was correctly transferred from treatment plan to the treatment documentation or patient chart. • Dose calculations and prescription should be verified as accurate and appropriate before treatment begins. • Confirmation of source location and loading, if possible, should be performed as soon after loading as possible.
nel, even for a PC-based system. The systems manager is in general responsible for system hardware and software maintenance, backups of planning system data and patient information, maintenance of the relevant computer networks and other kinds of intercomputer communication, security of the computer systems, and other such tasks. The computer systems manager must work under the general supervision and responsibility of the responsible physicist, so that computer system management is in agreement with the general needs of the clinical use of the planning system. The computer systems manager should be knowledgeable about the most commonly used operating system commands. Even with the turnkey treatment planning systems which are in use in many radiation oncology clinics, a working knowledge of the computer's operating system commands is needed to maximize the usefulness of the treatment planning computer. File management tasks such as copying files and performing backups can often be simplified when performed within the context of the computer's operating system. In addition, the systems manager should become familiar with all software present on the treatment planning computer, including that which is not necessarily part of the treatment planning system.

Computer system management tasks
One of the most important tasks involved in computer systems management is hardware and software maintenance for the computer system. Hardware maintenance may involve service contracts or dedicated hardware service personnel. The systems manager will advise the responsible physicist on the necessity and economic feasibility of a service contract, and is usually the individual authorized to contact the service organization for unscheduled repairs as well as for scheduled maintenance. Software maintenance is in many respects a much more complicated task. Decisions about upgrading new system software are not trivial, as it is possible for the planning software to have some level of incompatibility with the new system software, potentially causing program errors or other problems. The computer system manager should additionally monitor disk space, user accounts, memory, and other resources.
Another task of the computer systems manager is acquisition of necessary supplies for system operation. A variety of printer and plotter supplies and magnetic media may be required for routine functioning of a treatment planning system. The system manager is likely to be best qualified to determine the exact supply needs for the system. Much critical information is stored on system hard disks, from the systems software and the RTP software to specific beam and patient data. Periodic backup of all this information is essential in case the files on the computer become corrupted or the hard disk fails. The frequency of backups should be determined according to ͑1͒ the effort needed to recreate the lost information; and ͑2͒ the frequency of changes in the information. A typical backup schedule is given below: • Daily: Incremental backups of all new or altered files.
This assures that all the work done each day is not lost.
• Weekly: Backups of all treatment plan-related files. Many treatment plans are started, optimized, and completed within a week's time. • Monthly: Backup of the entire system, including the system software, RTP software, beam data files, and treatment plan files.

Data management tasks
Noncurrent patient data should be archived to appropriate media when available disk space is filled. The archived data may be necessary for legal reasons, for research studies, and/or for future use if the patient returns later for further treatment. The ability to accurately restore and use treatment plan data from archives, and its merger with current data, must be tested, as it can be a significant source of problems. Compatibility of archived data with the current version of treatment planning system is another source of problems. It is important that the RTP system developers ensure availability of a migration path.
Before an archival medium is chosen, consideration should be given to the amount of memory required. For 3D planning with CT and possibly MR images and multiple 3D dose distributions, the data for a single patient can take between 50 and 100 MBytes of space. Careful records should be maintained to facilitate retrieval of archived data. Documented policies and procedures for archiving and retrieving patient information should be developed, followed, and maintained. Bootable backups containing the appropriate systems and RTP software should be kept. Procedures should also provide for archiving of magnetic tape every 5-10 yr to preclude loss of information due to degeneration of the medium and possibly for off-site storage of important backup tapes.

Computer networks
The use of computer networks has become an important part of the RTP process in many institutions. CT and/or other imaging data are often input into the treatment planning system over a computer network connection. Multiple workstations are often linked by network so that all the workstations can share the same patient data. Communication with other parts of the departmental computer system, including record and verify systems, is also made possible with the use of network connections. Each of these links may be critical to the planning process and must be maintained by the computer systems manager. Security for all of these network connections will be discussed below.

System security
Security for the treatment planning system hardware, software, networks, and patient and beam data is an important issue which should be carefully managed by the planning system and computer systems managers. Procedures should be present to limit access to the treatment planning application software and treatment planning system data. The use of passwords for access to any treatment planning system or its patient and beam data should be implemented for all systems and is a requirement for networked computers.
As RTP systems have become more sophisticated, security issues have become significantly more complex. Several security issues are listed in Table 7-1.
True security for the RTP system requires a combined hardware/software strategy, with continuous review of new situations such as network access and/or capabilities changes as they occur.

Chapter 8: Summary of recommendations
This chapter summarizes some of the important recommendations of the task group. The appropriate section of the report for further details is listed in the parentheses at the end of each bullet.
• Adequate resources must be allocated to successfully implement an appropriate quality assurance program for treatment planning. The radiation oncology physicist must be given adequate time and resources to design, implement, and carry out the QA program. ͑Preface͒ • To meet the goals of the RTP QA program, adequate equipment and staffing of all the specialties, including radiation oncologists, radiation oncology physicists, medical radiation dosimetrists and radiation therapists, is necessary. ͑Part A͒ • It is important to realistically assess the staffing required for the QA program, particularly when new sophisticated systems are introduced into a department. Increasingly sophisticated treatment planning will likely call for more support for RTP QA to ensure the systems are used safely and that the QA procedures can be performed. ͑Part A͒ • Various certifications should be required for the staff involved in radiotherapy treatment planning ͑Part A͒: -Radiation oncologists should be certified by the tion Therapy Technology as defined by the American Registry of Radiologic Technologists, and hold an unrestricted state license in radiation therapy technology, where applicable. • The radiation oncology physicist in each clinic should review this report, use its guidelines to determine those issues that are of most importance, and then concentrate the RTP QA program on those issues. ͑Chap. 1.2͒ • This report is not a prescriptive listing of everything that must be done to perform adequate RTP QA, but is intended to give a summary of issues to be considered when creating the RTP QA program for a particular institution. ͑Chap. 1.4͒ • Users of a particular commercial treatment planning system should band together, with or without the assistance of the vendor of that system, to help each other create and perform the comprehensive QA which is required for that particular planning system. ͑Chap. 1.4͒ • It is critical that each institution name one radiation oncology physicist to be the ''responsible physicist'' for treatment planning in that institution, with overall responsibility for implementation, quality assurance, clinical use of treatment planning, and vendor contacts. ͑Chaps. 1.4 and 7.1.1͒ • The radiation oncology physicist must determine the accuracy of the RTP system for a range of clinical situations and how that expectation of accuracy must be modified to account for local situations. ͑Chap. 1.7͒ • The radiation oncology physicist must carefully design a rigorous set of specifications for acquisition of a RTP system if one wants to create a formal acceptance test which can verify that the system works as specified. ͑Chap. 2.1͒ • Specifications must be written with particular acceptance tests in mind. An acceptance test procedures document should then be written and agreed to by both user and vendor. ͑Chap. 2.3͒ • Most commissioning test procedures and priorities need to be individualized due to dependence on the RTP system and on individual institution's use of the various features. ͑Chap. 3͒ • The AAPM should form another task group specifically charged to develop a report on use and quality assurance of dataset registration techniques. • As treatment planning in the institution becomes more sophisticated, the range of dosimetric testing must expand and the physicist must carefully organize the testing and define appropriate limits for the testing. ͑Chap. Access to the RTP software should be limited, although it should be available to all individuals entitled to use the system, including dosimetrists and physicians. Much more stringent security is required for access to basic datasets used by the system.

4.1͒
Records should be kept of all individuals who have changed RTP system basic data, indicating the reason for changing the data as well as the changes made in the data.
Patient planning data must be protected, both against undesired modification and for protection of patient confidentiality.
Security for the planning software, the data files associated with the dose calculation algorithms and the patient treatment planning data require that significant security controls be designed into the system.
Network security must prevent all unwanted incursions into the planning system hardware, software, or patient data.
The RTP system computers should be secure against unexpected network accesses, particularly in light of the history of viruses appearing on the Internet as well as cases of unauthorized entry into computer systems.
• Complex anthropomorphic tests are useful for evaluating the overall system precision, but their usefulness in explaining discrepancies is limited. ͑Chap. 4.1͒ • Some commissioning tests and data are used to test multiple aspects of the planning system. These tests should be designed to be as independent as possible, so that appropriate analysis is performed. ͑Chap. 4.1͒ • The verification of external beam and brachytherapy dose calculations for clinical use is a very important part of RTP system commissioning. A comprehensive series of test cases must be planned, measured, calculated, compared, analyzed, and evaluated before any dose calculations are used clinically. ͑Chap. 4.1͒ • The particular test cases designed as part of the commissioning and QA programs for any particular institution depend on the RTP system involved, the way the system is ͑or will be͒ used clinically, and many other clinic-and system-dependent factors. Optimizing the test procedure for each clinic is essential if the QA program is to be effective yet achievable. ͑Chap. 4.1͒ • Self-consistency within the measured dataset ͑to be used for dose calculation commissioning and verification checks͒ is of primary importance and can be achieved by acquiring a set of relative measurements which are then interrelated by a small subset of either relative or absolute measurements. ͑Chap. 4.2.1͒ • The task group recommends that vendors of RTP systems provide sophisticated data input, storage, analysis, renormalization, display, and other capabilities inside their RTP systems to help users utilize the measured data. ͑Chap. 4.2.3͒ • Vendors should specify the data required by their system in the system documentation, and make this information available to users before purchase of the system. ͑Chap. 4.3͒ • Only data that has been measured on the specific treatment machine being commissioned into the RTP system should be used, unless it is known that the treatment units in question have exactly the same characteristics. Other beam data or ''representative'' data provided by an accelerator vendor ͑or by others͒ should never be used for dose calculation verification testing. ͑Chap. 4.3͒ • A data log book for documenting data acquisition, data handling, renormalization, and/or data smoothing procedures used in preparation and analysis of the beam data should be maintained. • Each radiation oncology physicist should review all the recommendations of this task group and the vendor of the RTP system and develop a program of periodic testing that will match the planning system characteristics and its user base. ͑Chap. 5͒ • A systematic review of all machine data spaced over the entire year should be considered. Careful prioritization of issues specific to each clinic is critical. ͑Chap. 5͒ • A series of reviews and training sessions for planning staff is recommended. ͑Chap. 5͒ • The most important part of the QA program is neither the dosimetric or nondosimetric tests; it is the design and implementation of a clinical planning/delivery process that incorporates QA elements to comprehensively check all aspects of the planning and delivery for each patient and each plan. ͑Chap. 6͒ • The computer systems manager must work under the general supervision and responsibility of the responsible physicist, so that computer system management is in agreement with the general needs of the clinical use of the planning system. ͑Chap. 7.1.2͒ • Security is critical for treatment planning software and data. Procedures must be implemented to limit access to the RTP software, system data, and patient data. ͑Chap. 7.5͒ • Numerous responsibilities of vendors and RTP system users ͑listed in Appendix A1͒ must be followed. ͑Appendix A1͒ • A comprehensive photon beam dataset, useful for algorithm verification, should be generated by the AAPM for use by vendors, users groups, and individual institutions as they perform their algorithm verification tests. ͑Appendix A3͒

Chapter 9: Conclusions
The creation of a comprehensive and practical quality assurance program for modern radiotherapy treatment planning is a large and uncompleted task. This task group report has as its goal the description of one way to approach that task, along with the description of many of the issues which must be considered while creating the QA program. A critical recommendation of this task group is that any RTP QA program must be individualized for the particular institution which is creating the program, so that it concentrates its effort on the high priority issues for that institution. It is hoped that the guidance provided by this report will make creating QA programs easier for the radiation oncology physicists who are responsible for this task.
Just as treatment planning use evolves in a clinic, it is clear that the QA program for treatment planning must also evolve so that it handles the evolving planning capabilities and uses. The task group clearly understands that as RTP evolves, particularly including the use of advanced 3D planning capabilities, so will the requirements of RTP QA. We look forward to ongoing reevaluation and revision of our recommendations as the field of treatment planning continues its advances and evolution.

ACKNOWLEDGMENTS
The task group would like to gratefully acknowledge the assistance of a number of consultants to the task group who have contributed significantly to the quality of this report. We particularly thank Jon Jacky, Ph.D., for his assistance in getting the work of the task group started, and Iain Bruinvis, Ph.D., for contributing his extensive clinical experience with treatment planning quality assurance, as well as supplying connections with the European approaches toward this subject. We would also like to thank Jeff Williamson, Ph.D., and Glenn Glasgow, Ph.D., who helped immensely in making the brachytherapy portions of this report more useful and more consistent with other ongoing quality assurance work in that area. Finally, the task group thanks Jatinder Palta, Nick Detorie, John Kent, Eric Klein, Rock Mackie, Radhe Mohan, and Mike Schell for helpful comments and suggestions.

Appendix 1: Vendor and user responsibilities
In this appendix we describe some of the responsibilities of the vendor and the user toward each other, in relation to QA for the treatment planning software. Detailed discussion of QA methodology typically used during development of software such as a RTP system is beyond the scope of this report. However, the more general topic of the responsibilities of the vendors or providers of the RTP software and the responsibilities of the users of that software is an important part of this report.

A1.1. Vendor responsibilities
Most radiation treatment planning systems are purchased by a clinic from a commercial vendor, although some centers with significant research programs may develop their own RTP systems. In general, for both commercial and noncommercial RTP systems, the basic quality of the software and the quality assurance procedures applied during its development and testing should be the responsibility of the vendor ͑or provider͒. In Sec. A1.2, the analogous responsibilities of the user of the software are delineated.
A1.1.1. Documentation. Extensive documentation on how the RTP software works should be provided by the vendor, including a description of the overall design, the theory of operation, the limitations and detailed explanations of what happens as each step of the planning process is performed. The documentation requirements are summarized in Table A1-1. Van Dyk 18 and Dahlin 55 also give recommendations for vendor documentation requirements.
A1.1.2. User training. The vendor must provide highquality training for the user. For sophisticated planning systems, this training should involve more than simply teaching the user the functions of the software buttons. It should also include useful planning strategies and other high-level issues that only the experienced user will encounter. Often, one kind of training is necessary for the treatment planner while the physicist who deals with beam data, calculation verification, and resolution of problems may require different or additional training. A1.1.3. Software quality assurance. The vendor should provide details of the software quality assurance program used to design, develop, test, document, and release the software. The vendor should attempt to give the user a clear basic description of the QA methodologies used, so that the user has a realistic idea of the types of QA testing that the vendor has performed. For general discussions of some of the issues involved in software QA, see, e.g., Refs. 56 and 57.
Users of commercial products have the ability to affect the software quality assurance programs of vendors. As suggested in the AAPM Task Group 35 report on Accelerator Safety for Computer-Controlled Medical Accelerators, 58 the user should require sufficient documentation from the provider of the software so that the user can be convinced that the system design, implementation, and quality assurance program are robust enough for the intended clinical use. This kind of documentation can be of significant assistance to users as they design their own QA programs. The user must pressure the vendor to provide as much information as can reasonably be provided by the vendor or assimilated by the user. One of the many motivations for this approach is that the user should be aware that there are usually errors in large software systems such as treatment planning systems. 16 Vendors can assist users in assessing the QA efforts expended on the RTP products in a number of ways: • Vendors should maintain a record of each problem found in the RTP system, either by users or internal staff, and how that problem was resolved. This record should be available to all customers and prospects. • Vendors should be willing to let their customers know why they believe the QA program for their product is sound. • Vendors should follow a rational software development process which can be explained to users. • Although many development materials may be proprietary, vendors should be prepared to show that they exist and to release parts of those documents if they address important user concerns. • Vendors should respond accurately and openly to users' questions about the number of staff, their training and experience, and the effort devoted to developing and maintaining the product.

A1.1.4. Version updates.
The arrival of a new software update for the treatment planning system always causes difficulty from a QA point of view, as the user always has to decide whether to implement the new version and how much testing to do before releasing it for clinical use. Old bugs may be fixed, but new bugs have probably been introduced. Usually there are also new functions to analyze and test. Table A1-2 lists some suggestions for vendor-supplied documentation that may help the user physicist determine what specific tests or other activities are required before a new version of the RTP system can be released for clinical use. The vendor must provide enough information so that the user can make intelligent choices about what needs to be tested without expecting to recommission the entire system. A1.1.5. Release of data formats. We strongly recommend that vendors adopt a standardized format convention such as DICOM-RT 59 for all files which are used for data import and export. Regardless of convention, vendors should release detailed descriptions of the formats and contents of these files, along with examples of correct implementation of

System management guide
The system management guide should contain information to help the user assure correct installation and use of the system.

Data requirements
The data ͑measurements and other͒ required by each calculation algorithm will help the user make an accurate assessment of the work which commissioning will require.
Test dataset A test dataset should be provided so that the user can verify the correct functioning of the system. This dataset should include data files and a test script. The expected test results should be provided, showing exactly the results of dose calculations and the appearance of graphic displays and other hardcopy. List of components that work differently from before.
Suggestions for tests that might be performed by the user.
Relevant results from beta testing of the new release.
Provide well-documented procedures and software to convert the old patient and/or treatment machine data to any new formats required by the new version.
List of known bugs and limits, with work-arounds, if available.
their data transfer mechanism. We also recommend that vendors use ͑and release information about͒ the DICOM image format for all images, and that the data formats used for input of data from water phantom systems also be released.
Other general use formats, such as the AAPM data exchange format, 60 should be maintained until the DICOM-RT convention becomes widely accepted. A1.1.6. Communication with users. Vendors should keep in touch with their users. Each vendor or system provider should establish an error-reporting procedure. This procedure should include not only a way for users to report errors to the vendor but also a method for the vendor to rapidly inform all users of errors, potentially confusing behavior of the planning system, or other information that the user should know. Each vendor should also establish a procedure for the users to obtain timely technical support. Both of these procedures should be documented and should be explained thoroughly to the users during initial RTP system training.
A1.1.7. Additional suggestions for vendors. In order to design a good QA program for a software package, it is helpful to have information about its design. Since the design of the system is certainly well-known to the vendor, vendors may assist the RTP QA efforts of their users by suggesting sets of tests that could be performed and by providing tools inside their RTP systems to help the users perform these tests. Table A1-3 lists a number of suggestions for vendors of RTP systems to aid the users. Perhaps the most important of these is the first, which recommends vendor assistance in forming a users group with the express purpose of cooperating in performing clinical QA testing. As this report makes clear, the scope of testing that should be performed is much larger than any one particular clinic can manage, especially in these days of strong financial pressure on hospitals and health care in general. The only realistic way to carry out a reasonably complete clinical QA program may be for users to share information, divide up the required testing, and share the results of those tests.

A1.2. User responsibilities
The user of the RTP system also bears a large amount of the responsibility for the QA of the software system and its use.
A1.2.1. Responsible physicist. It is essential that each institution designate a responsible physicist to supervise and manage all aspects of the RTP system installation, implementation, testing, and use at that site and to act as the interface for communication with the vendor. This person is a key part of the QA program. The responsible physicist should receive extra training so that he/she can fulfill this responsibility.
A1.2.2. Documentation. We have all experienced the situation where a software user says in desperation: ''I guess I'll have to break down and read the documentation.'' The responsible physicist at each site is responsible for assuring that all users have adequately read and understood the vendor's documentation. The vendor's documentation of course needs its own quality assurance program, and cooperation among the user's group may be the best way to identify missing or inadequate parts of the vendor's documentation.
A1.2.3. User training. The above statement also applies to training. The user is the one who is responsible for learning how to use the RTP system correctly. No amount of vendor effort can overcome lack of effort on the user's part.
A1.2.4. Software quality assurance. We stated previously that the vendor must provide the user with as much information as possible about the software QA procedures in order to convince the user of the correctness of the methods used. The user must attempt to assimilate and use the provided information correctly. A1.2.5. Version updates. Testing and implementation of a new RTP system software update are an important part of the physicist's clinical responsibilities. As was stated in Chap. A1.1.4, determining the testing required for a new version of software is a difficult problem. The user must analyze all the information about the update which is provided by the vendor and must prioritize the kinds of testing which are suggested by that analysis. Changes to critical parts of the system, such as monitor unit calculations, dose calculations, machine and beam functionality, or changes in the anatomical modeling or contour and image input features may require detailed testing, as they may affect important results from the system. Other changes may not require as much testing, although the radiation oncology physicist must always analyze how the RTP system is used in his/her particular clinic and make decisions about testing based on that knowledge. Encourage each user institution to designate a Responsible Physicist who will be responsible for the planning system and its use at that institution. Assure that this person is adequately trained to handle most planning system problems and issues. Support the creation and use of a standardized dataset for algorithm verification.
Suggest test procedures that could be performed by radiation oncology physicists to verify system operation and/or dose calculation accuracy.
Develop and implement tools inside the RTP system to assist in clinical QA testing, including: • batch mode calculation tests • tools to input and use measured data • analysis tools for calculation verification testing ͑for example, tools in Table 4-3͒ • redundant checks of critical calculations • tools to create phantom image datasets • tools for testing validity and protections of data files and RTP software • provide information so the user can carry out analysis of the normalization-MU calculation process • incorporate automated checks of geometric and dosimetric information into the beam and plan normalization process into the system design 13 A1.2.6. Use of data formats. User-developed software that uses vendor-provided data formats to export or import data into the RTP system must adhere to the same kinds of QA testing and verification as the vendor's software. It is the user's responsibility to carefully check all data transfer functions to insure that both sets of code and the interface between them actually work as intended.
A1.2.7. Education and communication with vendor. It is the responsibility of the responsible physicist to assure that all communication with the vendor and all documentation and training provided by the vendor are appropriately used by the users. Software bugs and other problems should be promptly reported to the vendor, and vendor information about errors or problem fixes should be quickly disseminated to all appropriate staff in the clinic.
7. Return to Anatomy Definition module, and change the image displayed for each of the cuts so that the correct density matrix image is displayed. Verify by eye and use of grayscale window/level that the density matrix uniformly covers the correct areas with appropriate densities.

Test Results and Evaluation
The density values displayed in the density measurement option depend on the assigned density as well as the CT number to electron density lookups which are used, so this translation must be documented for the assigned densities. One way to document the checks is to use the hardcopy output from the plan, and to note in pencil on the plot the verified points and densities.

Analysis and Summary
Summarize and investigate any unexpected behavior in the density results. option to check all the planning system output information against the hardcopy printout, and against the known ͑or at least desired͒ information inside the system.

Test Results and Evaluation
Document any differences between displayed values, hardcopy values, and known plan parameters.

Analysis and Summary
Summarize and investigate any unexpected behavior in the output.

Appendix 3: Photon dose calculation commissioning
This photon dose calculation test plan is suggested as an example of one way to organize the bulk of the testing associated with clinical commissioning of photon beam calculations. The tests are laid out according to test situations ͑e.g., open fields͒, rather than grouped by type such as algorithm tests or clinical verification tests. However, Table A3-1 gives a summary of the types of check made for each test situation. The body of this appendix gives descriptions of the kinds of tests which might be required for commissioning for each test situation.
This test plan is meant only as an example and not a prescription of the testing required. The test plan for a given institution should be based on that institution's particular requirements and should be developed only after the radiation oncology physicist carefully evaluates the importance of each class of experiments and prioritizes the commissioning procedures so that the clinically most important checks are performed first.

A3.1. Depth dose
One of the most critical and basic tests of any dose calculation algorithm is the ability to accurately predict the depth dose for standard open field situations. Here, calculations of the fractional depth dose ͑FDD͒ and tissue phantom ratio/tissue maximum ratio ͑TPR/TMR͒ are compared against measured data, as in Table A3-2.

A3.2. Output factors
Correct use of output factors is essential for extracting monitor units from the RTP system. Table A3-3 describes some of the necessary checks of the various required output factors, in which calculated results should be compared against the measured data.

A3.3. Open field data
The basic starting condition for any dose calculation modeling and/or verification is open fields. Table A3-4 lists open field checks which can be made with 2D isodose curves and charts, or with full 3D comparisons if the data and the analysis tools are available.

A3.4. Patient shape effects
The effect of the shape of the patient is studied with simple phantom studies in which the specific effects caused by the shape differences are easy to study ͑Table A3-5͒.

A3.5. Wedges
Verify dose calculations using measurements for each physical ͑or dynamic͒ wedge and each photon beam ͑Table A3-6͒. If a 3D dose matrix is calculated, the dose distribution must be checked ͑at a minimum͒ in both the axial and sagittal planes. For all situations, the phantom is placed at a standard SSD and all measurements are normalized at a specified depth, usually isocenter. Axial and sagittal isodose measurements are made in planes containing the central axis. Further extended SSD calculations should also be verified.

A3.6. Blocks
Block tests are listed in Table A3-7. Blocked field dose calculations are often used in two ways in a RTP system: ͑1͒

Oblique incidence
The oblique incidence data should be obtained at the largest angle possible. A 30ϫ30 field at 30 degree oblique incidence may be barely possible in some water tanks, and a 10ϫ10 field at a 40 degree oblique angle may also work.
Surface irregularity Use a step phantom to look at the effects of non-flat surface contours using a 30ϫ30 field incident on a large ͑5 cm͒ step in the surface of the phantom. Repeat the calculation with the beam displaced laterally by half of the dose grid spacing to assess effect of dose grid size.
Tangential geometry Measure dose delivered to axial plane for square phantom by 10ϫ20 tangential fields. Normalize the MU so absolute dose at isocenter is known. Compare isodose lines.
Square phantom 20ϫ20 or 25ϫ25 beam normal to a large square phantom. Compare measurements with beam centered on phantom and with beam off-center and flashing off one edge. Asymmetric and shaped fields Wedged asymmetric and/or shaped fields also should be verified, at least at a standard SSD.

A3.7. Multileaf collimator
Testing of the multileaf collimator ͑Table A3-8͒ is similar in principle to the verification checks used for blocked fields.

A3.8. Asymmetric fields
These tests check asymmetric use of MLC and/or jaws, including use with wedges and blocks/MLC ͑Table A3-9͒. One way to approach this is to use a 10ϫ10 field which is scanned from the center of the field to one of the corners of the collimator ͑as listed below͒. A larger field could also be checked in a similar manner.
All measurements are taken at a standard SSD and are normalized to the central-axis value at a specified depth for a 10ϫ10 symmetric field. Field directions are based on the IEC standard values of X1 and X2 for the normally transverse direction and Y 1 and Y 2 for the normally longitudinal collimator motions. The minimum testing required for asymmetric fields is quite dependent on the sophistication of the dose calculation algorithm used for these fields. In some algorithms, testing for asymmetric fields should include most of Tables A3-3, A3-4, and A3-6.

A3.9. Density corrections
The purpose of these tests is to validate the algorithm for density corrections, so the tests must be based on the nature of the correction method used. For example, if the algorithm uses a simple equivalent path length approach, the verifica-   tion of the algorithm can be performed with very simple 1D phantom tests. More complicated algorithms will require more complicated tests. In addition, however, it is important to document the accuracy of the calculational algorithm with a series of geometries that are more clinically relevant. The basis of the current correction tests ͑Table A3-10͒ is the benchmark set of inhomogeneity correction measurements made by Rice et al. 61 These data are generally limited to measurements along the central axis of the beam only, with several different geometries and two different beam qualities having been tested. When more general 2D and 3D inhomogeneity data are available, those test cases should also be included in this testing.

A3.10. Compensators
The kinds of tests which are used for compensators depends a great deal on the kind of compensation that is per-formed ͑Table A3-11͒. Missing tissue compensation uses only the patient shape to create the compensator for each field, then creates in the anatomical model a flat surface for dose calculations which approximates the expected behavior of the compensator. Dose compensation is more complex, as the algorithm uses calculated dose distributions and not just patient shape to design the compensator. Dose compensation algorithms may also optimize dose for several beams at the same time.

A3.11. Anthropomorphic phantoms
Several anthropomorphic phantom tests can be used for a final complete test of the entire calculation algorithm ͑Table A3-12͒. These test cases should be similar to treatment techniques used in the clinic.

Appendix 4: Electron dose calculation commissioning
In this appendix, as in Appendix 3 for photon beams, we give an example test plan which might apply to electron beam dose calculation commissioning. Tests include both algorithm verification and commissioning of individual beams. With planning, the same test can often be used to serve both purposes. Determination of exactly what tests are required will depend on careful analysis of the specific algorithm͑s͒ involved, the kinds of electron beams and their energies, and how these beams are used in that particular clinic.

A4.1. Depth dose and open fields
Data are obtained at the standard treatment distance ͑typically SSDϭ100 cm͒.    Mantle field Verify dose in coronal midline plane of phantom using TLD or film.
Tangential breast fields Include lung. Verify dose in axial plane.
3-field non-coplanar plan Verify dose in axial, sagittal, and/or coronal planes. standard field sizes which are chosen to agree with the various applicator sizes.

A4.2. Output factors
Correct use of output factors is essential for extracting monitor units from the RTP system. If the RTP system supports monitor unit calculations then a number of factors relevant to the monitor unit calculation must be evaluated ͑Table A4-2͒.

A4.3. Extended distance
Open field behavior at several SSDs may need to be verified if these distances are used for clinical treatments ͑Table A4-3͒.

A4.4. Shaped fields
Measurements for a series of shaped fields are necessary for systems in which effects of blocking are taken into account, as listed in Table A4-4.

A4.5. ECWG test cases
A comprehensive set of test cases has been described by the Electron Contract Working Group ͑ECWG͒. 62 This dataset was designed to be used for comparison of various algorithms in situations illustrating both standard measurement geometries and more complicated clinical geometries.
Although not designed to cover every possible circumstance, this dataset does address most of the normally used clinical geometries for electron beam treatment. All test cases are based on two electron energies ͑9 and 20 MeV͒ obtained from the Varian CLinac 1800 linear accelerator. The specific data measured for each test case were determined by the ECWG. The following general guidelines for measurements were used for each test case: ͑1͒ one or more depth dose curves; ͑2͒ five or more profiles for each transverse plane ͑often both radial and axial transverse planes͒; ͑3͒ beam's eye view ͑BEV͒ plane dose measurements using film in solid water. The geometry for each of the 28 ECWG experiments has been described, 62 and the specific dose measurement planes which were used for each of the experiments are listed ͑Table A4-5͒. This benchmark dataset, which is available to the community ͑see Ref. 62͒, is a good choice for basic algorithm verification testing.

Appendix 5: Brachytherapy dose calculation commissioning
This test plan is suggested as an example of one way to organize the testing associated with clinical commissioning

Output factor
Typically obtained at same field sizes used for standard FDD data: • SSD: 100 cm • Norm pt: 15ϫ15, at d max .
Effective source distance ͑ESD͒ Measure output as a function of distance to determine effective source distance to use for inverse square law corrections.
Output for shaped fields Many clinics determine output factors for a set of standard shaped fields.  of brachytherapy dose beam calculations. This proposal covers the most typical brachytherapy sources and procedures which are used. For those clinics that perform more complex or specialized procedures, or those that use new and/or different source types, additional tests will be required. The general types of tests recommended for commissioning various brachytherapy sources are listed in

A5.1. Source entry methods
The methods used to enter sources into the RTP system must be tested carefully. Some examples are listed in Table   A5-2. Note that handling changes in source location inside the patient, as a function of time, is clearly beyond the scope of the present report.

A5.2. Source library
Correct implementation of sources in the library which contains the inventory of sources known to the RTP system is critical to accurate brachytherapy planning and dose calculations. This is a critical issue both for initial commissioning, and for routine QA checks: • During commissioning, and also in later checks, each property or attribute described for each source in the source library should be verified. • Source information should be checked not only in the library itself but also in calculated dose distributions. • Compatibility of algorithm and underlying dataset with the clinical application should be assessed ͑e.g., con-sider the appropriateness of isotropic point source approximations, applicator shielding corrections, whether an anisotropy constant should be used, whether special protocols require special data, etc.͒. • Consider the compatibility of source strength quantities, units and conversion factors with vendor and institutional calibration practices. • Table A5-3 lists some of the relevant information that should be checked inside the source library.

A5.3. Source strength and decay
Since nearly all brachytherapy dose calculations are used in an absolute dose or dose rate mode ͑typically as total dose delivered, or dose/hour͒, the verification of the components of the calculation which directly affect the absolute dose are critical. Many older RTP systems will use factors such as

Orthogonal films
• Generate sample source distributions, project them onto two films ͑different Source-Film Dis-tances͒, enter sources with digitizer. • Make some random misidentifications of sources on the two films to make sure the system responds to this issue correctly. • Set the magnification factor incorrectly to check this functionality. • Misalign sources to determine how that system handles possible misalignment problems.
Stereo shift films Use same kinds of tests as for orthogonal films.
Keyboard entry Verify keyboard entry.

CT-based source localization
If CT-based brachytherapy source localization is available and will be used clinically, then this method must be tested. Complete tests may require CT scans of a phantom implanted with dummy seeds in known positions to ensure that CT artifacts or other problems do not interfere with the source identification and localization.

Catheter Trajectory Geometry
Modern RTP systems for high and low-dose afterloader machines often have algorithms which reconstruct the trajectory͑s͒ of the catheter͑s͒ used for the afterloaded sources. These algorithms deserve separate and careful verification checks.

Stereotactic implants
If CT-based stereotactic brachytherapy treatment is available and will be used clinically, then this process must be carefully tested. Numerous issues must be considered, including slice thickness and separation, partial volume effects, etc.

A5.4. Single source dose calculations
For brachytherapy, it is useful to separately consider the algorithm verification and clinical commissioning tests which should be used: • Each dose calculation algorithm used should be checked against independent computer calculations or exact or approximate manual calculations across the expected clinical range of use. • In addition, each implementation of an algorithm for a specific source type should be checked, ideally against published reference data ͑Monte Carlo or measured data͒ if available, or against manual approximations if the other data are not available. Note that both algorithm verification and calculation verification checks are occurring here. If there is lack of good agreement between calculation and data, it does not necessarily mean that the system is functioning incorrectly. In addition, general planning of brachytherapy dose calculation tests should include consideration of the issues listed in Table A5-5. Each source type which is modeled inside the RTP system must have its basic dosimetric calculation results verified  Verification checks of the source library ͑see section A5.2͒.
Comparison of single point, 2-D and 3-D dose distributions with hand calculations for a single source, for each source type in the source library.
Comparison of point, 2-D and 3-D dose distributions with hand calculations for multiple source configurations, for at least one source type.
Checks of any anisotropy or orientation-dependent features of the dose distribution for each type of source. If anisotropy is being neglected, it should be so noted in the dose distribution documentation.
Confirmation of absolute dose or dose rate values with changes in activity, decay constant, units for source strength, dose specification ͑e.g., dose rate or total dose͒.
Any applicator shielding effects included or neglected should be explained and documented.
Verify correct behavior of dose calculations, sometimes including tissue multiple scattering and attenuation, at selected distances from the source. ͑Table A5-6͒. Issues such as geometric factors, anisotropy connections, and the possible use of average anisotropy factors for isotropic calculations need to be thoroughly understood by the user.

A5.5. Multiple source dose calculations and optimization algorithms
Due to the importance of the absolute dose for most brachytherapy plans, it is important to verify the behavior of multiple source implants and to assure that the summations of contributions from various sources is correct. Table A5-7 lists some suggested kinds of tests for different source types, selected due to their widespread use.
In addition to simply adding multiple sources, RTP systems for standard implants and particularly for high-dose rate afterloaders may contain optimization algorithms which assist the user in determining the location and loading ͑or dwell times͒ of the source͑s͒ to be used in the implant. These algorithms, which may contain fairly complex use of dose volume histogram analysis or other rather new algorithms, should be carefully tested, not only to test the robustness of the optimization, but also to check the understanding and training of the user in making appropriate use of the optimization features.

A5.6. Global system tests
After verification that multiple source implants work correctly, it is appropriate to perform a number of global system tests, some examples of which are shown in Table A5-8. These tests, modeled after common clinical brachytherapy procedures, are designed to test the overall behavior of the system, including source input, identification of sources from the source library, source arrangement, dose calculation, and evaluation of the dose distribution. The procedure for each of these system tests should follow, as closely as possible, the normal procedures used in the clinic.

A5.7. Other Tests
Several additional procedures or types of brachytherapy planning must be commissioned and tested if they will be clinically used ͑Table A5-9͒.  Source optimization If available, use standard anatomical and dose constraints to verify that optimization algorithm behaves as expected over a series of situations and constraints.

HDR dwell time optimization
Use expected anatomical and dose constraints to confirm the correct behavior of the dwell time optimization algorithm contained in some RTP systems used with HDR systems.