Maximum likelihood analysis of free‐response receiver operating characteristic (FROC) data
Abstract
Receiver operating characteristic (ROC) methodology is widely used in evaluating medical imaging modalities. While appropriate in some cases, it has several drawbacks when the detection task, e.g., nodule detection, involves localizing the abnormality. Free‐response receiver operating characteristic (FROC) methodology offers a more natural framework to describe observer performance in such studies and has other advantages. Due to the lack of a statistical analysis procedure comparable to the maximum likelihood procedure (rocfit program) available for ROC studies, the FROC method has not gained widespread acceptance. This work presents and solves a two parameter model for the statistical analysis of FROC data. The model assumes that the probability density of the signal stimuli is normally distributed, as is the probability density for producing one or more false positives per image. A program (frocfit) is described for estimating the parameters and their uncertainties from experimental data. An index of performance is proposed to quantify observer performance in FROC experiments. Application of this methodology to several FROC data sets produced good to excellent fits.
Citing Literature
Number of times cited according to CrossRef: 145
- Yulei Jiang, Receiver Operating Characteristic (ROC) Analysis of Image Search-and-Localize Tasks, Academic Radiology, 10.1016/j.acra.2019.12.020, (2020).
- Woo Kyung Moon, Yao-Sian Haung, Chin-Hua Hsu, Ting-Yin Chang Chien, Jung Min Chang, Su Hyun Lee, Chiun-Sheng Huang, Ruey-Feng Chang, Computer-Aided Tumor Detection in Automated Breast Ultrasound using a 3-D Convolutional Neural Network, Computer Methods and Programs in Biomedicine, 10.1016/j.cmpb.2020.105360, (105360), (2020).
- Jay Patel, Steven Collins, Birsen Sirkeci-Mergen, A framework to analyze decision strategies for multi-band spectrum sensing in cognitive radios, Physical Communication, 10.1016/j.phycom.2020.101139, (101139), (2020).
- Amira Jouirou, Abir Baâzaoui, Walid Barhoumi, Multi-View Information Fusion in Mammograms: A Comprehensive Overview, Information Fusion, 10.1016/j.inffus.2019.05.001, (2019).
- Ching-Chun Huang, Manh-Hung Nguyen, X-Ray Enhancement Based on Component Attenuation, Contrast Adjustment, and Image Fusion, IEEE Transactions on Image Processing, 10.1109/TIP.2018.2865637, 28, 1, (127-141), (2019).
- Huangjing Lin, Hao Chen, Simon Graham, Qi Dou, Nasir Rajpoot, Pheng-Ann Heng, Fast ScanNet: Fast and Dense Analysis of Multi-Gigapixel Whole-Slide Images for Cancer Metastasis Detection, IEEE Transactions on Medical Imaging, 10.1109/TMI.2019.2891305, 38, 8, (1948-1958), (2019).
- Furqan SHAUKAT, Kamran JAVED, Gulistan RAJA, Junaid MIR, Muhammad Laiq Ur Rahman SHAHID, Automatic Lung Nodule Detection in CT Images Using Convolutional Neural Networks, IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 10.1587/transfun.E102.A.1364, E102.A, 10, (1364-1373), (2019).
- Wenyan Zhu, Hongyan Zhu, Multi-signal detection and parameter estimation fusion with an improved utility function, Digital Signal Processing, 10.1016/j.dsp.2019.102641, (102641), (2019).
- AC Lang, RK Schulze, Detection accuracy of maxillary sinus floor septa in panoramic radiographs using CBCT as gold standard: a multi-observer receiver operating characteristic (ROC) study, Clinical Oral Investigations, 10.1007/s00784-018-2414-1, 23, 1, (99-105), (2018).
- Andriy I Bandos, Nancy A Obuchowski, Evaluation of diagnostic accuracy in free-response detection-localization tasks using ROC tools, Statistical Methods in Medical Research, 10.1177/0962280218776683, 28, 6, (1808-1825), (2018).
- Harold Kundel, Calvin F. Nodine, A Short History of Image Perception in Medical Radiology, The Handbook of Medical Image Perception and Techniques, 10.1017/9781108163781, (11-22), (2018).
- Berkman Sahiner, Nicholas Petrick, Evaluation of CAD and Radiomic Tools, The Handbook of Medical Image Perception and Techniques, 10.1017/9781108163781, (389-406), (2018).
- Huangjing Lin, Hao Chen, Qi Dou, Liansheng Wang, Jing Qin, Pheng-Ann Heng, undefined, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 10.1109/WACV.2018.00065, (539-546), (2018).
- Xuechen Li, Linlin Shen, Suhuai Luo, A Solitary Feature-Based Lung Nodule Detection Approach for Chest X-Ray Radiographs, IEEE Journal of Biomedical and Health Informatics, 10.1109/JBHI.2017.2661805, 22, 2, (516-524), (2018).
- Xin Wang, Yi Guo, Yuanyuan Wang, Jinhua Yu, Automatic breast tumor detection in ABVS images based on convolutional neural network and superpixel patterns, Neural Computing and Applications, 10.1007/s00521-017-3138-x, 31, 4, (1069-1081), (2017).
- Frank W. Samuelson, Craig K. Abbey, The Reproducibility of Changes in Diagnostic Figures of Merit Across Laboratory and Clinical Imaging Reader Studies, Academic Radiology, 10.1016/j.acra.2017.05.007, 24, 11, (1436-1446), (2017).
- Waleed M. Gondal, Jan M. Kohler, Rene Grzeszick, Gernot A. Fink, Michael Hirsch, undefined, 2017 IEEE International Conference on Image Processing (ICIP), 10.1109/ICIP.2017.8296646, (2069-2073), (2017).
- Furqan Shaukat, Gulistan Raja, Ali Gooya, Alejandro F. Frangi, Fully automatic detection of lung nodules in CT images using a hybrid feature set, Medical Physics, 10.1002/mp.12273, 44, 7, (3615-3629), (2017).
- Masahiro Hashida, Ryousuke Kamezaki, Makoto Goto, Junji Shiraishi, Quantification of hazard prediction ability at hazard prediction training (Kiken-Yochi Training: KYT) by free-response receiver-operating characteristic (FROC) analysis, Radiological Physics and Technology, 10.1007/s12194-016-0374-1, 10, 1, (106-112), (2016).
- Zaiyang Long, Michael R. Bruesewitz, Emily N. Sheedy, Michele A. Powell, Jacqualynn C. Kramer, Randall R. Supalla, Chance M. Colvin, Jessica R. Bechel, Christopher P. Favazza, James M. Kofler, Shuai Leng, Cynthia H. McCollough, Lifeng Yu, Technical Note: Display window setting: An important factor for detecting subtle but clinically relevant artifacts in daily CT quality control, Medical Physics, 10.1118/1.4966698, 43, 12, (6413-6417), (2016).
- Dev P. Chakraborty, Xuetong Zhai, On the meaning of the weighted alternative free‐response operating characteristic figure of merit, Medical Physics, 10.1118/1.4947125, 43, 5, (2548-2557), (2016).
- Peter Bannas, Laura C. Bell, Kevin M. Johnson, Mark L. Schiebler, Christopher J. François, Utaroh Motosugi, Daniel Consigny, Scott B. Reeder, Scott K. Nagle, Pulmonary Embolism Detection with Three-dimensional Ultrashort Echo Time MR Imaging: Experimental Study in Canines, Radiology, 10.1148/radiol.2015150606, 278, 2, (413-421), (2016).
- Mohammad Majid al-Rifaie, Ahmed Aber, Dispersive Flies Optimisation and Medical Imaging, Recent Advances in Computational Optimization, 10.1007/978-3-319-21133-6_11, (183-203), (2016).
- Astha Jaiswal, William J. Godinez, Roland Eils, Maik Jorg Lehmann, Karl Rohr, Tracking Virus Particles in Fluorescence Microscopy Images Using Multi-Scale Detection and Multi-Frame Association, IEEE Transactions on Image Processing, 10.1109/TIP.2015.2458174, 24, 11, (4122-4136), (2015).
- Joel G. Fletcher, Lifeng Yu, Zhoubo Li, Armando Manduca, Daniel J. Blezek, David M. Hough, Sudhakar K. Venkatesh, Gregory C. Brickner, Joseph C. Cernigliaro, Amy K. Hara, Jeff L. Fidler, David S. Lake, Maria Shiung, David Lewis, Shuai Leng, Kurt E. Augustine, Rickey E. Carter, David R. Holmes, Cynthia H. McCollough, Observer Performance in the Detection and Classification of Malignant Hepatic Nodules and Masses with CT Image-Space Denoising and Iterative Reconstruction, Radiology, 10.1148/radiol.2015141991, 276, 2, (465-478), (2015).
- Jing Zhang, James I. Silber, Maciej A. Mazurowski, Modeling false positive error making patterns in radiology trainees for improved mammography education, Journal of Biomedical Informatics, 10.1016/j.jbi.2015.01.007, 54, (50-57), (2015).
- Joel G. Fletcher, Amy K. Hara, Jeff L. Fidler, Alvin C. Silva, John M. Barlow, Rickey E. Carter, Adam Bartley, Maria Shiung, David R. Holmes, Nicolas K. Weber, David H. Bruining, Lifeng Yu, Cynthia H. McCollough, Observer performance for adaptive, image-based denoising and filtered back projection compared to scanner-based iterative reconstruction for lower dose CT enterography, Abdominal Imaging, 10.1007/s00261-015-0384-1, 40, 5, (1050-1059), (2015).
- Chung-Ming Lo, Rong-Tai Chen, Yeun-Chung Chang, Ya-Wen Yang, Ming-Jen Hung, Chiun-Sheng Huang, Ruey-Feng Chang, Multi-Dimensional Tumor Detection in Automated Whole Breast Ultrasound Using Topographic Watershed, IEEE Transactions on Medical Imaging, 10.1109/TMI.2014.2315206, 33, 7, (1503-1511), (2014).
- I.L. Štep̌án-Buksakowska, J.M. Accurso, F.E. Diehn, J. Huston, T.J. Kaufmann, P.H. Luetmer, C.P. Wood, X. Yang, D.J. Blezek, R. Carter, C. Hagen, D. Hořínek, A. Hejčl, M. Roček, B.J. Erickson, Computer-Aided Diagnosis Improves Detection of Small Intracranial Aneurysms on MRA in a Clinical Setting, American Journal of Neuroradiology, 10.3174/ajnr.A3996, 35, 10, (1897-1902), (2014).
- Wook-Jin Choi, Tae-Sun Choi, Automated pulmonary nodule detection based on three-dimensional shape-based feature descriptor, Computer Methods and Programs in Biomedicine, 10.1016/j.cmpb.2013.08.015, 113, 1, (37-54), (2014).
- Yuchen Qiu, Jie Song, Xianglan Lu, Yuhua Li, Bin Zheng, Shibo Li, Hong Liu, Feature Selection for the Automated Detection of Metaphase Chromosomes: Performance Comparison Using a Receiver Operating Characteristic Method, Analytical Cellular Pathology, 10.1155/2014/565392, 2014, (1-9), (2014).
- Alex Ing, Christian Schwarzbauer, Cluster Size Statistic and Cluster Mass Statistic: Two Novel Methods for Identifying Changes in Functional Connectivity Between Groups or Conditions, PLoS ONE, 10.1371/journal.pone.0098697, 9, 6, (e98697), (2014).
- Masaki Katsura, Izuru Matsuda, Masaaki Akahane, Koichiro Yasaka, Shohei Hanaoka, Hiroyuki Akai, Jiro Sato, Akira Kunimatsu, Kuni Ohtomo, Model-Based Iterative Reconstruction Technique for Ultralow-Dose Chest CT, Investigative Radiology, 10.1097/RLI.0b013e31827efc3a, (1), (2013).
- John M. Irvine, Richard J. Wood, David Reed, Janet Lepanto, undefined, 2013 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), 10.1109/AIPR.2013.6749326, (1-9), (2013).
- Dev P. Chakraborty, A Brief History of Free-Response Receiver Operating Characteristic Paradigm Data Analysis, Academic Radiology, 10.1016/j.acra.2013.03.001, 20, 7, (915-919), (2013).
- Andriy I. Bandos, Howard E. Rockette, David Gur, Subject‐centered free‐response ROC (FROC) analysis, Medical Physics, 10.1118/1.4799843, 40, 5, (2013).
- Nicholas Petrick, Berkman Sahiner, Samuel G. Armato, Alberto Bert, Loredana Correale, Silvia Delsanto, Matthew T. Freedman, David Fryd, David Gur, Lubomir Hadjiiski, Zhimin Huo, Yulei Jiang, Lia Morra, Sophie Paquerault, Vikas Raykar, Frank Samuelson, Ronald M. Summers, Georgia Tourassi, Hiroyuki Yoshida, Bin Zheng, Chuan Zhou, Heang‐Ping Chan, Evaluation of computer‐aided detection and diagnosis systems, Medical Physics, 10.1118/1.4816310, 40, 8, (2013).
- Mohammad Majid al-Rifaie, Ahmed Aber, Ahmed Majid Oudah, undefined, 2012 IEEE International Conference on Bioinformatics and Biomedicine Workshops, 10.1109/BIBMW.2012.6470317, (280-287), (2012).
- Michael King, Michael Ljungberg, Sven-Erik Strand, Michael King, Introduction to Task-Based Assessment of Image Quality for Investigators Employing Monte Carlo Simulation of Imaging Systems, Monte Carlo Calculations in Nuclear Medicine, Second Edition, 10.1201/b13073-16, (287-312), (2012).
- Min-Chun Yang, Chiun-Sheng Huang, Jeon-Hor Chen, Ruey-Feng Chang, Whole Breast Lesion Detection Using Naive Bayes Classifier for Portable Ultrasound, Ultrasound in Medicine & Biology, 10.1016/j.ultrasmedbio.2012.07.006, 38, 11, (1870-1880), (2012).
- undefined Xin Liu, I. S. Yetik, undefined, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 10.1109/EMBC.2012.6346947, (4422-4425), (2012).
- Brandon D. Gallas, Heang-Ping Chan, Carl J. D’Orsi, Lori E. Dodd, Maryellen L. Giger, David Gur, Elizabeth A. Krupinski, Charles E. Metz, Kyle J. Myers, Nancy A. Obuchowski, Berkman Sahiner, Alicia Y. Toledano, Margarita L. Zuley, Evaluating Imaging and Computer-aided Detection and Diagnosis Devices at the FDA, Academic Radiology, 10.1016/j.acra.2011.12.016, 19, 4, (463-477), (2012).
- Lucretiu M. Popescu, PET Energy-Based Scatter Estimation in the Presence of Randoms, and Image Reconstruction With Energy-Dependent Scatter and Randoms Corrections, IEEE Transactions on Nuclear Science, 10.1109/TNS.2012.2201170, 59, 5, (1958-1966), (2012).
- Sheng Chen, Kenji Suzuki, Computerized Detection of Lung Nodules on Chest Radiographs, Machine Learning in Computer-Aided Diagnosis, 10.4018/978-1-4666-0059-1.ch006, (122-144), (2012).
- Ryohei Nakayama, Akiyoshi Hizukuri, Koji Yamamoto, Nobuo Nakako, Naoki Nagasawa, Kan Takeda, Computer-Aided Detection Scheme for Sentinel Lymph Nodes in Lymphoscintigrams using Symmetrical Property Around Mapped Injection Point, Journal of Digital Imaging, 10.1007/s10278-011-9396-8, 25, 1, (148-154), (2011).
- Ehsan Samei, Nariman Majdi-Nasab, James T. Dobbins, H. Page McAdams, Biplane Correlation Imaging: A Feasibility Study Based on Phantom and Human Data, Journal of Digital Imaging, 10.1007/s10278-011-9392-z, 25, 1, (137-147), (2011).
- Dev P. Chakraborty, New Developments in Observer Performance Methodology in Medical Imaging, Seminars in Nuclear Medicine, 10.1053/j.semnuclmed.2011.07.001, 41, 6, (401-418), (2011).
- Nobukazu Tanaka, Kentaro Naka, Hiroko Fukushima, Junji Morishita, Fukai Toyofuku, Masafumi Ohki, Yoshiharu Higashida, Digital magnification mammography with matched incident exposure: physical imaging properties and detectability of simulated microcalcifications, Radiological Physics and Technology, 10.1007/s12194-011-0116-3, 4, 2, (156-163), (2011).
- Howard Rockette, Free-Response ROC Analysis, Statistical Evaluation of Diagnostic Performance, 10.1201/b11031-12, (163-200), (2011).
- Liansheng Larry Tang, N. Balakrishnan, A random-sum Wilcoxon statistic and its application to analysis of ROC and LROC data, Journal of Statistical Planning and Inference, 10.1016/j.jspi.2010.06.011, 141, 1, (335-344), (2011).
- Sheng Chen, Kenji Suzuki, Heber MacMahon, Development and evaluation of a computer‐aided diagnostic scheme for lung nodule detection in chest radiographs by means of two‐stage nodule enhancement with support vector classification, Medical Physics, 10.1118/1.3561504, 38, 4, (1844-1858), (2011).
- Lucretiu M. Popescu, Nonparametric signal detectability evaluation using an exponential transformation of the FROC curve, Medical Physics, 10.1118/1.3633938, 38, 10, (5690-5702), (2011).
- I. Smal, M. Loog, W. Niessen, E. Meijering, Quantitative Comparison of Spot Detection Methods in Fluorescence Microscopy, IEEE Transactions on Medical Imaging, 10.1109/TMI.2009.2025127, 29, 2, (282-301), (2010).
- Michel Bilello, Neeti Suri, Jaroslaw Krejza, John H. Woo, Linda J. Bagley, Alexander C. Mamourian, Arastoo Vossough, James Y. Chen, Bradd R. Millian, Todd Mulderink, Clyde E. Markowitz, Elias R. Melhem, An Approach to Comparing Accuracies of Two Flair MR Sequences in the Detection of Multiple Sclerosis Lesions in the Brain in the Absence of Gold Standard, Academic Radiology, 10.1016/j.acra.2010.01.019, 17, 6, (686-695), (2010).
- F. Zanca, D. P. Chakraborty, G. Marchal, H. Bosmans, Consistency of methods for analysing location-specific data, Radiation Protection Dosimetry, 10.1093/rpd/ncq030, 139, 1-3, (52-56), (2010).
- Luca Bertuccelli, W. Beckers, Mary Cummings, undefined, AIAA Guidance, Navigation, and Control Conference, 10.2514/6.2010-7863, (2010).
- D. P. Chakraborty, A status report on free-response analysis, Radiation Protection Dosimetry, 10.1093/rpd/ncp305, 139, 1-3, (20-25), (2010).
- Xi Long, W. Louis Cleveland, Y. Lawrence Yao, Multiclass detection of cells in multicontrast composite images, Computers in Biology and Medicine, 10.1016/j.compbiomed.2009.11.013, 40, 2, (168-178), (2010).
- P. Khurd, undefined Bin Liu, G. Gindi, Ideal AFROC and FROC Observers, IEEE Transactions on Medical Imaging, 10.1109/TMI.2009.2031840, 29, 2, (375-386), (2010).
- L M Popescu, undefined, IEEE Nuclear Science Symposuim & Medical Imaging Conference, 10.1109/NSSMIC.2010.5874350, (3009-3014), (2010).
- John M. Irvine, undefined, 2009 IEEE Applied Imagery Pattern Recognition Workshop (AIPR 2009), 10.1109/AIPR.2009.5466318, (1-8), (2009).
- Masahiro Yanagawa, Osamu Honda, Shigeyuki Yoshida, Yusuke Ono, Atsuo Inoue, Tadahisa Daimon, Hiromitsu Sumikawa, Naoki Mihara, Takeshi Johkoh, Noriyuki Tomiyama, Hironobu Nakamura, Commercially Available Computer-Aided Detection System for Pulmonary Nodules on Thin-Section Images Using 64 Detectors-Row CT, Academic Radiology, 10.1016/j.acra.2009.01.030, 16, 8, (924-933), (2009).
- Yuichiro Tsuchiya, Yoshie Kodera, Rie Tanaka, Shigeru Sanada, Quantitative Kinetic Analysis of Lung Nodules Using the Temporal Subtraction Technique in Dynamic Chest Radiographies Performed with a Flat Panel Detector, Journal of Digital Imaging, 10.1007/s10278-008-9116-1, 22, 2, (126-135), (2008).
- References, Journal of the ICRU, 10.1093/jicru/ndn017, 8, 1, (57-62), (2008).
- Xi Long, W. Louis Cleveland, Y. Lawrence Yao, Multiclass cell detection in bright field images of cell mixtures with ECOC probability estimation, Image and Vision Computing, 10.1016/j.imavis.2007.07.004, 26, 4, (578-591), (2008).
- Dev P. Chakraborty, Validation and Statistical Power Comparison of Methods for Analyzing Free-response Observer Performance Studies, Academic Radiology, 10.1016/j.acra.2008.07.018, 15, 12, (1554-1566), (2008).
- P. Sundaram, A. Zomorodian, C. Beaulieu, S. Napel, Colon polyp detection using smoothed shape operators: Preliminary results, Medical Image Analysis, 10.1016/j.media.2007.08.001, 12, 2, (99-119), (2008).
- Tomohiro Hirose, Norihisa Nitta, Junji Shiraishi, Yukihiro Nagatani, Masashi Takahashi, Kiyoshi Murata, Evaluation of Computer-aided Diagnosis (CAD) Software for the Detection of Lung Nodules on Multidetector Row Computed Tomography (MDCT), Academic Radiology, 10.1016/j.acra.2008.06.009, 15, 12, (1505-1512), (2008).
- D. P. Chakraborty, Hong‐Jun Yoon, Operating characteristics predicted by models for diagnostic tasks involving lesion localization, Medical Physics, 10.1118/1.2820902, 35, 2, (435-445), (2008).
- Tao Song, Andriy I. Bandos, Howard E. Rockette, David Gur, On comparing methods for discriminating between actually negative and actually positive subjects with FROC type data, Medical Physics, 10.1118/1.2890410, 35, 4, (1547-1558), (2008).
- Swatee Singh, Georgia D. Tourassi, Jay A. Baker, Ehsan Samei, Joseph Y. Lo, Automated breast mass detection in 3D reconstructed tomosynthesis volumes: A featureless approach, Medical Physics, 10.1118/1.2953562, 35, 8, (3626-3636), (2008).
- R. Bharat Rao, Oksana Yakhnenko, Balaji Krishnapuram, KDD cup 2008 and the workshop on mining medical data, ACM SIGKDD Explorations Newsletter, 10.1145/1540276.1540288, 10, 2, (34-38), (2008).
- F. Zanca, C. Van Ongeval, J. Jacobs, G. Marchal, H. Bosmans, A quantitative method for evaluating the detectability of lesions in digital mammography, Radiation Protection Dosimetry, 10.1093/rpd/ncn049, 129, 1-3, (214-218), (2008).
- Charles E. Metz, ROC analysis in medical imaging: a tutorial review of the literature, Radiological Physics and Technology, 10.1007/s12194-007-0002-1, 1, 1, (2-12), (2007).
- Maricel G. Kann, Sergey L. Sheetlin, Yonil Park, Stephen H. Bryant, John L. Spouge, The identification of complete domains within protein sequences using accurate E-values for semi-global alignment, Nucleic Acids Research, 10.1093/nar/gkm414, 35, 14, (4678-4685), (2007).
- Jun Ge, Lubomir M Hadjiiski, Berkman Sahiner, Jun Wei, Mark A Helvie, Chuan Zhou, Heang-Ping Chan, Computer-aided detection system for clustered microcalcifications: comparison of performance on full-field digital mammograms and digitized screen-film mammograms, Physics in Medicine and Biology, 10.1088/0031-9155/52/4/008, 52, 4, (981-1000), (2007).
- Dev Chakraborty, Hong-Jun Yoon, Claudia Mello-Thoms, Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data, Academic Radiology, 10.1016/j.acra.2006.10.015, 14, 1, (4-18), (2007).
- T P Hutchinson, Free-response operator characteristic models for visual search, Physics in Medicine and Biology, 10.1088/0031-9155/52/10/L01, 52, 10, (L1-L3), (2007).
- David Gur, Howard E. Rockette, Andriy I. Bandos, “Binary” and “Non-Binary” Detection Tasks: Are Current Performance Measures Optimal?, Academic Radiology, 10.1016/j.acra.2007.03.014, 14, 7, (871-876), (2007).
- Lyle D. Broemeling, Detection and Localization in Test Accuracy: A Bayesian Perspective, Communications in Statistics - Theory and Methods, 10.1080/03610920601126027, 36, 8, (1555-1564), (2007).
- Yoshikazu Uchiyama, Ryujiro Yokoyama, Hiromich Ando, Takahiko Asano, Hiroki Kato, Hiroyasu Yamakawa, Haruki Yamakawa, Takeshi Hara, Toru Iwama, Hiroaki Hoshi, Hiroshi Fujita, Computer-Aided Diagnosis Scheme for Detection of Lacunar Infarcts on MR Images, Academic Radiology, 10.1016/j.acra.2007.09.012, 14, 12, (1554-1561), (2007).
- Junji Shiraishi, Feng Li, Kunio Doi, Computer-Aided Diagnosis for Improved Detection of Lung Nodules by Use of Posterior-Anterior and Lateral Chest Radiographs, Academic Radiology, 10.1016/j.acra.2006.09.057, 14, 1, (28-37), (2007).
- Lucretiu M. Popescu, Nonparametric ROC and LROC analysis, Medical Physics, 10.1118/1.2717407, 34, 5, (1556-1564), (2007).
- Hong Jun Yoon, Bin Zheng, Berkman Sahiner, Dev P. Chakraborty, Evaluating computer‐aided detection algorithms, Medical Physics, 10.1118/1.2736289, 34, 6Part1, (2024-2038), (2007).
- Yuji Ikedo, Daisuke Fukuoka, Takeshi Hara, Hiroshi Fujita, Etsuo Takada, Tokiko Endo, Takako Morita, Development of a fully automatic scheme for detection of masses in whole breast ultrasound images, Medical Physics, 10.1118/1.2795825, 34, 11, (4378-4388), (2007).
- Tomasz Arodź, Marcin Kurdziel, Tadeusz J. Popiela, Erik O.D. Sevre, David A. Yuen, Detection of clustered microcalcifications in small field digital mammography, Computer Methods and Programs in Biomedicine, 10.1016/j.cmpb.2005.10.002, 81, 1, (56-65), (2006).
- Dev P. Chakraborty, Analysis of Location Specific Observer Performance Data: Validated Extensions of the Jackknife Free-Response (JAFROC) Method, Academic Radiology, 10.1016/j.acra.2006.06.016, 13, 10, (1187-1193), (2006).
- Xi Long, Louis Cleveland, Y. Lawrence Yao, Automatic detection of unstained viable cells in bright field images using a support vector machine with an improved training procedure, Computers in Biology and Medicine, 10.1016/j.compbiomed.2004.12.002, 36, 4, (339-362), (2006).
- Darrin C. Edwards, Charles E. Metz, Analysis of proposed three-class classification decision rules in terms of the ideal observer decision rule, Journal of Mathematical Psychology, 10.1016/j.jmp.2006.05.004, 50, 5, (478-487), (2006).
- Lucreţiu M Popescu, Robert M Lewitt, Small nodule detectability evaluation using a generalized scan-statistic model, Physics in Medicine and Biology, 10.1088/0031-9155/51/23/020, 51, 23, (6225-6244), (2006).
- D P Chakraborty, A search model and figure of merit for observer data acquired according to the free-response paradigm, Physics in Medicine and Biology, 10.1088/0031-9155/51/14/012, 51, 14, (3449-3462), (2006).
- A.C. Offiah, L. Moon, C.M. Hall, A. Todd-Pokropek, Diagnostic accuracy of fracture detection in suspected non-accidental injury: the effect of edge enhancement and digital display on observer performance, Clinical Radiology, 10.1016/j.crad.2005.09.004, 61, 2, (163-173), (2006).
- Junji Shiraishi, Qiang Li, Kenji Suzuki, Roger Engelmann, Kunio Doi, Computer‐aided diagnostic scheme for the detection of lung nodules on chest radiographs: Localized search method based on anatomical classification, Medical Physics, 10.1118/1.2208739, 33, 7Part1, (2642-2653), (2006).
- Mehul P. Sampat, Mia K. Markey, Alan C. Bovik, Computer-Aided Detection and Diagnosis in Mammography, Handbook of Image and Video Processing, 10.1016/B978-012119792-6/50130-3, (1195-1217), (2005).
- C. Leon Partain, Heang-Ping Chan, Juri G. Gelovani, Maryellen L. Giger, Joseph A. Izatt, Ferenc A. Jolesz, Krishna Kandarpa, King C. P. Li, Michael McNitt-Gray, Sandy Napel, Ronald M. Summers, G. Scott Gazelle, Biomedical Imaging Research Opportunities Workshop II: Report and Recommendations, Radiology, 10.1148/radiol.2362041876, 236, 2, (389-403), (2005).
- Rachna Arora, Harold L. Kundel, Craig A. Beam, Perceptually Based FROC Analysis1, Academic Radiology, 10.1016/j.acra.2005.06.015, 12, 12, (1567-1574), (2005).
- Hans Bornefalk, Estimation and Comparison of CAD System Performance in Clinical Settings1, Academic Radiology, 10.1016/j.acra.2005.02.005, 12, 6, (687-694), (2005).
- Nancy A. Obuchowski, ROC Analysis, American Journal of Roentgenology, 10.2214/ajr.184.2.01840364, 184, 2, (364-372), (2005).
- Mónica Penedo, Miguel Souto, Pablo G. Tahoces, José M. Carreira, Justo Villalón, Gerardo Porto, Carmen Seoane, Juan J. Vidal, Kevin S. Berbaum, Dev P. Chakraborty, Laurie L. Fajardo, Free-Response Receiver Operating Characteristic Evaluation of Lossy JPEG2000 and Object-based Set Partitioning in Hierarchical Trees Compression of Digitized Mammograms, Radiology, 10.1148/radiol.2372040996, 237, 2, (450-457), (2005).
- Harvey O Coxson, Elisabeth M Baile, Gregory G King, John R Mayo, Diagnosis of Subsegmental Pulmonary Emboli, Journal of Thoracic Imaging, 10.1097/01.rti.0000155044.82156.ad, 20, 1, (24-31), (2005).
- See more
