Photoshopping Colonoscopy Video Frames

Yuyuan Liu, Yu Tian, Gabriel Maicas, Leonardo Zorron Cheng Tao Pu, Rajvinder Singh, Johan W. Verjans, Gustavo Carneiro

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The automatic detection of frames containing polyps from a colonoscopy video sequence is an important first step for a fully automated colonoscopy analysis tool. Typically, such detection system is built using a large annotated data set of frames with and without polyps, which is expensive to be obtained. In this paper, we introduce a new system that detects frames containing polyps as anomalies from a distribution of frames from exams that do not contain any polyps. The system is trained using a one-class training set consisting of colonoscopy frames without polyps - such training set is considerably less expensive to obtain, compared to the 2-class data set mentioned above. During inference, the system is only able to reconstruct frames without polyps, and when it tries to reconstruct a frame with polyp, it automatically removes (i.e., photoshop) it from the frame - the difference between the input and reconstructed frames is used to detect frames with polyps. We name our proposed model as anomaly detection generative adversarial network (ADGAN), comprising a dual GAN with two generators and two discriminators. To test our framework, we use a new colonoscopy data set with 14317 images, split as a training set with 13350 images without polyps, and a testing set with 290 abnormal images containing polyps and 677 normal images without polyps. We show that our proposed approach achieves the state-of-the-art result on this data set, compared with recently proposed anomaly detection systems.

Original languageEnglish
Title of host publicationISBI 2020 - 2020 IEEE International Symposium on Biomedical Imaging
PublisherIEEE Computer Society
Number of pages5
ISBN (Electronic)9781538693308
Publication statusPublished - 22 May 2020
Externally publishedYes
Event17th IEEE International Symposium on Biomedical Imaging, ISBI 2020 - Iowa City, United States
Duration: 3 Apr 20207 Apr 2020

Publication series

NameProceedings - International Symposium on Biomedical Imaging
ISSN (Print)1945-7928
ISSN (Electronic)1945-8452


Conference17th IEEE International Symposium on Biomedical Imaging, ISBI 2020
CountryUnited States
CityIowa City


  • adversarial learning
  • anomaly detection
  • Deep learning
  • one-class classification
  • colonoscopy
  • polyps

ASJC Scopus subject areas

  • Biomedical Engineering
  • Radiology Nuclear Medicine and imaging

Cite this

Liu, Y., Tian, Y., Maicas, G., Cheng Tao Pu, L. Z., Singh, R., Verjans, J. W., & Carneiro, G. (2020). Photoshopping Colonoscopy Video Frames. In ISBI 2020 - 2020 IEEE International Symposium on Biomedical Imaging (pp. 1642-1646). [9098406] (Proceedings - International Symposium on Biomedical Imaging; Vol. 2020-April). IEEE Computer Society.