ABSTRACT:
Colon cancer screening modalities are essential in preventing colon cancer. Polyps arepre-cursors to cancers that can be detected and removed in their early stages before theydevelop into cancer. Optical colonoscopy (OC) is an invasive colon cancer screeningprocedure where polyps can immediately be removed with an endoscope that traversesthe colon. Virtual colonoscopy (VC), offers a non-invasive procedure that usescomputed tomography scans to reconstruct a 3D mesh. The 3D mesh can be explored ina virtual environment with tools such as surface coverage visualization. Translatingbetween these domains can combine the advantages of both colonoscopy proceduresand help doctors find more polyps that might otherwise be missed. During colonoscopyprocedures, polyps can be missed due to hard to detect polyps that can blend into thecolon wall and polyps that fail to enter the colon's field of view. In this proposal, wediscuss various ways to translate between the two domains to bridge the gap betweenthe two modalities and address both causes of missed polyps. Standard domaintranslation approaches such as CycleGAN, focus on learning a one-to-one mapping,which is not effective for colonoscopy tasks. In these works, we present domaintranslation methods for VC/depth prediction, haustral fold detection, and missingsurface visualization to help provide a more complete view of the geometry of the colonto tackle out of view polyps, in addition to a method to add temporal consistency tothese frame based models. Missing surface detection along with realistic OC synthesisalso offers data augmentation methods to improve missed polyps in the camera's field ofview. Additionally, these image based techniques applied in colonoscopy aregeneralizable and shown in a nueordegradation task. While frame based approaches areuseful, colonoscopy procedures create videos that require temporal consistency to beanalyzed properly. To extend the previously mentioned frame based approaches tovideos, RT-GAN is presented to builds temporal models using frame models as a base