While white light imaging (WLI) of endoscopy has been set as the gold standard for screening and detecting oesophageal squamous cell cancer (SCC), the early signs of SCC are often missed (1 in 4) due to its subtle change of early onset of SCC. This study firstly enhances colour contrast of each of over 600 WLI images and their accompanying narrow band images (NBI) applying CIE colour appearance model CIECAM02. Then these augmented data together with the original images are employed to train a deep learning based system for classification of low grade dysplasia (LGD), SCC and high grade dysplasia (HGD). As a result, the averaged colour difference ( †E) measured using CIEL∗a∗b∗ increased from 11.60 to 14.46 for WLI and from 17.52 to 32.53 for NBI in appearance between suspected regions and their normal neighbours. When training a deep learning system with added enhanced contrasted WLI images, the sensitivity, specific and accuracy for LGD increases by 10.87%, 4.95% and 6.76% respectively. When training with enhanced both WLI and NBI images, these measures for LGD increases by 14.83%, 4.89% and 7.97% respectively, the biggest increase among three classes of SCC, HGD and LGD. In average, the sensitivity, specificity and accuracy for these three classes are 88.26%, 94.44% and 92.63% respectively for classification of SCC, HGD and LGD, being comparable or exceeding existing published work.