This study attempts to investigate the validity of translation tasks as a reading comprehension assessment in Korean high school classrooms and to identify the traits of the students who are good readers of English but not good translators. First the study explores the correlation between a translation task and a recall task for reading comprehension. Results showed a significant correlation between the two task scores (r=.902, p<0.01). However, there was a discrepancy between the High English Proficiency (HEP) and the Low English Proficiency (LEP) groups (r=.777, p<0.01) vs. (r=.912, p<0.01). Five students from the HEP group showed a significant discrepancy in z-scores of the two tasks. In an attempt to identify the reasons for the discrepancy, the study reports the English learning backgrounds, reading habits and attitude of the five students. An interview of the students revealed that those who are good readers of English but not good translators learned English from native speakers of English or in an English speaking environment beginning at an early age and had less experience in reading classes taught through the grammar translation method. The findings suggest that despite the close correlation between recall and translation tasks, assessing reading comprehension through translation may fail to identify students who are already good readers of English but not good translators.