Objective Unique subsets of body size phenotypes seem to be more resistant to the development of obesity-associated metabolic disorders. However, the association between nonalcoholic fatty liver disease (NAFLD) and body size phenotypes has not been studied yet. The aim of the present study was to compare the proportion of NAFLD according to the body size phenotypes, and evaluate the association of NAFLD and body fat distribution. Methods A total of 1936 participants were selected from the Korean Genome and Epidemiology Study. Computed tomography (CT) was evaluated for measures of NAFLD and body fat distribution. Subjects were classified into four groups by body mass index (BMI) and metabolic syndrome. Results The proportions of NAFLD were 8.9%, 27.1%, 25.5%, and 45.3% in metabolically healthy normal weight (MHNW), metabolically abnormal but normal weight (MANW), metabolically healthy obese (MHO), and metabolically abnormal obese (MAO) subjects, respectively. After adjusting for confounding factors, LAI was independently associated with lower visceral fat in both genders, and higher subcutaneous fat of the abdomen and lower extremity in women. In a multivariate regression analysis, the odds ratio for NAFLD was 2.70 (95% confidence interval (CI) = 1.79~4.05) in MANW subjects, 2.70 (95% CI = 1.90~3.83) in MHO subjects, and 5.68 (95% CI = 4.02~8.02) in MAO subjects compared with MHNW subjects. These associations were still significant even after adjusting for visceral fat area. Conclusion The proportion of NAFLD was similar between groups with the MANW and MHO phenotypes. NAFLD may be differently associated with regional distribution of body fat between men and women.