Background: Lack of medication adherence to anti-diabetic therapy causes suboptimal glycemic control and leads to treatment failure and increased mortality from diabetes complications. The study aimed to assess medication adherence among type 2 diabetes mellitus patients at outpatient setting and its correlation with glycemic control. Methods: A cross-sectional observational study was conducted between January 2019 and March 2019 at several outpatient clinics in Ibb city, Yemen. A total of 370 patients of type 2 diabetes mellitus who use oral hypoglycemic drug therapy for at least 3 months were enrolled in the study. Fasting and postprandial blood glucose as well as glycosylated hemoglobin were measured and details of drug therapy were noted. Morisky Medication Adherence Scale was used to assess medication adherence. Results: Only 1.1% of type 2 diabetes mellitus patients had high medication adherence to oral hypoglycemic drugs, while 61.9% had moderate and 37% had low medication adherence. Most common medication adherence issues identified in type 2 diabetes mellitus patients were forgetting to take their medication when traveling, stopping of medication once feeling better, and difficulty to adhere to medication plan. Only 12.1% were having optimally controlled glycemic levels, whereas 87.8% were having uncontrolled glycemic levels. Conclusions: Despite having a good overall medication adherence among type 2 diabetes mellitus patients, the glycemic control still low in most patients. The study shows urgent need for reviewing anti-diabetic pharmacotherapy, counseling, and health education of patients.