Abstract: In order to realize the pepper size grading by machine, according to pepper shape that is approximate to an arc, the authors designed a sorting system of pepper based on image processing. 100 peppers were selected for the research, the peppers were first photographed, then the images were processed for extracting the edge of the pepper and removing the pepper’s handle, at the end, four vertex coordinates were calculated by the minimum bounding rectangle diameter on the edge of the peppers with removed handles. A mid-point of the pepper was selected to calculate the arc length of the pepper. A comparison of the measured results and the detection results showed that the separation accuracy was 95%. The study could provide certain reference for automatic pepper grading according to the pepper size.