Abstract: The investigation proposed the optimal linear combination of RGB color model which could recognize citrus fruit disease and insect pest accurately and improve the informationalized level of citrus fruit production. First, several points from the damage pattern region and normal region in the representative image were selected and these points of sampling of red, green, blue value and their average values were recorded. Then linear programming objective function was designed and solved based on larger gray difference in damage pattern region and normal region to establish recognition model. Finally, combined recognition model with threshold segmentation method was used to process 96 citrus fruits images, and the recognition accuracy, false detection rate and missing detection rate were found to be 81.25%, 14.58% and 4.17%, respectively, the result was good. Compared with 2R-G-B, G-B and R-B, this method had larger gray difference in damage pattern region and normal region, showed the highest recognition accuracy and lowest false detection rate and missing detection rate. The thesis concludes that the linear combination of RGB color model can be used in the citrus fruit disease and insect pest recognition.