Abstract: Abstract: Citrus trees bear fruits biennially, and the quantity of the fruits depends on the quantity of flowers. Therefore, the flower counting becomes more significant to manage citrus planting. Normally, a large number of flowers ranging from one to ten thousand in per plant make it difficult to manually count the quantity of citrus flowers. In image recognition, it makes much more difficult to detect the numbers of citrus flowers because the flowers are distributed densely, as well shaded by branches, leaves and other flowers. Therefore, this paper adopts the idea of the instance segmentation, where the mask-RCNN can simplify the relatively complex object segmentation by simple detection. The network was divided into five modules, including the convolutional backbone, feature map processing, mask calculation, bounding box regression and classification. The gradient disappearance that caused by the deep network was solved with the low error rate of the training models using the deep residual network. The FPN network was used for multi-scale recognition to increase the detection probability of citrus flowers with small mesoscale and dense distribution. The RoI Align method was used as the feature map pooling to avoid the loss of spatial symmetry. This proposed network can overcome some problems that the small images of citrus flowers cannot be recognized, and obtain the total number of citrus flowers by counting the bounding box of the instances of citrus flowers and buds. But the multi-layer/branch network structure increases the processing time of the training and validation. In this paper, the ResNeXt-50-FPN convolutional backbone was used to replace ResNeXt-101-FPN; the class-agnostic mask branch was used to replace the class-specific mask branch in order to reduce the time of training and validation; and the end to end training method was utilized to improve the accuracy of this model and further cut down the training time. In the Mask R-CNN analysis, the instance segmentation network was optimized to achieve the high recognition rate of citrus flowers. A comparative experiment was also conducted on the optimization of the convolution backbone, mask branch and network training patterns. The average precision was used to evaluate the models, while the error rate was set as another assessment indicator. The result shows that the proposed Mask R-CNN-based network with ResNeXt-50-FPN convolutional backbone and class-agnostic mask branch has the AP value of 36.3 when training by the end to end method, indicating that the average error rate is 11.9% for 1 031 flowers in 200 photos, and the identification time of single picture is 7 seconds. This finding demonstrates that the Mask R-CNN network can effectively detect the quantity of the citrus flowers under the condition of densely distribution with seriously shading. This proposed method also can reduce the time of model training and flower recognition, and provide the accurate data for early quantity control of citrus flowers for the great application in the production of citrus.