Abstract: An accurate recognition of crop pests has been one of the most important steps to control the pest occurrence for the higher crop yield. It is still a great challenge to effectively determine the characteristics of crop pests, where the appearance of crop pests belonging to the same species significantly varies with the growth periods, while the morphological features of crop pests also vary in the different species resemble each other. However, the manual identification and traditional Support Vector Machine (SVM) machine learning cannot fully meet the production needs of pest recognition in modern agriculture at present. Deep learning can be widely expected to identify pest species in recent years. Nevertheless, there is a large computational cost in the current Convolutional Neural Networks (CNN) for the feature extraction, due to the complex structure, thus leading to the lower recognition accuracy on a large number of dataset. This study aims to propose a pest intelligent recognition with high-performance, lightweight, and easy to apply for the production needs of smart agriculture. An improved EfficientNet-based scheme was established for crop pest recognition. First, the Coordinate Attention (CA) mechanism was introduced into the EfficientNet network structure, further locating the Region of Interest (ROI) area in a pest image using feature location information, which in turn improved the feature representation capability of the model. Second, the combined training strategy of data augmentation was developed to improve the diversity of pest samples, the robustness, and the generalization of the model. Third, an Adam optimization was used to further improve the convergence performance of the model. Last, a transfer learning strategy was also involved to initialize the parameters of the model. As such, a deep learning network named CA-EfficientNet was established to integrate these approaches, where the public large-scale dataset IP102 was taken as the network model training and performance testing in an experimental simulation. The results show that the CA-EfficientNet reached an accuracy of 69.45%, which was 4.01 percentage points higher than before, and 2.32 percentage points larger than the state-of-the-art GAEnsemble method for pest recognition. The amount of parameters dataset was 5.38 M in the improved CA-EfficientNet, and only 3.89%, 22.72%, and 52.63% of that for the VGG, ResNet-50, GoogleNet. In summary, the scheme remarkably improved the accuracy of recognition for a large type of crop pests at the cost of slightly more parameters than the baseline EfficientNet. As a result, the proposed scheme can be well facilitated to fully meet the needs of crop pest recognition in smart agriculture.