Abstract: Rice is one of the most important staple crops worldwide. China is also the largest rice producer. Among them, nitrogen (N) is an essential nutrient for rice growth; The rice yield and nitrogen use efficiency (NUE) are the critical influencing factors to determine food security and agricultural sustainability. However, the excessive application of nitrogen fertilizer can reduce the NUE to increase production costs and then cause environmental pollution. Therefore, it is crucial to accurately predict the rice yield and NUE for optimal nitrogen management and high production efficiency, in order to mitigate the environmental impacts. Particularly, remote sensing can be expected for the application of unmanned aerial vehicle (UAV) multispectral imagery in recent years. The UAV remote sensing can also be combined with machine learning. The crop growth can be efficiently acquired to significantly enhance the precision management of agricultural production. This study aims to predict the rice yield and NUE using UAV multispectral imaging and machine learning. A two-year field experiment was conducted in Fengyang County, Anhui Province, China. Multiple nitrogen levels and rice varieties were selected in the field. UAV multispectral images were collected at three growth stages (tillering, heading, and grain filling). The recursive feature elimination (RFE) algorithm was used to select the sensitive vegetation indices (VIs), texture features (TFs), and their combined features. Six models of machine learning-random forest (RF), support vector machine (SVM), multilayer perceptron (MLP), gradient boosting machine (GBM), deep neural network (DNN), and long short-term memory (LSTM)—were employed to establish the direct prediction models, according to the “sensitive remote sensing features–yield and NUE”. Additionally, an indirect prediction was proposed using calculated features. The NUE prediction was also evaluated after UAV remote sensing. A comparison was then made on the performance of direct and indirect models. The results showed that: (1) The prediction of rice yield and NUE at different growth stages exhibited varying dependencies on the remote sensing features. At the tillering stage, the spectral-based vegetation indices contributed more to the prediction accuracy. At the grain-filling stage, the texture features played a more significant role in the NUE prediction. Difference vegetation index (DVI), visible atmospherically resistant indices (VARI), and modified normalized difference blue index (mNDblue), as well as the texture mean (Mean), were sensitive to the yield across multiple growth stages. Similarly, the reciprocal ratio vegetation index (repRVI), correlation (Cor), and Mean were consistently sensitive to the NUE. (2) Among all models, the highest accuracy of deep neural network (DNN) was achieved in the direct predictions of yield and NUE. At the grain-filling stage, the TF-based model provided the most accurate predictions of yield (R²= 0.94, RMSE = 479.59 kg/hm²). At the tillering stage, the hybrid feature-based model demonstrated the best predictive performance for the NUE, with the agronomic nitrogen use efficiency (aNUE) and nitrogen partial factor productivity (NPFP) in the R² values of 0.71 (RMSE = 4.45 kg/kg) and 0.78 (RMSE = 12.79 kg/kg), respectively. (3) Compared with the direct models, the indirect model has significantly improved the NUE prediction accuracy. The R² values for aNUE and NPFP increased by 18.589% and 14.733%, respectively, while the RMSE values decreased by 54.41% and 90.02%, respectively. The framework of indirect prediction for the NUE was used to nondestructively and dynamically assess earlier growth stages. Traditional limitations were also avoided to reduce the data from the maturity stage. The UAV multispectral imagery with machine learning effectively enhanced the prediction accuracy of the rice yield and NUE. The indirect prediction also surpassed the direct approach. The DNN model also outperformed traditional machine learning in predictive accuracy. These findings can also provide valuable technical support and decision-making on the precision management of rice production in intelligent agriculture.