Nutrient deficiencies are considered major constraints to rice production in sub-Saharan Africa (SSA), but a systematic, continent-wide analysis of these constraints is lacking. The objective of this study was to assess responses of rice yields to macronutrients (N, P, and K) across major rice production systems [irrigated lowland rice (IL), rainfed lowland rice (RL), and rainfed upland rice (RU)] and diverse agro-ecological zones (semi-arid, sub-humid, humid, and highlands) in SSA. Diagnostic trials were conducted in 17 countries in 18–60 farmers’ fields in each of 30 sites (totaling 1037 farmers’ fields), of which 12 sites were in IL, 15 in RL and 3 in RU. All trials had an NPK treatment, and three treatments in which either nitrogen (N), phosphorus (P), or potassium (K) was omitted from the NPK treatment (minus N, P, and K plots). The trials were conducted during one or two seasons. Regression between yields in the NPK and minus N plots was used for calculating N fertilizer requirement, following the approach used for the development of a nutrient management decision support tool for rice (RiceAdvice) in SSA. Site mean yields in the NPK treatment ranged from 1.4 to 7.3 t ha−1. On average, these yields were 1.1 t ha−1 higher than farmers’ yields previously observed in the same sites. N was generally the most limiting nutrient, followed by P. Rice yields without N, P, and K were 68, 84, and 89% of yields in the NPK treatment, and were positively correlated with those in the NPK treatment irrespective of production systems and agro-ecological zone (AEZ). Results from a regression model across sites indicated that achieving 3, 5, and 7 t ha−1 requires N fertilizer application at an average rate of 56, 91, and 122 kg N ha−1, respectively. However, there was significant treatment by site interaction on rice yield. Site mean yields were significantly lower without N, P, and K in 93, 60, and 50% of sites as compared to the NPK treatment. Yield response to macronutrients varied across sites and was not related to production system, nor AEZ. Results from this study will help further improvement of RiceAdvice for providing decision support tailored to the particular site, zone, or production system, thereby closing yield gaps, improving fertilizer nutrient efficiency and preventing negative environmental consequences of fertilizer use.