Understanding the sensitivity of ecosystem production and respiration to climate change is critical for predicting terrestrial carbon dynamics. Here we show that the primary control on the inter-annual variability of net ecosystem carbon exchange switches from production to respiration at a precipitation threshold between 750 and 950 mm yr-1 in the contiguous United States. This precipitation threshold is evident across multiple datasets and scales of observation indicating that it is a robust result and provides a new scaling relationship between climate and carbon dynamics. However, this empirical precipitation threshold is not captured by dynamic global vegetation models, which tend to overestimate the sensitivity of production and underestimate the sensitivity of respiration to water availability in more mesic regions. Our results suggest that the short-term carbon balance of ecosystems may be more sensitive to respiration losses than previously thought and that model simulations may underestimate the positive carbon-climate feedbacks associated with respiration.