Ketamine has been found to have rapid and potent antidepressant activity. However, despite the ubiquitous brain expression of its molecular target, the N-methyl-d-aspartate receptor (NMDAR), it was not clear whether there is a selective, primary site for ketamine's antidepressant action. We found that ketamine injection in depressive-like mice specifically blocks NMDARs in lateral habenular (LHb) neurons, but not in hippocampal pyramidal neurons. This regional specificity depended on the use-dependent nature of ketamine as a channel blocker, local neural activity, and the extrasynaptic reservoir pool size of NMDARs. Activating hippocampal or inactivating LHb neurons swapped their ketamine sensitivity. Conditional knockout of NMDARs in the LHb occluded ketamine's antidepressant effects and blocked the systemic ketamine-induced elevation of serotonin and brain-derived neurotrophic factor in the hippocampus. This distinction of the primary versus secondary brain target(s) of ketamine should help with the design of more precise and efficient antidepressant treatments.