In cancer treatment, developing ideal anticancer drug delivery systems to target tumor microenvironment by circumventing various physiological barriers still remains a daunting challenge. Here, in our work, a series of pH- and temperature-responsive nanogels based on poly(N-isopropylacrylamide-co-1-propene-2-3-dicarboxylate-co-2-acrylamido-2-methyl-1-propanesulfonate [poly(NIPAAm-IA-AMPS)] cross-linked by ethylene glycol dimethacrylate (EGDMA) were synthesized by random copolymerization. The molar ratio between monomer-comonomers-cross-linker was varied to fine-tune the optimum responsiveness of the nanogels. These optimized nanogels were further coupled to N,O-carboxymethyl chitosan (NOCC) stoichiometrically using EDC-NHS coupling chemistry to enhance the swelling behavior at lower pH. Interestingly, these NOCC-g-nanogels, when dispersed in aqueous media under sonication, attain nanosize and retain their high water-retention capacity with conspicuous pH and temperature responsiveness (viz. nanogel shrinkage in size beyond 35 °C and swelled at acidic pH) in vitro, as reflected by dynamic light scattering data. Doxorubicin (DOX), a potent anticancer drug, was loaded into these nanogels using the physical entrapment method. These drug-loaded nanogels exhibited a slow and sustained DOX release profile at physiological temperature and cytosolic pH. Furthermore, confocal and TEM results demonstrate that these nanogels were swiftly internalized by MCF-7 cells, and cell viability data showed preferential heightened cytotoxicity toward cancer cells (MCF-7 and MDA-MB231) compared to the MCF10A cells (human breast epithelial cell). Furthermore, intracellular DNA damage and cell cycle arrest assays suggest a mitochondrial mediated apoptosis in MCF-7 cells. This study substantiates our NOCC-g-nanogel platform as an excellent modality for passive diffusive loading and targeted release of entrapped drug(s) at physiological conditions in a controlled way for the improved therapeutic efficacy of the drug in anticancer treatment.