COVID-19 is an ongoing global crisis in which the development of effective vaccines and therapeutics will depend critically on understanding the natural immunity to the virus, including the role of SARS-CoV-2-specific T cells. We have conducted a study of 42 patients following recovery from COVID-19, including 28 mild and 14 severe cases, comparing their T cell responses to those of 16 control donors. We assessed the immune memory of T cell responses using IFNγ based assays with overlapping peptides spanning SARS-CoV-2 apart from ORF1. We found the breadth, magnitude and frequency of memory T cell responses from COVID-19 were significantly higher in severe compared to mild COVID-19 cases, and this effect was most marked in response to spike, membrane, and ORF3a proteins. Total and spike-specific T cell responses correlated with the anti-Spike, anti-Receptor Binding Domain (RBD) as well as anti-Nucleoprotein (NP) endpoint antibody titre (p<0.001, <0.001 and =0.002). We identified 39 separate peptides containing CD4 + and/or CD8 + epitopes, which strikingly included six immunodominant epitope clusters targeted by T cells in many donors, including 3 clusters in spike (recognised by 29%, 24%, 18% donors), two in the membrane protein (M, 32%, 47%) and one in the nucleoprotein (Np, 35%). CD8+ responses were further defined for their HLA restriction, including B*4001-restricted T cells showing central memory and effector memory phenotype. In mild cases, higher frequencies of multi-cytokine producing M- and NP-specific CD8 + T cells than spike-specific CD8 + T cells were observed. They furthermore showed a higher ratio of SARS-CoV-2-specific CD8 + to CD4 + T cell responses. Immunodominant epitope clusters and peptides containing T cell epitopes identified in this study will provide critical tools to study the role of virus-specific T cells in control and resolution of SARS-CoV-2 infections. The identification of T cell specificity and functionality associated with milder disease, highlights the potential importance of including non-spike proteins within future COVID-19 vaccine design.