The genetic modification of autologous T cells with chimeric antigen receptors (CARs) represents a breakthrough for gene engineering as a cancer therapy for hematologic malignancies. By targeting the CD19 antigen, we have demonstrated robust and rapid anti-leukemia activity in patients with heavily pre-treated and chemotherapy-refractory B cell acute lymphoblastic leukemia (B-ALL). We demonstrated rapid induction of deep molecular remissions in adults, which has been recently confirmed in a case report involving a child with B-ALL. In contrast to the results when treating B-ALL, outcomes have been more modest in patients with chronic lymphocytic leukemia (CLL) or other non-hodgkin's lymphoma (NHL). We review the clinical trial experience targeting B-ALL and CLL and speculate on the possible reasons for the different outcomes and propose potential optimization to CAR T cell therapy when targeting CLL or other indolent NHL. Lastly, we discuss the pre-clinical development and potential for clinical translation for using CAR T cells against multiple myeloma and acute myeloid leukemia. We highlight the potential risks and benefits by targeting these poor outcome hematologic malignancies.