The transcriptional and epigenetic regulation of CD8+ T cell differentiation is critical for balancing pathogen eradication and long-term immunity by effector and memory CTLs, respectively. In this study, we demonstrate that the lysine demethylase 6b (Kdm6b) is essential for the proper generation and function of effector CD8+ T cells during acute infection and tumor eradication. We found that cells lacking Kdm6b (by either T cell-specific knockout mice or knockdown using short hairpin RNA strategies) show an enhanced generation of memory precursor and early effector cells upon acute viral infection in a cell-intrinsic manner. We also demonstrate that Kdm6b is indispensable for proper effector functions and tumor protection, and that memory CD8+ T cells lacking Kdm6b displayed a defective recall response. Mechanistically, we identified that Kdm6b, through induction of chromatin accessibility in key effector-associated gene loci, allows for the proper generation of effector CTLs. Our results pinpoint the essential function of Kdm6b in allowing chromatin accessibility in effector-associated genes, and identify Kdm6b as a potential target for therapeutics in diseases with dysregulated effector responses.