Early temperament, such as socio-emotional development and activity level, varies widely, yet its underlying biological associations are not understood. We identified genetic variation associated with infant and toddler temperament using genome-wide association meta-analyses. We studied parent-rated emotionality, activity, shyness and sociability (n = 43,963-72,663) in the second and third postnatal years and a cross-age average. Cross-age single nucleotide polymorphism heritabilities for emotionality, activity, shyness and sociability were 6.79% (95% confidence interval (CI), (4.71%, 8.87%)), 9.55% (95% CI, (7.04%, 12.06%)), 15.26% (95% CI, (12.24%, 18.28%)) and 3.42% (95% CI, (1.30%, 5.54%)), respectively. Ten genome-wide significant loci were discovered. Two loci colocalized with expression quantitative trait loci in the adult cortex: RHEBL1 (posterior probability, 0.93; associated with activity) and MR1 (posterior probability, 0.99; with emotionality). Genetic correlations were observed between early temperament and later outcomes, such as emotionality and adult neuroticism, activity and attention deficit/hyperactivity disorder (ADHD), sociability and autism, and shyness and adult extraversion. Multi-ancestry (n = 56,083-78,894) and European-ancestry analyses gave similar results. Infant and toddler temperament is associated with genetic variation and shows genetic continuity with later outcomes.