Sex chromosomes are advantageous to mammals, allowing them to adopt a genetic rather than environmental sex determination system. However, sex chromosome evolution also carries a burden, because it results in an imbalance in gene dosage between females (XX) and males (XY). This imbalance is resolved by X dosage compensation, which comprises both X chromosome inactivation and X chromosome upregulation. X dosage compensation has been well characterized in the soma, but not in the germ line. Germ cells face a special challenge, because genome wide reprogramming erases epigenetic marks responsible for maintaining the X dosage compensated state. Here we explain how evolution has influenced the gene content and germ line specialization of the mammalian sex chromosomes. We discuss new research uncovering unusual X dosage compensation states in germ cells, which we postulate influence sexual dimorphisms in germ line development and cause infertility in individuals with sex chromosome aneuploidy.