By comparing siblings attending the same school at different points in time, we investigate whether the effect of peer quality on long-term labor market outcomes varies with parental background. We find that exposure to better peers—who have higher mean parental education—increases lifetime earnings of disadvantaged students, coming from families with low parental education, but penalizes privileged students from better educated families. These results suggest that desegregation policies that allocate disadvantaged students to schools with better peers produce long-term benefits. We discuss mechanisms and show that human capital accumulation, ordinal rank and network effects contribute to explain our findings.