PURPOSE: Cancer-related cognitive impairment is common during cancer treatment; however, it is unclear whether the impairment persists over time. Our study aimed to examine long-term cognitive impairment among older breast cancer survivors. METHODS: Participants included 2420 community-dwelling women aged 65 years or older at enrollment (1986-1988) (404 breast cancer cases and 1:5 matched cancer-free controls) from the Study of Osteoporotic Fractures. Participants were followed for 20 years with measured cognitive function repeated up to 6 times. Cognitive impairment was defined by the Modified Mini-Mental State Examination and Trail Making Test B. Generalized linear models were used to model risk of cognitive impairment in relation to breast cancer status and time from breast cancer diagnosis. RESULTS: Compared with controls, cognitive impairment in women with breast cancer significantly accelerated after cancer diagnosis. We also observed a more pronounced cognitive impairment after cancer diagnosis for women diagnosed with breast cancer at age  ≥  80 years or at advanced stage for both measures. CONCLUSION: Our study with more than 20 years of follow-up data found that breast cancer survivors had accelerated cognitive impairment after cancer diagnosis, especially among women diagnosed at older age or at advanced stage, relative to women without cancer. IMPLICATIONS FOR CANCER SURVIVORS: Breast cancer survivors may be encouraged to engage in both physical activity and cognitive training.