IMPORTANCE: Recent SARS-CoV-2 Omicron variant sublineages, including BA.4 and BA.5, may be associated with greater immune evasion and less protection against COVID-19 after vaccination. OBJECTIVES: To evaluate the estimated vaccine effectiveness (VE) of 2, 3, or 4 doses of COVID-19 mRNA vaccination among immunocompetent adults during a period of BA.4 or BA.5 predominant circulation; and to evaluate the relative severity of COVID-19 in hospitalized patients across Omicron BA.1, BA.2 or BA.2.12.1, and BA.4 or BA.5 sublineage periods. DESIGN, SETTING, AND PARTICIPANTS: This test-negative case-control study was conducted in 10 states with data from emergency department (ED) and urgent care (UC) encounters and hospitalizations from December 16, 2021, to August 20, 2022. Participants included adults with COVID-19-like illness and molecular testing for SARS-CoV-2. Data were analyzed from August 2 to September 21, 2022. EXPOSURES: mRNA COVID-19 vaccination. MAIN OUTCOMES AND MEASURES: The outcomes of interest were COVID-19 ED or UC encounters, hospitalizations, and admission to the intensive care unit (ICU) or in-hospital death. VE associated with protection against medically attended COVID-19 was estimated, stratified by care setting and vaccine doses (2, 3, or 4 doses vs 0 doses as the reference group). Among hospitalized patients with COVID-19, demographic and clinical characteristics and in-hospital outcomes were compared across sublineage periods. RESULTS: During the BA.4 and BA.5 predominant period, there were 82 229 eligible ED and UC encounters among patients with COVID-19-like illness (median [IQR] age, 51 [33-70] years; 49 682 [60.4%] female patients), and 19 114 patients (23.2%) had test results positive for SARS-CoV-2; among 21 007 hospitalized patients (median [IQR] age, 71 [58-81] years; 11 209 [53.4%] female patients), 3583 (17.1 %) had test results positive for SARS-CoV-2. Estimated VE against hospitalization was 25% (95% CI, 17%-32%) for receipt of 2 vaccine doses at 150 days or more after receipt, 68% (95% CI, 50%-80%) for a third dose 7 to 119 days after receipt, and 36% (95% CI, 29%-42%) for a third dose 120 days or more (median [IQR], 235 [204-262] days) after receipt. Among patients aged 65 years or older who had received a fourth vaccine dose, VE was 66% (95% CI, 53%-75%) at 7 to 59 days after vaccination and 57% (95% CI, 44%-66%) at 60 days or more (median [IQR], 88 [75-105] days) after vaccination. Among hospitalized patients with COVID-19, ICU admission or in-hospital death occurred in 21.4% of patients during the BA.1 period vs 14.7% during the BA.4 and BA.5 period (standardized mean difference: 0.17). CONCLUSIONS AND RELEVANCE: In this case-control study of COVID-19 vaccines and illness, VE associated with protection against medically attended COVID-19 illness was lower with increasing time since last dose; estimated VE was higher after receipt of 1 or 2 booster doses compared with a primary series alone.