OBJECTIVES: HealthPartners developed a checklist, the School Environment Index (SEI), that it uses to help elementary schools identify opportunities to improve student nutrition and increase physical activity. The objective in this pilot study was to assess whether the SEI, as administered, can be used to measure the progress of these programs. STUDY DESIGN: The authors focused their evaluation on the National Quality Forum measure evaluation components of reliability and validity; feasibility; and use and usability to assess the SEI's performance. METHODS: The authors used data from 214 SEIs completed by the 69 schools that participated in the school challenge in at least 1 of the years 2015 through 2019. Between 29 and 53 schools participated in a particular year. RESULTS: Cronbach's α was 0.79, intraclass correlation was 0.36 (95% CI, 0.22-0.53), and sensitivity to change was 0.41 (95% CI, 0.17-0.66) per 1-year change in the standardized SEI score. The median (interquartile range) time required to complete the survey was 11 (7-21) minutes. On only 8 surveys was an entire domain of the SEI skipped or only a single response to the domain recorded. CONCLUSIONS: The SEI shows adequate internal consistency and sensitivity to change in this pilot evaluation. It is also feasible and useful to identify opportunities to improve practices and policies related to student nutrition and physical activity in partnership with the participating elementary schools. However, it lacks reliability as used. Increasing the number of respondents per school might moderate the impact of individual respondents and thereby increase reliability.