BACKGROUND: Workplace evaluation is one of the first steps in reducing the risk of injuries and illnesses, and is part of several programs that promote a participatory approach to occupational health among small business owners. The usefulness of written safety evaluations is contingent upon non-safety professionals obtaining accurate and reliable results. OBJECTIVES: The purpose of this study was to better understand auto body shop owners' ability to correctly identify occupational health and safety issues within their businesses. METHODS: In this study, 11 auto body shop owners used a 25-question checklist, developed specifically for this industry, to identify key safety and health problems. Owner results were compared with those of an industrial hygienist (IH) experienced in using the assessment form. RESULTS: The average number of safety problems identified by the IH was twice as large as the number identified by business owners (P = 0.02). The average percentage agreement of answers between owners and the IH was 81% (SD = 21%). Shop owners were more accurate in assessing the presence of written safety programs and records than the presence of unsafe work conditions. Overall, owners' sensitivity (ability to correctly identify a safety-deficient item) was low (0.22). CONCLUSIONS: Collision shop owners had some difficulty correctly identifying many unsafe/non-compliant items or situations in their facility. Naive users' ability to correctly identify potentially hazardous situations--sensitivity--should be the metric of concern for the validity of safety assessments, and efforts should be directed at bringing this number as close to one as feasible.