INTRODUCTION: Professionalism is a key component of medical education and training. However, there are few tools to aid educators in diagnosing unprofessional behavior at an early stage. The purpose of this study was to employ policy capturing methodology to develop two empirically validated checklists for identifying professionalism issues in early-career physicians. METHOD: In a series of workshops, a professionalism competency model containing 74 positive and 70 negative professionalism behaviors was developed and validated. Subsequently, 23 subject matter experts indicated their level of concern if each negative behavior occurred 1, 2, 3, 4, or 5 or more times during a six-month period. These ratings were used to create a "brief" and "extended" professionalism checklist for monitoring physician misconduct. RESULTS: This study confirmed the subjective impression that some unprofessional behaviors are more egregious than others. Fourteen negative behaviors (e.g. displaying obvious signs of substance abuse) were judged to be concerning if they occurred only once, whereas many others (e.g. arriving late for conferences) were judged to be concerning only when they occurred repeatedly. DISCUSSION: Medical educators can use the professionalism checklists developed in this study to aid in the early identification and subsequent remediation of unprofessional behavior in medical students and residents.