Supplemental sampling frame data as a means of assessing response bias in a hierarchical sample of university faculty [presentation] Presentation uri icon
Overview
abstract
  • Recent methodological discussions in survey research have addressed the relationship between low response rate and the potential for response bias. Sampling frames supplemented with data on all sample elements can be used to assess the existence and severity of response bias and make analytic adjustments as appropriate. Hierarchical sampling frames afford the opportunity to quantify individual and contextual characteristics that affect response likelihood. A sampling frame of randomly selected faculty members nested within departments and institutions was constructed from publicly available data and used to assess response likelihood from each level of the hierarchy. Anonymous surveys were mailed to each of 10 faculty members in each of 10 departments at 50 universities. Survey items asked about questionable research practices of respondents and their colleagues, and were therefore sensitive in nature. Returned numbered postcards confirmed survey completion so that response status but not survey responses could be linked to each sample element. The sampling frame included characteristics of universities (public/private, AAU/Carnegie membership, region, NIH rank), departments (field, size), and faculty (rank, sex). Multilevel logistic regression predicted confirmed complete status from faculty, department and university characteristics. Postcards were returned from 50 universities, 476 departments and 1402 faculty. Faculty less likely to confirm survey completion were those from private and western universities; in anthropology, economics and some medical specialty departments; and those with unknown gender or assistant, associate or unknown academic rank. Faculty in some allied health departments were more likely to confirm completion. Creating an enhanced hierarchical sampling frame from publicly available data was feasible. Supplemental information made a systematic assessment of individual and contextual response biases possible so that predicted response likelihoods could be used to adjust the primary analyses. A moderate response rate was observed but response bias was less severe than anticipated.

  • Research
    keywords
  • Data Collection
  • Questionnaires
  • Research Methods