Challenges in Participant Recruiting For Usability Testing

Effective participant recruiting is crucial to collecting reliable data during usability testing of high-tech products and services. Not only should test participants reflect the characteristics of the targeted users of the product or service, but they also must be likely to use it. Only then will their experiences and opinions yield reliable data for identifying meaningful improvements. This paper presents real-world challenges in recruiting participants for usability testing of new-technology products and services for which no firmly established customer base exists. It discusses techniques for meeting recruiting challenges, before and after test sessions begin, based on Tec-Ed usability-test case studies. INTRODUCTION Effective participant recruiting is crucial to collecting reliable data during usability testing of high-tech products and services. Not only should test participants reflect the characteristics of the targeted users of the product or service, but they also must be the likely users. Only then will their experiences and opinions yield reliable data for identifying meaningful improvements. This paper discusses real-world challenges Tec-Ed has faced in recruiting participants for usability testing of new-technology products and services for which no firmly established customer base exists. It provides techniques for meeting the recruiting challenges, listed below, that occur during various phases of a usability study. Before the sessions begin—the planning phase: • Developing a target user profile to help identify likely users. • Using rigor in defining screening criteria. • Deciding whether and what to offer participants as honoraria. • Considering the effect of the usability test site on participant attendance. During actual recruiting: • Recruiting “backups” to handle “no-shows.” • Including participants whose characteristics almost match the user profile. • Recognizing possible job seekers and technology spies or competitors. After the sessions begin: • Dealing with unsuitable participants who “beat the screener.” • Understanding why a “good” participant might not be, and why a difficult one is valuable. This paper also discusses how Tec-Ed recruiters and usability specialists met those challenges, providing examples from usability tests of products or services such as: • A two-way paging device with virtual keyboard and email capability. • A new telephone conferencing system and related communications software. • The email component of a larger Web-based productivity package. • A commercial Web site that sells and delivers software. In addition, it provides handouts that supplement the information discussed. Please note that this paper does not address defining requirements for statistically significant comparisons between groups of participants with different characteristics; marketing strategies, targets, and demographics; or the qualities of a good recruiter. Nor does it address the detailed screening process used during recruiting. Home