Schedule
To maximize pilot success you should consider developing a detailed schedules folks know what to expect.
Recruiting
Recruiting participants is a key step in any pilot and establishing a brief screener survey can be a very helpful way to select participants. Make sure you identify the characteristics of participants that are essential (e.g. Are they old enough to participate? Do they have the right device? Do they speak the right language? Are they available to meet all the pilot requirements?). Afterward, you can design and administer a survey to make sure you are selecting eligible participants for your pilot.
You want to keep the screener survey short (ideally less than 10 questions) or else you may scare folks away. In addition, make sure to capture key contact information so that you can follow up with selected participants. Also keep in mind that any personal information you collect (names, emails, phone numbers, etc) is likely protected under privacy laws, so use best practices to secure this data, mark it as confidential, only share when necessary, and never make it public.
Warning! Be careful about cherry-picking. It is tempting to select ideal candidates for your pilot. These are participants who might be the easiest to work with and require the least help. However, if you cherry-pick, you will end up with a non-representative sample of users, which could skew your findings.
We recommend you think carefully about who your target audience is, and be sure to include both mainstreams and extremes.
Mainstreams: Your most likely users, the one that represents the ‘average user’
Extremes: Edge cases and extreme users (extremely old or young users, very tech-savvy or tech illiterate)
Screener Surveys:
Here is an example of a screener survey that we used to select participants for a recent pilot in Kenya. We sent the survey out to over 100 participants and narrowed the group to 35 eligible participants.
To determine eligibility, we asked questions about technology (owning an Android phone was essential), age and location (participants needed to be older than 18 and living in Nairobi), and availability (to meet requirements, participants needed to attend scheduled in-person sessions). In addition, we asked two ‘test’ questions (#11 and #12) to ensure that folks were answering questions diligently and carefully.
Participant Tracker
Once you have your participants selected, you should make sure that you have a way to keep track of them. This is especially important so that you can easily contact participants, identify specific persons of interest, and ensure that each is fulfilling the requirements of the pilot. Here is one template that we’ve used in Sierra Leone to keep track.
Pilot Onboarding
In pilots you will need to have some way to onboard participants - that is to get them access to your product or solution and to explain to them the rules and requirements for the project. If you are able, having an in-person meeting to explain the project is very useful. We’ve also done remote onboarding - using phone calls or written instructions sent virtually to participants as well. Pick the format that works best for your participants, but some things that are important to cover include:
Introducing the team, solution, organizations involved
Explaining the objectives of the project
Explaining the requirements for participation and the compensation
Securing informed consent
Giving access to your product or solution
Here’s an example of an onboarding deck we used in a past pilot in the Philippines.
Pilot Support
Things will go wrong. That is the nature of pilots! But you should have some idea of how to provide support or answer questions when those issues arise. Don’t be afraid of messiness, but here are some tips to make sure that your pilot goes a little smoother.
Set expectations: Make it clear to your participants that you are testing out a new product and that finding bugs and faults is a good thing!
Provide folks with a forum to share feedback and issues. Consider creating an intake survey, an email address, or a shared group chat where folks can discuss issues they see quickly.
Consider facilitating peer-to-peer support. In past pilots, participants can be a great resource to help each other. You may want to facilitate this forum by creating
Offboarding
Now you’ve finished your pilot - make sure you end on a good note. It’s our philosophy to design *with* users rather than for them. As such, it’s generally good practice to share some early results from the project. Ending on a positive note with your pilot participants is key - they could be useful evangelists for your product. And you may want to call on them in the future for testing or feedback.