Testing devices without a specified user population
Finola Austin, human factors engineering manager at Owen Mumford, outlines a framework for manufacturers conducting human factors studies for platform devices, to help anticipate the needs of future consumers and meet best practice cost-effectively.
The development of medical devices must incorporate usability testing to ensure devices function as intended, while ensuring the safety of patients and other users. This requirement is established in industry standards and regulation. The EU Medical Device Regulation, which came into effect in May 2021, has a stronger focus on human factors testing than the preceding Medical Device Directive.1 The regulation demands that manufacturers must reduce the risks related to ergonomic features of a device as far as possible. It is therefore essential to assess device use with the various potential users, including medical professionals, lay users (such as carers) and patients; the latter category in particular may have a range of possible needs, depending on their condition. In addition, manufacturers will need to consider factors such as the intended use environment, training needs and the prior experience of end users.
Drug delivery device manufacturers may find it challenging to conduct such testing for platform devices such as auto-injectors and safety syringes. Platform devices are purposely designed to accommodate a range of formulations for different therapies; there is, therefore, a broad group of potential end users whose needs must be addressed. The challenge for manufacturers is to demonstrate that their platform device can be used safely and effectively by a wide user population, which will also include people with different levels of physical and cognitive abilities. Testing strategies and user recruitment will have to be carefully thought out to ensure sufficient representation and risk mitigation. The following steps provide a sound starting point for planning an inclusive strategy for a drug delivery platform device.
Decide on initial sample size
Current standards advise that early development stages should include 5–8 participants per distinct user group, to reveal any major usability issues.2 The sample size may not necessarily provide further insight: one study showed that doubling the sample size from 5 to 10 subjects only increased the mean percentage of usability problems identified from 85.6% to 94.7% (Table I).3 A less formal test with easier-to-access participants, such as members of the device manufacturer’s company, can precede this initial
testing. It is recommended, however, that manufacturers aim for a wide group of representative participants as early as possible, so that issues are identified and resolved early during device development.
Identify user groups
The number of user groups for platform device testing is likely to be higher than for drug delivery devices with specified intended users or defined therapy areas. Owen Mumford uses a framework that divides
subjects into seven user groups (Table II), encompassing the factors that may affect how a device is used. The first four groups cover categories of people who may use the device or aid a patient in using the device. The remaining three groups cover perceptual, cognitive and action abilities; that is, abilities that can affect how a user operates a device. Ideally, each use-impairment group should be mutually exclusive, to ensure that the device features are thoroughly tested within each group.
Broaden group diversity
To fully understand how and why issues may arise with device use, it may be valuable for the sample groups to include secondary characteristics, such as gender, ethnicity and hand dominance. For instance, if the ‘Action ability’ group mostly comprises patients with neurological conditions such as Parkinson’s, adding patients with biomechanical impairments such as repetitive injuries will make the sample more representative and may reveal additional areas for improvement. Furthermore, there may be commercial reasons to broaden user groups, to meet specific business objectives or demonstrate a minimum representation of a specific comorbidity, for example, to ensure users are included for key markets or segments critical for commercial success.
Prepare validation samples
After completing initial testing phases and addressing any issues, validation testing must be done to confirm that user needs have been met, risks mitigated and that there are no serious use concerns with the device. For this stage, US and UK regulators recommend 15–20 participants per user group, as this is considered large enough to account for any differences between users within the defined population. This final testing is critical to reassure prospective pharmaceutical customers that the device design has incorporated the needs of their intended user groups.
Plan carefully
As there may be a wide range of potential users of a platform device, it is critical to have a comprehensive sampling strategy that will help to anticipate safety and efficiency issues as cost-effectively as possible. The steps above can be used to guide a strategy that is tailored to manufacturers’ specific requirements. The plan should also consider how many studies are needed, the most appropriate stage of development to conduct studies, and the appropriate level of prototype fidelity. This thorough approach ensures confidence in the final drug delivery device for manufacturers and their potential pharmaceutical customers.