In February 2016, the European Commission established a Working Group on mHealth tasked with developing guidelines “for assessing the validity and reliability of the data that health apps collect and process”. Since this Working Group was set up, there have been a series of face-to-face meetings, open stakeholder meetings, conference calls and online questionnaires. Two drafts of the guidelines have also been published for consideration, as discussed in our previous posts here and here.

Last month, the Working Group, drawn from patients, healthcare professionals, industry, public authorities, payers and social care insurance, research and academia, finally published its report on the draft guidance. Members of the Working Group were invited to give their views on the assessment criteria, what they understood by each of the criteria and whether they considered them relevant for the purposes of assessing the validity and reliability of health apps.

To the extent that any consensus could be found on the criteria for the assessment of apps, six criteria were considered to be relevant: privacy, transparency, reliability, validity, interoperability and safety. Two further criteria achieved majority support: technical stability and effectiveness.

However, the Working Group’s discussions were plagued by “areas of apparent disagreement and different understanding of the implications, use and meaning of the criteria during app assessment”, such that the Working Group was unable to come to any agreement on the scope, purpose or targets for health app assessment guidelines. Their divergent understandings were not helped by the fact that the range of technologies that constitute health apps is constantly evolving, nor by the passing of new legislation and guidelines at EU and Member State level (e.g., the Medical Devices Regulation and the General Data Protection Regulation).

The Working Group was, therefore, forced to conclude: “Clearly, an important lesson from this exercise is the need to follow a step-wise approach, starting with a solid agreement on scope and terminology, especially if the Guidelines are to be developed by a multi-stakeholder group.” As such, it seems that the guidelines are currently not being progressed in their current form.