Abstract

Introduction: Members of the public interested in changing their health behaviour and comfortable with use of social media can download and purchase commercially developed lifestyle and or healthcare computer software. This is facilitated by the increase in technologically advanced, often elegantly designed, small screen portable in-built computerised systems on mobile smart-phones. Aims: With increasing awareness of use of Vitamin D for improved health outcomes, the aims for this study were to identify and evaluate any Vitamin D mobile smart-phone healthcare applications (apps) available for public or consumer use. Methods: Incorporating a multi-phase mixed study design, in phase one Vitamin D apps on mobile smart-phones were identified and categorised by consensus using the Delphi technique. In phase two a database search was undertaken to gain an understanding of research undertaken on use of healthcare apps and any evaluation of outcomes. In phase three a modified validated mobile applications rating scale (MARS) was utilised to evaluate the apps identified in phase one. Each app that provided some health information was assigned objective scores for information quality, and subjective quality and app specific rating scores. Discussion: No Vitamin D apps were identified where evaluation in research trial settings has shown significant beneficial health outcomes. However the literature search helped to identify criteria that form a potential checklist for any future development of a Vitamin D app for consumer use. Use of the modified MARS, with scores assigned for validated information quality criteria, identified two apps which were rated highly but with use restricted to medical teams, and seven apps focusing on dietary intake. Results: Nine apps were identified supporting some level of healthcare delivery and assigned MARS scores. Seven of these apps focused primarily on Vitamin D nutritional support. There were no high quality Vitamin D apps identified that could help consumers with decision on Vitamin D supplementation.

1. Background

Mobile healthcare applications (Apps) were downloaded an estimated 660 million times by June 2013 and industry experts project that around 1.7 billion mobile users will use healthcare apps on a fairly regular basis by 2018 ¹. Concerns however around the impact and quality of care provided to patients who use health apps in clinical practice have been raised ^{2, 3}. Caution has been advised when using apps as they may not incorporate robust scientific evidence ^{2, 3, 4, 5}, may not have had expert clinically oriented input ² or be based on behavioural change theory ^{4, 6, 7, 8}, factors considered important for apps to deliver positive health outcomes.

The Chief Medical Officer of England recently recommended Vitamin D supplementation for the whole population during the autumn and winter months ⁹. Current uptake is known to be poor with potential risk to future health ¹⁰. Mobile healthcare applications may be a useful source of information on Vitamin D usage for consumers and may increase uptake of supplements, provided the app incorporates appropriate dosing and product advice and is designed to help facilitate health professional discussion ¹¹.

In this paper, we report findings from work undertaken to identify the Vitamin D apps currently available for consumer use, with evaluation for any beneficial outcomes including quality assurance for these apps, in order to improve the standard of healthcare delivery.

2. Aims

To identify and evaluate the number and quality of Vitamin D apps currently available for public use

3. Methods

This study incorporated a multi-phase mixed study design.

Phase one: Identification and categorisation. A search for Vitamin D apps available at one given point in time was undertaken on the Apple iOS and Android mobile smart-phones systems. Identified apps were categorised based on the proposed aims of app. Using the Delphi technique ¹², this process involved the research team working towards interactive consensus with exchange of views and review of the apps by three members (NT, GO, MBl), with individual input at 3 stages to agree the final categories and categorisation of the Vitamin D apps.

Phase two: Research on use of apps and evaluation of outcome. A Medline, Embase and CINAHL databases search was undertaken using the following search terms; Vitamin D, Vitamin D deficiency, apps, smartphone / or computers handheld, rickets, mobile application / or Mobile apps / or Mobile phone / or mobile telephone and cell phones. Titles of the publications and abstracts if available were reviewed in line with study inclusion (use of apps and evaluation of outcome) and exclusion (not relevant to Vitamin D in particular) criteria, followed by assessment of full relevant papers. The search was extended to include Google Scholar using the search terms Vitamin D, rating scales and apps.

A validated mobile application rating scale was identified (MARS) ¹³. Stoyanov et al ¹³ reported their work on developing the MARS in 2015, with testing by a multidisciplinary team comprising health professionals (health psychologists) and web developers, and with validation of the scale undertaken for apps used in the mental health field ¹³. The MARS has also been used by other research teams, with intention to study the quality of apps in young adults with cancer ¹⁴ and in care delivery for patients with heart failure ¹⁵.

Phase three: Use of a modified validated rating scale to evaluate apps. The MARS ¹⁶ begins with initial data collection to classify the app enabling descriptive and technical information gathering for each app. This section includes identification of developer, version being assessed and purpose of the app as stipulated by the developer, affiliations, age group targeted by developer and technical supporting aspects, for example password protected access or a community discussion forum. The MARS team ¹⁶ state that some of these identifying data may need to be accessed via ITunes for the iOS systems or Google Play for the Android smart-phone apps. Within this classification section, data is also collected on focus of the app [what the app targets; with 12 criterion listed relating to mental healthcare provision, and option to record any additional relevant data under 'other'] and the theoretical background for the app, that is what strategies are supported by the app to enhance care delivery for the user.

Our research team discussed the background to the development of the MARS scale as described by Stoyanov et al in their publication ¹³, with decision to use the MARS to rate the Vitamin D apps, with scoring by three raters with a health professional background and experience of using mobile smart-phone apps as general members of the public [that is with no advanced information technology expertise]. The apps to be rated would be randomly allocated using the apps identified in the first phase of this study, and that rating of the Vitamin D apps would be under-pinned by view to deliver a high standard of healthcare to consumers looking for support for good bone health and good vitamin D levels. Criteria for data collection that were agreed under the 'focus' and 'theoretical background' sections therefore included:

1. 'Focus' of app: what the app targets (select all that apply): Increase Well-being; Behaviour Change; Goal Setting; Physical health; Other

2. 'Theoretical background’/Strategies (all that apply): Assessment; Feedback; Information/Education; Monitoring/Tracking; Goal setting; Other

The MARS scale ¹⁶ comprises a further 6 sections as follows: Section A focuses on engagement potential, that is, is the app fun to use, interesting, inter-active and well -targeted to audience? Section B focuses on functionality, with questions such as is the app easy to learn, use and navigate? Section C considers the aesthetics of the app including factors such as for example, design and overall visual appeal, colour scheme, and stylistic consistency. Section D focuses on the information provided by the app developer, with scoring for high quality information including text and feedback facilities, measures and references from a credible source. Raters are advised to select N/A if this app component was irrelevant, which indicates that the MARS development group ¹³ accepted that there would be poor quality information apps available to the public, but which may be aesthetically pleasing or where the primary purpose for the app may be entertainment or gaming.

Section E provided raters the option of scoring the app under test for subjective quality with section F looking at app-specific items. Section F criterion can be adjusted and used to assess the perceived impact of the app on the user’s knowledge, attitudes, intentions to change as well as the likelihood of actual change in the target health behaviour.

After discussion within our research group, the decision was to score the Vitamin D apps identified in the first phase of this study using the MARS sections D, E and F criterion; with each health professional rater to spend between 5-10 minutes familiarising themselves by downloading and navigating the app under evaluation before rating the app for information quality - section D. It was also understood that to truly study for change in behaviour the app / apps would have to be tested with a consumer population; this was a limitation in our work as in our study each rater acted as a ‘health professional’ consumer to score the app.

The apps selected for scoring were those categories most relevant to health care provision. Scoring was undertaken by three research team members with a health professional background (NT, MBo, GO), with randomised allocation of the apps to be rated. MARS scores were allocated by each rater (n=3) for criteria validated for information quality (MARS section D) ¹⁶ and for criteria that supported subjective assessment to allow for the expected variation in individualised perspective on use of the smart-phone app (MARS section E and F) ¹⁶.

4. Results

With use of search term 'Vitamin D' on the Iphone iOS system smart-phone, a total of 51 apps were identified as available for use by interested members of the public, as either a free or chargeable resource. This was at one point in time with the search undertaken on 1st June 2016. The apps were categorised based on the proposed aim of the app (Table 1). Assessment of Vitamin D apps available to the public on the Android smart-phone system was then undertaken, with 21 apps identified on 28th July, 2016. The main purpose for use of the app was agreed by the research team using the Delphi technique ¹² as described above.

Table 1 provides detailed information on the Vitamin D apps on the Iphone iOS system; these include advice on safe sun exposure (n=13), nutritional advice and support (n=19), commercial in orientation with provision of coupons towards purchase (n=3; system commonly used in the USA), apps which help with Vitamin D levels testing, tracking and guidance to help the user discuss the results with a doctor (n=6) and one app designed by a research institute and specifically targeting improvement in child health. 3 apps were designed to be used for educational purposes, for example in fields of chemistry and architecture with focus on the steroid structure of the Vitamin D molecule. 6 apps had been designed for the US leisure industry, and included general, simplified people oriented advice on safe sun exposure, exercise and relaxation treatments. The categories were found to be applicable for 17 of the 21 Android apps (Table 1), with 5 apps excluded, as one was in Chinese and the other four were not relevant.

In total, 51 and 21 Vitamin D apps were identified on the I-phone iOS and Android smart-phones respectively, available as a free or chargeable resource. Categories included: safe sun exposure (n=13; 6 respectively), nutritional advice and support (n=19;7), Vitamin D levels testing, tracking and for discussion with doctor (n=6;2). Some apps were designed for the US leisure industry (safe sun exposure, exercise, relaxation treatments) (n=6;1), for commercial use (n=3;1), with three iOS apps for educational utility (eg. chemistry, architecture). One iOS app had been developed to support delivery of child healthcare.

Those apps in the categories relevant to health care provision (shown in bold in Table 1; n=35) were then rated using the modified MARS. The MARS Information Quality scores ranged between 8 and 26 (see Table 2). The app for use exclusively by clinicians providing osteoporosis care scored highly at 26, with the Australian child health app assigned a score of 22. The remaining apps (n=7) focused mainly on advice to support dietary Vitamin D intake. One commercial app available on both the iOS and Android systems (MARS 20, 22 respectively), with these scores demonstrating good inter-rater consensus, targeted consumers with chronic conditions and supplementing with Vitamin D. MARS scores (Table 2) ranged from between 4 and 15 for subjective quality (Section E) and between 5 and 23 for App specific scoring (Section F).

5. Discussion

People interested in changing their health behaviour are downloading and purchasing commercially developed lifestyle and or healthcare computer software. This is facilitated by the increase in technologically advanced, often elegantly designed, small screen portable in-built computerised systems on mobile smart-phones.

A recent European wide survey ³ showed large variation in medical electronic devices and apps used by young professionals working in the field of radiation oncology. The researchers commented on the importance of verifying the consistency of information found within apps, to help avoid potential errors that could eventually prove to be detrimental for patients. The recommendation was that specific quality assurance criteria be developed for medical apps, including the suggestion that a comprehensive web site for all reliable applications and tools, which might be useful for daily clinical practice, be compiled. Authors also suggest ^{2, 5} that apps associated with good quality content are those that have been developed by or are linked with a medical professional. Edlin and Deshpande ² identified inadequate input from the medical profession in their study, and highlight the urgent need for quality evaluation, regulation, and information security, which they state is required for smart-phones and handheld devices for these to become an integral and safe part of delivery of health care. Most work reported under the research theme utilise clinical input and adherence with evidence based guidelines and or practice to support quality content, with researchers identifying other relevant issues that need to be considered for effective and appropriate use of apps ⁷.

Measures or criteria necessary for high quality health apps include content quality, accuracy and reliability, links to clinical input and evidence based guidelines or practice, app validation under real world conditions, the addressing of technical quality (eg. functional bugs; patient consent), and behavioural and strategy index scoring to allow for measurement of change in healthcare behaviour which hopefully will be on a positive health outcome trajectory.

Turning to Vitamin D specific apps , a French research team ¹⁷ supported by Yoplait (www.yoplait.com), have developed an app that provides the public with information to help them work out their risk for osteoporosis and fracture and which they could use to have a discussion with their GP. The aim stated for the development of this app, the 'Bone Thief', is to raise awareness and develop good life habits. A system designed to be self utilised as an electronic coach, the app comprises several headings including choosing a diet rich in Calcium and Vitamin D, exhibition [exposure] to sun and weekly physical activity. Although developed for both the Apple iOS and Android platforms, this particular app was not identified with the search term Vitamin D when assessment of app availability was checked at the time points for our research in the first phase. This could be because the app provides information on the osteoporosis condition per se, and is still in the development stage. There were two apps identified that had similar objectives to the 'Bone Thief', called the Food4Bones app and the NOF Clinicians guide to prevention and treatment of Osteoporosis (Table 3). Both these apps are supported by the UK's National Osteoporosis Society, but no beneficial outcomes evaluation has been published in relation to these two apps. The learning reported ¹⁷ considered the process for development which included brainstorming sessions between experts and patient advisors to agree content, but there were no other condition or disease related outcomes studied.

Buller et al have published their work ^{18, 19, 20} reporting on the development of the app Solar Cell, designed for the Android smartphone, and where the aim is to provide real time sun protection advice. The pilot stage included the running of focus groups for conceptual determination ¹⁸, followed by four rounds of usability testing to develop the app interface. This work found that adults desired sun protection advice, they identified few barriers to app use and were willing to input personal data. Further research ¹⁹ included a randomised controlled trial pre-test post-test design with 10 week follow up for usability testing and to assess if the system was interoperable across handsets and networks. The system was found to be highly user friendly with a mean of 5.06 (Range 0 = worst imaginable to 7 = best imaginable) ²⁰. With regards to Vitamin D related outcomes, the research for the Solar cell app found that it helped with some improved sun protection. Use of the app was lower than expected. When utilised, it was associated with increased sun protection but with no significant difference in number of sunburns in the past 3 months from point of study. The research team commented ^{18, 19} that strategies to increase use of mobile apps would be needed for more effective deployment in the general adult population.

The Australian SunSmart app was identified on the iOS system and has been evaluated ²¹. This app was developed with the aim of communicating complex UV light and frequency exposure information via an easy to use tool to help with daily decisions around safe sun exposure. Initially developed for the iOS Iphone and Ipad systems, the app is now available also on the Android and Samsung platforms. Essentially an extension of the international standard Global Solar UV Index, the app also includes Vitamin D tracker and sunscreen calculator functions. Paid and unpaid media were utilised to promote the app. 87% users said that the app met or exceeded their expectation with 86% agreeing that the app made them more aware of times for sun protection. More than half of app users said that they referred to the SunSmart app daily. No further objective beneficial outcomes have been reported. The research team note that the ongoing challenge is with need for regular updates and refreshing of the app ²¹.

Research undertaken to study the validity and reproducibility of the dietary component of a mobile Vitamin D calculator app ²² found that the mean vitamin D and calcium intake and risk classifications did not differ significantly between the 2 measures studied, namely real time information input into the app versus dietary recalls. Goodman et al ²² suggest that this app would be a good tool for use by the general public to help increase both the awareness and intake of nutrients. Interestingly work on the Nutriphone app where the aims are to develop a comprehensive system of analysis for multiple vitamins and micronutrients on the smart-phone has involved comparison of the app system with the well-established ELISA test kit for serum samples ²³. With a smart-phone accessory, an app, and a test strip, the system is reported to accurately measure 25- hydroxy vitamin D levels, with an accuracy better than 15nM and precision of 10nM, thus allowing for rapid quantification of Vitamin D levels on a smart-phone device.

In summary no Vitamin D app was identified where evaluation in research trial settings has shown significant beneficial health outcomes. All the apps discussed above are at the development stage at present. Our study had some limitations. Our research aimed to identify Vitamin D apps available on the iOS and Android systems and this was at one point in time. However it is accepted that development and availability of apps is increasing at a fast pace and with time. Our work did not identify any good quality Vitamin D app per se for consumer use that would support supplement choice and dosing. What is apparent is that the criteria necessary for high quality health apps which researchers consider important include content quality, accuracy and reliability, links to clinical input and evidence based guidelines or practice, app validation under real world conditions, the addressing of technical quality and behavioural and strategy scoring to allow for measurement of change in healthcare behaviour. These criteria help to form a checklist for any potential future development of a Vitamin D app for consumer use.

Using a mobile applications rating scale, MARS ^{13, 16}, we identified 9 apps that supported a reasonable level of healthcare delivery. However considering 2 of these apps, one was specifically designed for use by clinicians in the USA to support osteoporosis patient health care delivery, and therefore is not available for general consumer use. The second app has been developed by a research unit in Australia, for use by parents of sick children needing daily administration of Vitamin D, incorporating reminder texts and which facilitates ongoing support from their specialist centre health professional team. The remaining 7 Vitamin D apps assigned a MARS score focused primarily on Vitamin D nutritional support.

6. Conclusion

There were no high quality Vitamin D apps identified for use by consumers to help decision on routine Vitamin D supplementation. With significant number of people currently vitamin D deficient and at risk of its complications, a validated mobile app for healthcare delivery should be developed. Multi-disciplinary, public and health professional involvement will be necessary to ensure quality of the app, and to optimize safety and efficacy when utilized for healthcare delivery. The Vitamin D app will need regular updating with time. Our group are the first to use a modified mobile apps rating scale (MARS) with focus mainly on assessing the quality of information. In the future health apps should be assessed and rated routinely for quality of information, so consumers know they can trust the health information being provided.

References