This paper is in the following e-collection/theme issue:

Digital Health, Telehealth and e-Innovation in Clinical Settings (401) eHealth Literacy / Digital Literacy (390) User Needs and Competencies (303) User Training and Skills (51) Adoption and Change Management of eHealth Systems (712)

Published on in Vol 8, No 2 (2019): Apr-Jun

Differences in the Level of Electronic Health Literacy Between Users and Nonusers of Digital Health Services: An Exploratory Survey of a Group of Medical Outpatients

Differences in the Level of Electronic Health Literacy Between Users and Nonusers of Digital Health Services: An Exploratory Survey of a Group of Medical Outpatients

Differences in the Level of Electronic Health Literacy Between Users and Nonusers of Digital Health Services: An Exploratory Survey of a Group of Medical Outpatients

Authors of this article:

Kamila Adellund Holt1 Author Orcid Image ;   Astrid Karnoe2, 3 Author Orcid Image ;   Dorthe Overgaard1 Author Orcid Image ;   Sidse Edith Nielsen4 Author Orcid Image ;   Lars Kayser2 Author Orcid Image ;   Michael Einar Røder4, 5 Author Orcid Image ;   Gustav From4 Author Orcid Image
Article Authors Cited by (45) Tweetations (8) Metrics

    Original Paper

    1Department of Nursing, Faculty of Health, University College Copenhagen, Copenhagen N, Denmark

    2Section of Social Medicine, Department of Public Health, University of Copenhagen, Copenhagen, Denmark

    3Danish Multiple Sclerosis Society, Copenhagen, Denmark

    4Medical Department, Herlev-Gentofte University Hospital, Copenhagen, Denmark

    5Steno Diabetes Center Odense, Odense University Hospital, Odense, Denmark

    Corresponding Author:

    Kamila Adellund Holt, RN, PhD

    Department of Nursing

    Faculty of Health

    University College Copenhagen

    Tagensvej 86

    Copenhagen N, 2200

    Denmark

    Phone: 45 26822657

    Email: kamh@kp.dk


    Background: Digitalization of health services ensures greater availability of services and improved contact to health professionals. To ensure high user adoption rates, we need to understand the indicators of use and nonuse. Traditionally, these have included classic sociodemographic variables such as age, sex, and educational level. Electronic health literacy (eHL) describes knowledge, skills, and experiences in the interaction with digital health services and technology. With our recent introduction of 2 new multidimensional instruments to measure eHL, the eHL questionnaire (eHLQ) and the eHL assessment (eHLA) toolkit, eHL provides a multifaceted approach to understand use and nonuse of digital health solutions in detail.

    Objective: The aim of this study was to investigate how users and nonusers of digital services differ with respect to eHL, in a group of patients with regular contact to a hospital outpatient clinic. Furthermore, to examine how usage and nonusage, and eHL levels are associated with factors such as age, sex, educational level, and self-rated health.

    Methods: Outpatients were asked to fill out a survey comprising items about usage of digital services, including digital contact to general practitioner (GP) and communication via the national health portal sundhed.dk, the eHLQ, and the eHLA toolkit, as well as items on age, sex, education, and self-rated health. In total, 246 patients completed the survey. A Mann-Whitney test was used to test for differences between users and nonusers of digital services. Correlation tests described correlations between eHL scales (eHEALSs) and age, education, and self-rated health. A significance level of .0071 was used to reject the null hypothesis in relation to the eHEALSs and usage of digital services.

    Results: In total, 95.1% (234/246) of the participants used their personal digital ID (NemID), 57.7% (142/246) were in contact with their GPs electronically, and 54.0% (133/246) had used the national health portal (sundhed.dk) within the last 3 months. There were no differences between users and nonusers of sundhed.dk with respect to age, sex, educational level, and self-rated health. Users of NemID scored higher than nonusers in 6 of the 7 dimensions of eHLQ, the only one which did not differ was dimension 2: Understanding of health concepts and language. Sundhed.dk users had a higher score in all of the 7 dimensions except for dimension 4: Feel safe and in control. The eHLA toolkit showed that users of sundhed.dk and NemID had higher levels of eHL with regard to tools 2, 5, 6, and 7. Furthermore, users of sundhed.dk had higher levels of eHL with regard to tools 3 and 4.

    Conclusions: Information about patients’ eHL may provide clinicians an understanding of patients’ reasons for not using digital health services, better than sociodemographic data or self-rated health.

    Interact J Med Res 2019;8(2):e8423

    doi:10.2196/ijmr.8423

    Keywords

    health literacy; computer literacy; questionnaires; telemedicine; consumer health informatics 


    Background

    The ongoing extensive digitalization of health services worldwide may be considered an advantage for many people, as the use of information and communications technology (ICT) ensures greater availability of services and better contact to health service professionals [1].

    In line with this development, in Denmark, public services are highly digitalized, and citizens communicate with public authorities via a digital portal. This includes electronic communication via a national email service called patient’s digital mailbox (e-Boks) [2]. Only people who are not able to access computers or use the digital services can be exempted from this (currently 8.9%) [2]. Digital access to all public services is governed by the national identification system NemID. In total, 98% of the population above 15 years has access to NemID [3]. Since 2009, the national health portal sundhed.dk has facilitated citizens’ access to the national, regional, and local health care services, their communication with health professionals, and their access to health-related information [4]. The access to nonperson-specific information about health services and the health-related information is publicly available, but access communication that includes personal data, for example, clinical data and communication with one’s general practitioner (GP) for renewal of drug prescriptions, requires a personal log-in using the NemID log-on. Public authorities communicate with citizens via e-Boks to send information letters from hospitals, including the outpatient clinic referrals.

    This increasing tendency toward mandatory digital communication with public services, including health care services, as is seen in Denmark, calls for attention to a problem: how can we include citizens who are not able to take advantage of the new digital opportunities and obtain the full benefits of digitalization?

    Previously, reasons for not using digital services or technologies were considered to be that users lack access to, have resistance to, or reject using the digital services as they do not find it beneficial [5].

    This simplified view has been challenged in recent years by studies that explore, in detail, the reasons for not using digital health services. A recent review identified several key barriers to successful adoption of digital health interventions [6]. These barriers were related to both personal attributes such as agency, motivation, personal life experiences, and values, and the context, that is, the health care providers’ approach to engaging and recruiting users, as well as the quality of the solution. In alignment with this, a Danish study, which examined why older people (aged above 58 years) use public digital services, identified that motivation, convenience, and skills were important factors for adopting digital solutions [7]. Two recent qualitative studies support this finding and also show that the patients’ context and condition may also influence their preferences. The first of the 2 qualitative studies is a Danish study based on 10 patients with more than 1 chronic condition (multimorbid patients), which argues that patients’ motivation to use ICT is positively related to the burden of their disease [8]. The other study from the Netherlands, including 17 patients with a chronic heart disease, reveals several other important factors for patients choosing to use an electronic health (eHealth) portal: a more personal contact to a coach, self-perceived computer skills, and factors related to how the platform is introduced and used [9].

    In a newly published study with 201 informants examining the willingness to use eHealth portals, authors found that the willingness to use a health portal was related to younger age, higher self-rated health, an education level above high-school level, higher acculturation status, higher computer literacy, and adequate health literacy [10].

    All these studies point to a complexity of reasons for adoption and use of digital health services, of which many relate to the individual’s knowledge, skills, perception, and experiences, including trust and motivation, but also relate to the way health professionals introduce new technologies [11].

    Electronic Health Literacy

    In 2015, Monkman and Kushniruk proposed a new model of Consumer Health Information System adoption [12]. In this model, they combine an understanding of the usability of the products with the users’ eHealth literacy (eHL)—a model which is in alignment with the suggestion by Kayser et al that it may be important to address users’ eHL to achieve design solutions that suit the user’s needs and capabilities better [13]. This calls for further investigation of the role of eHL as a means to understand the adoption and usage of digital health services in the context of the user interacting with the services and technology.

    The original concept of eHL or digital health literacy was introduced in 2006 by Normann and Skinner [14], and it highlights the users’ competence needed to engage with digital health services. Using the Normann and Skinner model and the related instrument, eHL scale (eHEALS), a positive correlation between information-seeking behavior and eHL has been demonstrated, for example, in 31 patients with rheumatoid arthritis in the Netherlands [15], 2371 parents of children with severe conditions in the United States [16], and in several thousand consumers in Israel [17]. The latter also reported that there was no relation between self-reported health and eHL [17].

    eHL has also been shown to correlate positively with the users’ educational level but correlate negatively to age [15,18].

    Although these studies have linked eHL to digital behavior, their findings were mainly based on the eHEALS instrument that directly evaluates information-seeking behavior on the internet in relation to health.

    Even though eHEALS is still a widely used tool [19-22], it does not provide sufficient understanding of the individual’s interaction with digital services and technology. In 2011, van der Vaart [15] already called for the need of a new understanding of eHL after the internet had been turned into a more dynamic Web 2.0 media. Moreover, in 2017, Griebel et al pointed to the need of new ways to describe eHL with a broader view on the digital health consumer perspective [23].

    With the development of 2 new tools, we have introduced a new understanding of people’s eHL, including knowledge, skills, perceptions, and experiences in relation to their usage of digital health services and health technology. One measure, the eHL questionnaire (eHLQ), is developed as an instrument to access the 7 dimensions of the eHL framework (eHLF), which describes users’ knowledge, skills, perception, and experiences in relation to digital health services and health technologies [24].

    The 35 items of eHLQ emerged from a condensation of more than 450 statements that constituted the fundament for the development of the eHLF. In this way, the final items capture a somewhat higher-order assessment of the respondent’s understanding and engagement in health information, which is more suitable for the intended usage as a psychometrically sound and valid instrument and is not intended to act as an inventory [24].

    The other instrument, the eHL assessment (eHLA) toolkit, examines eHL by combining specific elements from health literacy and digital literacy as both self-reported and performance tests [25].

    Objectives

    With this new multifaceted approach, we are able to contribute to a better understanding of how users differ from nonusers of digital services, not only with respect to personal attributes such as age, sex, educational level, and self-rated health but also with a particular focus on the individuals’ knowledge, skills, perception, and experiences with digital health services.

    Consequently, our research question is how can a multifaceted evaluation of individuals’ eHL be used to understand usage and nonusage of digital health services and how are usage and eHL related to age, sex, educational level, and self-rated health?


    Study Design

    We used a quantitative cross-sectional study design, collecting data using Danish versions of the eHLQ and eHLA instruments, both validated in a Danish population. In total, 246 patients diagnosed with diabetes, other endocrine conditions, and/or gastrointestinal diseases were included. The patients were consecutively enrolled when visiting the outpatient clinic at the Gentofte Hospital, Denmark, between November 2015 and March 2016.

    Patients were excluded if they were under the age of 18, had insufficient cognitive functions, or did not understand Danish. The distribution of the questionnaire containing the 2 instruments was undertaken by the nurses at the outpatient clinic, who also assessed whether the respondent had sufficient cognitive functions to participate. In some cases, the nurses also judged from an ethical perspective that the patients, for reasons not stated in the protocol, should be excused from participation in the study (see Figure 1). Patients either filled out the questionnaire in the waiting room or completed it at home and returned it in a prepaid envelope. Patients did not receive reminders.

    A total of 553 patients were given the questionnaire to complete. Of these, 246 filled in and returned the questionnaire, whereas 307 did not respond, resulting in a response rate of 44.4% (246/553; Figure 1).

    Questionnaires

    The questionnaire battery contained eHLQ, eHLA, and questions concerning the patient’s sociodemographics, digital behavior, and self-rated health.

    The questionnaire battery also included questions about whether informants were exempted from using e-Boks, had used their NemID within the previous 6 months, and whether they had logged into sundhed.dk within the previous 3 months. Finally, questions about the informants’ communication with their GP were included.

    Educational Level

    The demographic variable education was aggregated to 4 levels:

    1. Comprehensive school equivalent to International Standard Classification of Education 2011 (ISCED-2011) levels 1 and 2 or European Qualifications Framework (EQF) level 2.
    2. Short education equivalent to ISCED and EQF levels 3, 4, and 5.
    3. Medium education equivalent to ISCED and EQF level 6.
    4. Long education equivalent to ISCED and EQF levels 7 and 8 [26,27].

    Self-Rated Health

    The levels reported in eHLQ and eHLA were correlated to self-rated health, which was measured on a 5-point Likert scale from excellent to poor [28,29]. For the statistical evaluation, the scale was reversed so that excellent health was given the highest score (5) and poor health the lowest score (1).

    Sociodemographic Data

    The participants’ sociodemographic characteristics are provided in Table 1. The mean age was 56.5 with a range from 18 to 89 years.

    ‎
    Figure 1. Flowchart for inclusion of patients in the study.
    View this figure

    Electronic Health Literacy Questionnaire

    eHLQ is a validated and psychometrically sound instrument that comprises 35 items covering 7 dimensions: (1) using technology to process health information, (2) understanding of health concepts and language, (3) ability to actively engage with digital services, (4) feel safe and in control, (5) motivated to engage with digital services, (6) access to digital services that work, and (7) digital services that suit individual needs [30]. Dimensions 1 and 2 describe the patient’s individual competence, dimensions 3 to 5 describe the interaction between the patient and the digital services, and dimensions 6 and 7 characterize the patient’s experience with digital systems or services. Each of the first 5 dimensions contains 5 items, whereas dimension 6 has 6 items and dimension 7 has 4 items. Each item has 4 options, strongly disagree, disagree, agree, and strongly agree, which yield 1 to 4 points, respectively.

    Electronic Health Literacy Assessment Toolkit

    eHLA is a validated and psychometrically sound instrument that contains 4 (1-4) health literacy tools and 3 (5-7) digital literacy tools. The tools describe (1) functional health literacy, (2) self-assessed health literacy, (3) familiarity with health and disease, (4) knowledge of health and disease, (5) digital familiarity, (6) digital confidence, and (7) digital incentives [25].

    The eHLA comprises 44 items: 10 items in tool 1 (functional health literacy), 9 items in tool 2 (self-assessed health literacy), 5 items in tool 3 (familiarity with health and disease), 6 items in tool 4 (knowledge of health and disease), 6 items in tool 5 (digital familiarity), 4 items in tool 6 (digital confidence), and 4 items in tool 7 (digital incentives). Tools 1 and 4 are performance tests. In tool 1, functional health literacy, 1 point is given for each correct answer, and in tool 4, knowledge of health and disease, 2 points are given for each correct answer and 1 point for opting out. The remaining 5 tools have a 4-option scale. In tool 2, the scale ranges from very difficult to very easy, in tool 3, the score ranges from noknowledge to full knowledge, in tool 5, from notat all familiar to completely familiar, and in tool 6, from very unconfident to very confident. The items in tool 7 are assessed on a scale ranging from strongly disagree to strongly agree.

    Table 1. The distribution of sociodemographics and self-rated health.
    TotalStatistics (N=246), n (%)
    Sex

    		Female137 (55.7)

    		Male109 (44.3)
    Education

    		Comprehensive school19 (7.2)

    		Short education70 (28.5)

    		Medium education84 (34.1)

    		Long education65 (26.4)
    Self-rated health

    		Poor8 (3.3)

    		Less well61 (24.8)

    		Well108 (43.9)

    		Extremely well59 (23.9)

    		Excellent9 (3.7)
    Patients’ condition

    		Diabetes92 (37.4)

    		Other154 (62.6)

    Statistical Analyses

    Descriptive statistics are reported as means and interquartile range for age, educational level, and self-rated health. Differences in scores between male and female and users and nonusers of sundhed.dk and NemID were tested using the nonparametric Mann-Whitney test. Differences between users and nonusers of sundhed.dk and NemID with respect to sex were tested using Pearson Chi-square test. We tested for correlation among eHL and age, educational level, and self-rated health. We interpreted the strength of the correlation in accordance with Brace (weak ≤±.2, ±.3 to .6 moderate, strong ≥±.7) [31]. A Bonferroni correction was made for univariate analyses for each of the 2 tools relating sociodemographic factors to each of the tools’ 7 dimensions (alpha/number of hypotheses→.05/7=.0071) [32].

    Ethics

    The research complied with the Helsinki declaration, and the study was approved by the Danish Data Protection Agency (2012-58-004 under the capital Region of Denmark local record number HGH-2018-021 I-suite 06245). Information about the survey was given to the patients in accordance with the inclusion criterion, and written informed consent was obtained beforehand from all the participants.


    Use of Digital Services Among Outpatients

    The results showed that 142/246 (57.7%) of the outpatients were in contact with their GP via email or econsultation on the GP’s website. The use of NemID was widespread: 234/246 (95.1%) patients had used it in the previous 6 months to communicate with public authorities, access home banking or a Web portal for citizens.

    A total of 133/246 (54.1%) patients had visited sundhed.dk within the previous 3 months; the majority of patients used the website to access their own eHealth record (Figure 2).

    There was no difference between males and females (P=.87) age (P=.22), self-rated health (P=.09), or educational level (P=.29) between users and nonusers of sundhed.dk. Users of NemID had a lower mean age 56 (45-68) years versus 67 (54-82) years (P=.01) and higher score of self-rated health, 3.0 (2-4) versus 2.2 (2-3) (P=.001), than nonusers. There was no difference in educational level between users and nonusers of NemID (P=.14), and there were no differences in usage between males and females (P=.68).

    Percentages in Figure 2 are calculated on the basis of 133 users of sundhed.dk.

    In total, 202 patients of the 234 patients with nemID (86.3%) had activated the functionality of receiving an email notification when an official institution sent a letter to the e-Boks. Only 6.4% (15/234) patients were assisted by friends or family in the use of different features in the digital mailbox. A total of 4.5% (11/234) patients were exempted from using the mandatory digital mailbox.

    In dimensions 1, 3, 5, 6, and 7, scores of eHLQ dimensions were higher for users than for nonusers for both NemID and sundhed.dk. In dimension 2, understanding of health concepts and language, users of sundhed.dk but not users of NemID scored significantly higher, and in dimension 4, feeling safe and in control, only NemID users but not sundhed.dk users scored higher than nonusers (Table 2).

    Results from the eHLA toolkit showed that users of sundhed.dk and NemID scored significantly higher in tools 2 (self-assessed health literacy), 5 (digital familiarity), and 7 (digital incentives). In addition, users of sundhed.dk also scored significantly better in tool 3 (familiarity with health and disease), and users of NemID scored significantly better in tool 6 (digital confidence; Table 3).

    ‎
    Figure 2. The participants’ (N=133) use of functionalities on sundhed.dk. eHealth: electronic health; GP: general practitioner.
    View this figure
    Table 2. Differences in the 7 dimensions in electronic health literacy questionnaire between users and nonusers of sundhed.dk and NemID.
    Dimension name in eHLQaMeanUsers of sundhed.dkUsers of NemID
    Yes/No, NMean (IQRb)P valueYes/No, NMean (IQR)P value
    1. Using technology to process health information2.7Yes, 1322.9 (2.6-3.4)<.001Yes, 2302.8 (2.4-3.2)<.001
    No, 942.5 (2.2-3.0)
    		No, 81.5 (1.0-2.0)
    2. Understanding of health concepts and language3.1Yes, 1323.2 (3.0-3.6).004Yes, 2303.2 (2.8-3.4).56
    No, 943.0 (2.6-3.4)
    		No, 82.9 (1.9-3.8)
    3. Ability to actively engage with digital services3.0Yes, 1323.1 (2.8-3.6)<.001Yes, 2303.0 (2.6-3.6)<.001
    No, 942.8 (2.4-3.4)
    		No, 81.6 (1.0-2.0)
    4. Feel safe and in control2.8Yes, 1312.9 (2.6-3.2).58Yes, 2282.9 (2.6-3.2).003
    No, 932.8 (2.5-3.2)
    		No, 82.1 (1.6-2.6)
    5. Motivated to engage with digital services2.7Yes, 1312.9 (2.6-3.4)<.001Yes, 2292.8 (2.4-3.2)<.001
    No, 952.5 (2.0-3.0)
    		No, 91.6 (1.2-2.0)
    6. Access to digital services that work2.7Yes, 1332.8 (2.3-3.0).007Yes, 2312.7 (2.3-3.0).001
    No, 952.6 (2.2-3.0)
    		No, 92.0 (1.7-2.5)
    7. Digital services that suit individual needs2.6Yes, 1302.7 (2.3-3.0).005Yes, 2262.6 (2.3-3.0).001
    No, 932.4 (2.0-3.0)
    		No, 91.7 (1.0-2.0)

    aeHLQ: eHealth literacy questionnaire.

    bIQR: interquartile range.

    Table 3. Differences in the 7 tools in electronic health literacy assessment between users and nonusers of sundhed.dk and NemID.
    Tool in eHLAaMeanUsers of sundhed.dkUsers of NemID
    Yes/No, NMean (IQRb)P valueYes/No, NMean (IQR)P value
    1. Functional health literacy9.5Yes, 1209.5 (9-10).75Yes, 2079.5 (9-10).89
    No, 869.4 (9-10)
    		No, 89.5 (9-10)
    2. Self-assessed health literacy3.3Yes, 1233.3 (2.9-3.7).004Yes, 2173.2 (2.9-3.6).007
    No, 933.1 (2.8-3.3)
    		No, 112.8 (2.6-3.2)
    3. Familiarity with health and disease3.1Yes, 1243.3 (3.0-3.8).006Yes, 2213.1 (2.6-3.8).58
    No, 952.9 (2.4-3.8)
    		No, 103.0 (2.8-3.4)
    4. Knowledge of health and disease9.7Yes, 1199.9 (9-12).01Yes, 2139.6 (8-11).41
    No, 929.3 (8-10)
    		No, 1010.1 (8-12)
    5. Digital familiarity3.5Yes, 1253.7 (3.5-4.0)<.001Yes, 2203.6 (3.5-4.0)<.001
    No, 953.2 (2.8-4.0)
    		No, 121.7 (1.0-2.3)
    6. Digital confidence3.4Yes, 1293.6 (3.3-4.0).02Yes, 2253.5 (3.3-4.0)<.001
    No, 923.3 (2.8-4.0)
    		No, 81.7 (1.0-2.3)
    7. Digital incentives3.5Yes, 1333.6 (3.3-4.0).005Yes, 2303.5 (3.0-4.0)<.001
    No, 933.3 (2.8-4.0)
    		No, 82.0 (1.3-2.8)

    aeHLA: eHealth literacy assessment.

    bIQR: interquartile range.

    Relation Among Electronic Health Literacy Questionnaire, Electronic Health Literacy Assessment and Age, Sex, Education, and Self-Rated Health

    Age is weakly and negatively correlated to eHLQ dimension 3 (ability to actively engage with digital services; Table 4). Using the Mann-Whitney test, we did not find any differences between the eHLQ scores for males and females. Educational level was weakly and negatively correlated to dimensions 4 (feel safe and in control) and 6 (access to digital services that work; Table 4).

    Patients’ self-rated health showed a positive, weak correlation with 4 of the 7 dimensions: 1 (using technology to process health technologies), 3 (ability to actively engage with digital services) 5 (motivated to engage with digital services) and 6 (access to digital services that work).

    Three of the 7 eHLA tools were associated with age: tool 6 (digital confidence was moderate, negative correlate). Tools 5 (digital familiarity) and 7 (digital incentives) showed a weak negative correlation. Educational level was weakly and positively correlated with tools 4 (knowledge of health and disease) and 5 (digital familiarity). Self-rated health was weakly and positively correlated with 2 (self-assessed health literacy), 5 (digital familiarity), 6 (digital confidence), and 7 (digital incentives;Table 5). The Mann-Whitney test for differences between sexes revealed a significantly higher score for males than females in tool 5 (digital familiarity, P=.005).

    Table 4. The correlations among dimensions for electronic health literacy questionnaire and age, education, and self-rated health.
    Dimension name in eHLQaAgeEducationSelf-rated healthb
    CoefficientP valueCoefficientP valueCoefficientP value
    1. Using technology to process health information−.12.010.09.080.16.002
    2. Understanding of health concepts and language−.02.610.14.010.1.05
    3. Ability to actively engage with digital services−.23<.0010.09.080.18.001
    4. Feel safe and in control−.01.8−.17.0020.08.12
    5. Motivated to engage with digital services−.10.030.04.460.23<.001
    6. Access to digital services that work−.07.1−.14.0050.14.005
    7. Digital services that suit individual needs−.09.05−.08.130.13.01

    aeHLQ: eHealth literacy questionnaire.

    bSelf-rated health: 1=poor health, 5=excellent health.

    Table 5. The correlation between tools in electronic health literacy assessment and age, education, and self-rated health.
    Tool in eHLAaAgeEducationSelf-rated healthb
    CoefficientP valueCoefficientP valueCoefficientP value
    1. Functional health literacy−.08.13.16.01.05.45
    2. Self-assessed health literacy−.05.31<.001.95.18.001
    3. Familiarity with health and disease.08.09.11.05.03.62
    4. Knowledge of health and disease.11.02.15.007.03.60
    5. Digital familiarity−.25<.001.22<.001.25<.001
    6. Digital confidence−.34<.001.13.02.20<.001
    7. Digital incentives−.17<.001.11.04.20<.001

    aeHLA: eHealth literacy assessment.

    bSelf-rated health: 1=poor health, 5=excellent health.


    The introduction of the 2 new, recently validated multidimensional measures of eHL, eHLQ and eHLA toolkit, allows us to examine patients’ digital behavior from a multifaceted approach, offering a better understanding of whether knowledge, skills, perception, or experiences are related to usage of digital services. As described in the following, this offers a richer understanding than just judging the users’ capabilities on the basis of their sociodemographic data such as age, sex, educational level, and self-rated health.

    Usage of Digital Services

    The lack of a difference with respect to age and sex between the users and nonusers of the digital health service sundhed.dk corresponds with the findings of Siren and Stellefson [7,33].

    We did find a difference in age between users and nonusers of the public digital service NemID. This may be explained by the relatively high adoption of NemID in the Danish society; consequently, those not using NemID are mainly excluded because of high age and disabilities. The latter is also supported by the finding that nonusers also had a lower score of self-reported health. Whether this relatively small but vulnerable group of 5% can benefit from digital inclusion remains to be investigated, but the health professionals should be aware of this particular group.

    Here, multidimensional measures such as eHLQ and eHLA can add to our understanding of areas that might need to be addressed, as discussed in the following.

    Although 95% of the participants in the study use NemID to access digital services, less than 60% of the participants have been in contact with their GP electronically or have taken advantage of the functions available on sundhed.dk. This could be explained by the fact that some NemID services, for example, the electronic mailbox for communications from public authorities, are mandatory to use, whereas electronic contact to one’s GP and use of the portal sundhed.dk are voluntary.

    The conflicting results in the literature regarding the association among eHealth usage and users’ age, sex, or educational level may be explained by the context and research question. A study where the actual usage is reported as in this study may differ from studies where the focus is on, for example, the users’ willingness to use a health portal or a medical record, such as reported by Trubitt et al, which found an association with age and education. This is in contrast to our finding that there were no differences between users and nonusers of sundhed.dk with respect to age and education [10].

    Electronic Health Literacy and Usage of Digital Services

    In general, users of NemID and sundhed.dk scored higher in most dimensions of the eHLQ. Moreover, users tended to score higher in 4 of the 7 eHLA tools for both sundhed.dk (tools 2, 3, 5, and 7) and NemID (tools 2, 5, 6, and 7). It should be noticed that sundhed.dk users had a higher score in the eHLQ dimension 2 as well as 2 of the 4 health literacy tools in eHLA, whereas the 2 other tools in eHLA that related to functional tests did not differ. This suggests that the users of sundhed.dk, because of their better understanding of the health-related language and concepts, are better equipped to understand the information and interact with the services in the health portal.

    The assumption that health literacy is a determinant of the usage of the digital health services can be supported by the finding that NemID users did not differ from nonusers with respect to eHLQ dimension 2 and the eHLA tools 1, 3, and 4, which indicates that usage of other public digital services is not related to the users’ health literacy.

    It is noteworthy that we did not find any differences in scores for the eHLQ dimension 4 (feeling safe and in control) between users and nonusers of sundhed.dk. This contrasts with our findings for users of NemID; here, a difference was found. An interpretation of this finding could be that although trust has no significance for the decision to use eHealth technologies such as the health portal sundhed.dk, the relatively few nonusers of NemID may have concerns about safety because of a lack of insight into how the services function.

    Our finding is in accordance with Siren and Knudsen [7], who also found that the feeling of being safe and in control is not in itself a significant factor for using digital health services. It can be speculated whether being a patient in the health care sector makes people more trusting when they access and use health services.

    Electronic Health Literacy in Relation to Sociodemographic Data

    The eHL level only differed between males and females with respect to 2 of the eHLA tools, but it did not differ in any of the dimensions in the eHLQ.

    It is of particular interest that the digital tool 5 (digital familiarity) in the eHLA toolkit showed a higher score in males compared with females. This is in accordance with the findings of Hargittai et al, who measured digital literacy in college students using a scale that has inspired the construction of this tool [34].

    Scales of eHLQ and eHLA that relate to digital skills showed significant negative correlations with age. This may reflect that people of older age generally have more problems finding information, which may contribute to less motivation to engage with technology [15]. Moreover, an increased need of health services by elder individuals may give rise to a feeling that the services are not sufficiently suited to their needs. This was reflected in a negative correlation between age and eHLQ dimension 7 (access to digital services that suit individual needs; P=.05); however, it was not significant after a correction for multiple comparisons using Bonferroni.

    For the eHLQ dimensions, educational level was weakly, negatively correlated with dimension 4 (feel safe and in control) and dimension 6 (access to digital services that work). The negative correlation in dimensions 4 (feel safe and in control) and 6 (access to digital services that work) may be because of a general skepticism toward digital services. The finding that people with a higher educational level tend to have less trust is in alignment with a recent study from a European Union (EU) project evaluating an eHealth solution, the Health Monitor, with both patients and health professionals. In this study, health professionals tended to have more concerns about data privacy than lay people [18].

    In contrast to the negative correlation between the 2 eHLQ dimensions and 2 of the eHLA tools; tool 4 (knowledge of health and disease) and tool 5 (digital familiarity) were positively correlated with the educational level. This finding is inconclusive, as several of the other tools within both health literacy and digital literacy are not related to educational level.

    Although studies based upon eHEALS point to an association among eHL and age and educational level [15-17,35], our data are, apart from the negative correlation between age and the digital scales, not conclusive. Combined with our finding, that age and educational level do not differ between users and nonusers of sundhed.dk, this suggests that other factors may contribute to the adoption of eHealth service usage.

    Electronic Health Literacy and Self-Rated Health

    Most interestingly, eHLA’s tool 2, which was derived from the European Health Literacy Survey HLS-EU, had a positive correlation with self-rated health similar to earlier reports in relation to the full HLS-EU instrument [36]. As self-rated health is often positively associated with health literacy [37], it would be expected that those tools and dimensions that are related to health literacy would also be positively correlated with self-rated health. Interestingly, eHLQ dimension 2 (understanding health concepts and language) as well as 3 of the 4 health literacy tools (1, 3, and 4) in eHLA did not exhibit such a correlation. However, 4 of the eHLQ scales as well as the 3 digital literacy tools in eHLA demonstrated such a correlation.

    This may reflect that people who perceive their own health to be good are more engaged in information and technology and are more motivated than those with poor self-rated health. This is in contrast to Neter and Brainin (2012) who examined the relation between self-rated health and eHL in an Israeli population study and to Milne et al (2015) who examined the relation between perceived health and eHL in patients with lung cancer. This may be explained by differences in the instrument used to assess the self-reported health as well as measurements of eHL [17,38].

    Limitations of the Study

    A limitation of the study is that it is an observational study conducted in an outpatient clinic in a Danish region where people in general have a rather high sociodemographic profile with respect to income and education.

    It should also be noted that the data about digital behavior build on self-reported information and not data acquired from the systems. This may introduce a bias.

    Another limitation is the mandatory usage of NemID in Denmar, which may have resulted in a selected population for this investigation compared with other countries with a lower degree of digitalization.

    Using 2 instruments, each with 7 scales in the evaluation, may have resulted in a multisignificance problem, where we can have obtained type I error. We have accommodated this by applying the Bonferroni correction. On the other hand, this may have introduced a risk of type II error because of insufficient power of the study caused by the sample size [32]. Further studies that are designed to test our findings, with enough power, are necessary before further conclusions can be drawn.

    Conclusions

    Our results contribute to the growing knowledge about which factors are important for use of digital health services. Our data show that there were no significant differences between users and nonusers of the digital health service sundhed.dk with regard to age, sex, or educational level. Therefore, these factors alone cannot be used to guide health professionals to understand their patients’ adoption and usage of sundhed.dk. However, significant differences were identified between users and nonusers in almost of all the tools of eHLA and eHLQ dimensions. This supports the notion that skills, motivation, and experience of health and digital services are related to the adoption and usage of technology [6,7].

    The results emphasize that multifaceted measurements of eHL may be able to capture the factors important to the adoption of digital health services and thereby serve to guide health professionals to better understand and support their patients to obtain the full benefits of the increasing digitalization of the health care sector.

    Further studies are needed to identify how the tools or the underlying dimensions can be best used to inform the clinicians and facilitate that more patients take advantage of digital health services and technologies and benefit from the ever-expanding evolution of digital health.

    Acknowledgments

    The authors would like to thank the nurses at the outpatient clinic at Gentofte Hospital for their help with data collection and all the patients willing to participate and complete the questionnaire. The authors also wish to thank Thanks Steen Ladelund for the initial statistical discussions. The authors thank Ms. Emily Duminski for helping in preparing the manuscript for submission. AK is a PhD fellow at the Danish Multiple Sclerosis Society and is also supported by the Innovation Fund Denmark.

    The University of Copenhagen owns the IPR to eHLA. Together with Deakin University, Melbourne, Australia, they also own the IPR of eHLQ. eHLA can be obtained via a free license from the University of Copenhagen. eHLQ can be licensed for free for noncommercial use from either the University of Copenhagen or Deakin University.

    Authors' Contributions

    KAH, AK, DO, SEN, LK, MER, and GF participated in the design of the study. KAH and SEN handled the collection of data. MER in collaboration with GF collected the patients’ clinical data from eHealth records. Management of data was supported by KAH, AK, DO, and LK. KAH, LK, and DO performed the statistical analysis supported by AK and Steen Ladelund. KAH drafted the first version of the manuscript assisted by DO. LK and AK wrote the second draft assisted by DO. MER and GF contributed in interpreting the data, results, and in the discussion. All authors critically revised the text and read and approved the final manuscript.

    Conflicts of Interest

    None declared.

    1. Kierkegaard P. eHealth in Denmark: a case study. J Med Syst 2013 Dec;37(6):9991. [CrossRef] [Medline]
    2. Danish Agency for Digitalisation.   URL: https://digst.dk/tal-og-statistik/ [accessed 2019-03-01] [WebCite Cache]
    3. Danish Agency for Digitalisation. NemID   URL: https://digst.dk/it-loesninger/nemid/ [accessed 2019-03-01] [WebCite Cache]
    4. Ministry of Interior and Health. The history of health   URL: https://www.sundhed.dk/borger/service/om-sundheddk/om-organisationen/historien-om-sundheddk/ [accessed 2019-03-01] [WebCite Cache]
    5. Wyatt SM, Thomas G, Terranova T. They Came, They Surfed, They Went Back to the Beach: Conceptualizing. Oxford, UK: Oxford Unviersity Press; 2002:23-40.
    6. O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak 2016 Sep 15;16(1):120 [FREE Full text] [CrossRef] [Medline]
    7. Siren A, Knudsen SG. Older adults and emerging digital service delivery: a mixed methods study on information and communications technology use, skills, and attitudes. J Aging Soc Policy 2017;29(1):35-50. [CrossRef] [Medline]
    8. Runz-Jørgensen SM, Schiøtz ML, Christensen U. Perceived value of eHealth among people living with multimorbidity: a qualitative study. J Comorb 2017;7(1):96-111 [FREE Full text] [CrossRef] [Medline]
    9. van Middelaar T, Beishuizen CR, Guillemont J, Barbera M, Richard E, Moll VC, HATICE consortium. Engaging older people in an internet platform for cardiovascular risk self-management: a qualitative study among Dutch HATICE participants. BMJ Open 2018 Dec 21;8(1):e019683 [FREE Full text] [CrossRef] [Medline]
    10. Trubitt M, Alozie O, Shokar G, Flores S, Shokar NK. Patterns and correlates of internet use, cell phone use, and attitudes toward patient portals among a predominantly Mexican-American clinic population. Telemed J E Health 2018 Feb 02;24(11):861-869. [CrossRef] [Medline]
    11. Chuttur M. Association for Information Systems AIS Electronic Library (AISeL). 2009. Overview of the Technology Acceptance Model: Origins, Developments and Future Directions   URL: https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1289&context=sprouts_all
    12. Monkman H, Kushniruk AW. The consumer health information system adoption model. Stud Health Technol Inform 2015;218:26-31. [Medline]
    13. Kayser L, Kushniruk A, Osborne RH, Norgaard O, Turner P. Enhancing the effectiveness of consumer-focused health information technology systems through eHealth literacy: a framework for understanding users' needs. JMIR Hum Factors 2015 May 20;2(1):e9 [FREE Full text] [CrossRef] [Medline]
    14. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res 2006 Jun;8(2):e9 [FREE Full text] [CrossRef] [Medline]
    15. van der Vaart R, Drossaert CH, de Heus M, Taal E, van de Laar MA. Measuring actual eHealth literacy among patients with rheumatic diseases: a qualitative analysis of problems encountered using Health 1.0 and Health 2.0 applications. J Med Internet Res 2013 Feb;15(2):e27 [FREE Full text] [CrossRef] [Medline]
    16. Knapp C, Madden V, Wang H, Sloyer P, Shenkman E. Internet use and eHealth literacy of low-income parents whose children have special health care needs. J Med Internet Res 2011;13(3):e75 [FREE Full text] [CrossRef] [Medline]
    17. Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res 2012 Jan;14(1):e19 [FREE Full text] [CrossRef] [Medline]
    18. Griebel L, Kolominsky-Rabas P, Schaller S, Siudyka J, Sierpinski R, Papapavlou D, et al. Acceptance by laypersons and medical professionals of the personalized eHealth platform, eHealthMonitor. Inform Health Soc Care 2017 Sep;42(3):232-249. [CrossRef] [Medline]
    19. Paige SR, Krieger JL, Stellefson M, Alber JM. Corrigendum to “eHealth literacy in chronic disease patients: an item response theory analysis of the eHealth literacy scale (eHEALS)”. Patient Educ Couns 2017;100(12):320-326. [CrossRef] [Medline]
    20. Petrič G, Atanasova S, Kamin T. III literates or illiterates? Investigating the eHealth literacy of users of online health communities. J Med Internet Res 2017 Oct 04;19(10):e331 [FREE Full text] [CrossRef] [Medline]
    21. Richtering SS, Morris R, Soh SE, Barker A, Bampi F, Neubeck L, et al. Examination of an eHealth literacy scale and a health literacy scale in a population with moderate to high cardiovascular risk: Rasch analyses. PLoS One 2017;12(4):e0175372 [FREE Full text] [CrossRef] [Medline]
    22. Chang A, Schulz P. The measurements and an elaborated understanding of Chinese eHealth Literacy (C-eHEALS) in chronic patients in China. Int J Environ Res Public Health 2018;15(7):1553. [CrossRef] [Medline]
    23. Griebel L, Enwald H, Gilstad H, Pohl AL, Moreland J, Sedlmayr M. eHealth literacy research-Quo vadis? Inform Health Soc Care 2018 Dec;43(4):427-442. [CrossRef] [Medline]
    24. Norgaard O, Furstrand D, Klokker L, Karnoe A, Batterham R, Kayser L. The e-health literacy framework: A conceptual framework for characterizing e-health users and their interaction with e-health systems. Knowledge Management & E-Learning: An International Journal (KM&EL). . ISSN 2015;7(4):2073 http://dro.deakin.edu.au/view/DU:30084909.
    25. Karnoe A, Furstrand D, Christensen KB, Norgaard O, Kayser L. Assessing competencies needed to engage with digital health services: development of the eHealth literacy assessment toolkit. J Med Internet Res 2018 May 10;20(5):e178 [FREE Full text] [CrossRef] [Medline]
    26. European Commission. International Standard Classification of Education (ISCED), 14   URL: https:/​/ec.​europa.eu/​eurostat/​statistics-explained/​index.php/​International_Standard_Classification_of_Education_(ISCED) [accessed 2019-03-09] [WebCite Cache]
    27. European Commission.   URL: https://ec.europa.eu/ploteus/search/site?f[0]=im_field_entity_type%3A97) [accessed 2019-03-01] [WebCite Cache]
    28. Idler EL, Benyamini Y. Self-rated health and mortality: a review of twenty-seven community studies. J Health Soc Behav 1997 Mar;38(1):21-37. [Medline]
    29. Idler E, Leventhal H, McLaughlin J, Leventhal E. In sickness but not in health: self-ratings, identity, and mortality. J Health Soc Behav 2004 Sep;45(3):336-356. [CrossRef] [Medline]
    30. Kayser L, Karnoe A, Furstrand D, Batterham R, Christensen KB, Elsworth G, et al. A multidimensional tool based on the eHealth literacy framework: development and initial validity testing of the eHealth Literacy Questionnaire (eHLQ). J Med Internet Res 2018 Feb 12;20(2):e36 [FREE Full text] [CrossRef] [Medline]
    31. Brace N, Kemp R, Sneglar R. SPSS for Psychologists: A Guide to Data Analysis Using SPSS for Windows (Versions 12 and 13). Basingstoke, United Kingdom: Palgrave Macmillan; 2006:-.
    32. Ranstam J. Multiple P-values and Bonferroni correction. Osteoarthritis Cartilage 2016 May;24(5):763-764 [FREE Full text] [CrossRef] [Medline]
    33. Stellefson ML, Shuster JJ, Chaney BH, Paige SR, Alber JM, Chaney JD, et al. Web-based health information seeking and eHealth literacy among patients living with Chronic Obstructive Pulmonary Disease (COPD). Health Commun 2018 Dec;33(12):1410-1424. [CrossRef] [Medline]
    34. Hargittai E. An update on survey measures of web-oriented digital literacy. Soc Sci Comput Rev 2008 Apr 07;27(1):130-137. [CrossRef]
    35. Park H, Moon M, Baeg JH. Association of eHealth literacy with cancer information seeking and prior experience with cancer screening. Comput Inform Nurs 2014 Sep;32(9):458-463. [CrossRef] [Medline]
    36. Sørensen K, Pelikan JM, Röthlin F, Ganahl K, Slonska Z, Doyle G, HLS-EU Consortium. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health 2015 Dec;25(6):1053-1058 [FREE Full text] [CrossRef] [Medline]
    37. Kickbush I, Pelikan J, Apfel F, Tsouros A. Health literacy: the solid facts. Copenhagen, Denmark: World Health Organization; 2013.
    38. Milne RA, Puts MT, Papadakos J, Le LW, Milne VC, Hope AJ, et al. Predictors of high eHealth literacy in primary lung cancer survivors. J Cancer Educ 2015 Dec;30(4):685-692. [CrossRef] [Medline]

    ‎
    e-Boks: Patient’s digital mailbox
    eHEALS: eHealth literacy scale
    eHealth: electronic health
    eHL: eHealth literacy
    eHLA: eHealth literacy assessment
    eHLF: eHealth literacy framework
    eHLQ: eHealth literacy questionnaire
    EQF: European Qualifications Framework
    GP: general practitioner
    ICT: information and communications technology
    ISCED: International Standard Classification of Education

    Edited by G Eysenbach; submitted 14.07.17; peer-reviewed by E Neter, B Xie; comments to author 24.11.17; revised version received 08.03.18; accepted 07.02.19; published 05.04.19

    Copyright

    ©Kamila Adellund Holt, Astrid Karnoe, Dorthe Overgaard, Sidse Edith Nielsen, Lars Kayser, Michael Einar Røder, Gustav From. Originally published in the Interactive Journal of Medical Research (http://www.i-jmr.org/), 05.04.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.i-jmr.org/, as well as this copyright and license information must be included.