Abstract

To address the need for cost-of-care conversations in prenatal care, the CONTINUE (cost conversations in routine prenatal care) study was conducted with prenatal patients to better understand the benefits of implementing a cost-of-care conversation “cost” tool into routine obstetrics (OB) care. This research team conducted a multi-phase, mixed-methods research study to identify 18 target benefits of a cost tool to initiate and standardize cost-of-care conversation and, subsequently, developed and validated a cost tool. The novel cost tool was piloted and data pertaining to target benefits were collected through interviews and surveys. To assess the cost tool’s utility, data from interviews and surveys were unified and exploratory factor analysis was performed to classify the underlying factor structure of the 18 benefit item responses. Data includes patients’ self-reported ratings of benefit items, as collected from third trimester prenatal patient participants who received the tool at the beginning of their prenatal care in three midwestern-based hospital clinics within one healthcare system. The present study details the factor analysis approach used to identify the three final factors that emerged from the data representing prenatal patients benefits of utilizing a cost tool. This analysis provides a framework for exploring patient-specific predictors of experiencing identified benefits (i.e. factors) of a novel cost tool incorporated into routine OB care.

1. Introduction

Although there are many reasons why patients miss prenatal appointments, research indicates that some groups of women are more likely to forgo prenatal care because of logistical and issues like hourly jobs, transportation, lack of childcare, and inadequate social support ¹. Cost-of-care conversations, or conversations between patients and providers about logistical and cost-related factors that exist for patients, could help mitigate barriers to prenatal care. Research has shown that cost-of-care conversations can lead to better health outcomes by improving patient adherence to treatment plans and enhancing patient engagement in the management of their care ^{2, 3}. Because prenatal care visits are typically scheduled to be 10-15 minutes each and biomedical needs are often prioritized in these visits, conversations about unmet patient needs and other cost-related barriers to care are not frequently initiated ^{4, 5}. The use of tools to supplement otherwise routine prenatal care have shown promise, particularly to promote communication between patients and providers ⁶.

To address the need for cost-of-care conversations in prenatal care, a multi-phased, mixed-methods pilot study (“CONTINUE”, or cost conversations in routine prenatal care) was conducted with pregnant patients to identify the benefits of implementing a cost tool into routine obstetrics (OB) care. In the first phase of the CONTINUE study, a cost tool was created to address the need for a means to initiate and standardize cost-of-care conversations in prenatal care. The tool was created to be personalized to the patient, reflecting the prenatal care plan decided between the patient and provider at the onset of prenatal care. The tool’s purpose is to provide patients with visual forecasting of their remaining prenatal care, including expected appointments, labs, scans, and tests, as well as estimated time required for each visit and other helpful resources that help patients navigate their care. The tool was designed to benefit patients; therefore, during the first phase of the CONTINUE study, 18 potential benefits of the cost tool (“target benefit items”) were also hypothesized ⁷.

In the second and final phase of the CONTINUE study, the cost tool was tailored to three different OB settings and prenatal patients were interviewed to validate the target benefit items as true potential benefits of the cost tool. After target benefit items were validated, the cost tool was incorporated into routine OB care in the same three clinics serving a diverse patient population. Patients were given the cost tool to use throughout their prenatal care and, at the end of their prenatal care, patients rated their agreement with each of the 18 benefits items they experienced from the use of the cost tool in interviews or surveys.

The purpose of this paper is two-fold. First, this paper will describe how 18 target benefit item responses derived from two different data collection methods, on different measurement scales, in a mixed-methods study were unified in order to conduct a factor analysis. Second, this paper will highlight how factor analysis can be applied in a mixed-methods pilot study to reveal key constructs underlying multiple items, thus enhancing findings. This team utilized factor analysis to identify common underlying factors that explain interrelationships between 18 target benefit items. The identification of factors derived from this analysis will establish the main benefits demonstrated among prenatal patients due to the use of a cost tool, and provide researchers a framework for exploring these benefits as outcomes of the use of a cost tool among prenatal patients. This has implications for widespread use of this tool, and others like it, to bridge the gap in cost-of-care conversations between prenatal patients and their providers.

2. Methods

The CONTINUE study is a multi-phased, mixed-methods observational pilot study that documented the effects of implementing a validated cost tool into routine care in three OB clinics in a midwestern-based healthcare system. Site providers were trained on how to personalize and share the cost tool with prenatal patients. The cost tool was available in both English and Spanish and offered to patients in their preferred language. Providers were encouraged to share the tool to all early-stage prenatal patients but were given discretion regarding with whom it was given. Tool implementation occurred between September 2020 and July 2021, after study approval was granted by the health system’s institutional review board (#20-264E).

At the end of tool implementation, prenatal patients who received the cost tool were passively recruited to participate, meaning flyers and posters with study contact info in both English and Spanish were displayed in clinic waiting rooms and clinic rooms for patients to initiate participation in either an in-person or virtual semi-structured interview or electronic survey to provide feedback on their experiences with the cost tool.

Prenatal patient participants were at least 18 years old, at least 27 weeks pregnant, and had received the tool at least 8 weeks prior to being interviewed or surveyed. Participants self-selected their participation method, as interview (in-person at a public location of the patient’s choosing or virtual) or electronic survey, as well as their preferred language (English or Spanish).

Target benefit items were originally determined from interviews with a different sample of prenatal patients using a card sort method validated in the field of human-centered design (HCD) ⁸. Broadly, per Chen, Neta and Roberts ⁸, the card sort method engages patients in the implementation effort and leads to better tailored and more adaptable products – in this study, a cost tool. In the case of this study, patient interviews and surveys were conducted to assess uptake, use, and benefits of the tool, utilizing the 18 target benefit items as outcomes of interest to assess patient-reported experiences with using the cost tool. Some examples of benefit items include: “The pregnancy planning guide helped me navigate insurance more effectively”, “The pregnancy planning guide helped me feel my financial situation was being considered”, “The pregnancy planning guide helped me show up on time”. Observed data collected from patient interviews and surveys were unified for analysis, as described below. Now, factor analysis is being applied to identify common factors underlying tool benefit items.

Factor analysis, in general, refers to all methods of data analysis that use matrix factors. It is a technique used to reduce a large number of items into a fewer number of factors, or constructs encompassing shared underlying characteristics of the original, observed items used to create them ⁹. The basic assumption of factor analysis is that, for a collection of observed items, there are underlying variables called factors that can explain the interrelationships among those items. Therefore, the aim of factor analysis is to reveal any latent factors that cause the observed items to covary, thereby reducing the observed items into a smaller number of common factors ¹⁰. It’s a flexible approach to a pragmatic method of analysis. Because there was no a priori theory of an underlying factor structure, exploratory factor analysis (EFA) was applied in this analysis, with a goal to identify factors. The goal was to eventually use these factors as outcomes experienced due to use of the tool in follow-up pilot study analyses.

Data (i.e. observed item responses) come from a pilot study assessing potential benefits of a pregnancy support tool implemented to encourage cost conversations between patients receiving prenatal care and their providers. The study included 71 prenatal patient participants receiving care from one of three OB clinics within a single, large midwestern-based healthcare system who had received the cost tool from their provider early in their prenatal care. The cost tool was available in both English or Spanish, and patients self-selected which language version they preferred to receive. Demographically, participants were primarily ages 26-35 (57.75%), White (71.43%), and Hispanic/Latino/Spanish (56.34%). Medically, participants were evenly split in insurance, risk status, parity; specifically, 56.34% were publicly insured during the pregnancy, 54.93% experienced low risk pregnancies, and 56.34% were experiencing subsequent pregnancies (56.34%). Finally, data were self-reported by patient participants via two different data collection methods, including interviews (52.11%) and surveys (47.89%). See Figure 1 for patient sample demographics.

Benefit items were assessed using one of the two data collection methods, as self-selected by participants. Because benefit items were derived from patient interviews using a card sort method during the initial phase of this mixed-methods study ⁷, the team captured benefit item experiences using the same method in the second and final phase. Consequently, the first method of data collection was a 60-minute interview. During the card sort activity within the interview, interview participants were given all 18 target benefit items on small cards and asked to sort target them into one of three categories: “Experienced”, “Neutral”, “Did not experience”. The second method of data collection was a 20-minute, 35-item self-report electronic survey. Four research team members agreed on the translation of all items from the card sort method into the 18 survey items. Survey participants were given all 18 target benefit items as survey statements and asked to rate each item into one of five categories: “Strongly Agree”, “Agree”, “Neutral”, “Disagree”, “Strongly Disagree”. Response options in interviews and surveys represented whether or not the target benefit was experienced, but response options in surveys also elicited the degree to which the target benefit was experienced or not.

For analysis purposes, benefit item responses from interviews and surveys were merged into one unified Likert-level system with three levels, resulting in survey data responses being collapsed from five categories into the three categories aligning with interview response options, specifically: Experienced (i.e. Strongly Agree and Agree), Neutral (i.e. Neutral) and Did not experience (i.e. Strongly Disagree and Disagree). Benefit item values were assigned numerical ratings (0=Did not experience, 1=Neutral, 2=Experienced) which were then used as the observed indicator values in EFA models to determine factor loadings.

Data management and analysis were performed by the study research team and conducted using SAS statistical software (Version 9.4; SAS Institute, Cary, NC). The 18 target benefit items were used as observed indicators in models. In the absence of data on the underlying structure of these constructs, the goal was the most flexible approach; therefore, EFA was decided a priori as the best approach. Factor analysis (proc factor) was performed on responses to the 18 target benefit items. The research team assumed the presence of common, unique and error variance in item responses, which is best accounted for by factor analysis ^{11, 12}. Furthermore, the priority of this analysis was to achieve the most useful interpretations to newly-defined dimensions under an assumption of common latent constructs ¹².

Maximum likelihood estimation (MLE; method=ml) was used to extract factors and was followed by an oblique rotation (rotation=promax). In the interest of flexibility, oblique rotation was chosen, as it allows factors to be correlated with one another and accounts for relationships between factors before determining an item’s relationship to the factor ^{13, 14}. Factor pattern and structure values were reviewed for final assignment of items within factors.

3. Results

Correlations between all items were initially explored. All correlations were moderately high, reiterating the appropriateness of oblique rotation ¹⁵. The full correlation matrix among all variables entered into the factor analysis is presented in Table 1. Of the 143 pairwise correlations, 139 correlations were significant at p<0.05 and 60 of the 139 correlations were significant at p<0.0001. Nonsignificant correlations were found between benefit 9 and benefits 13-16. Across all associations, the average absolute association was 0.43, a moderate correlation, suggesting the items were overlapping in construct and factor analysis was suitable.

An initial review of the factor analysis revealed small partial correlations compared to original correlations. Additionally, the Kaiser’s Measure of Sample Adequacy (MSA) was 0.84, suggesting there were enough variables in the analysis to reliably define common factors ¹⁶. Finally, the squared multiple correlations (SMCs) were all fairly large, indicating the results of the EFA approach would likely be similar to a PCA approach regardless ¹⁷; regardless, we moved forward with the EFA, as decided a priori.

In the first iteration of the factor analysis model, four criteria were used to select the most meaningful number of factors to retain ^{11, 18}. First, the eigenvalue-one criterion, or the criterion of eigenvalues being greater than 1.00, indicated up to three potential factors. Second, a review of the scree plot revealed substantial breaks at two or three factors, in line with the previous criterion. Third, the proportion of variance was 67.43% with 1 factor, 16.83% with a second factor, 6.52% with a third factor and 4.29% with a fourth factor, again suggesting the ideal final model would include either two or three factors for the most parsimonious model. EFA models with two and three factors were performed and reviewed by the research team. Items that cross-loaded, based on a definition of a regression coefficient within 0.02, were assigned to factors by the research team. Finally, the fourth criterion, interpretability, or interpreting the substantive meaning of the retained factors and verifying it makes sense with what is known about the constructs under investigation, was discussed among the research team to select the number of factors to retain.

The ultimate goal of this analysis approach was to identify the most interpretable model distinguishing factors that could serve as outcomes for further data analyses. Thus, this was considered when the final model was selected. Another a priori decision was to retain all 18 original benefit item, regardless of low standardized regression coefficients (<0.40). In cases where items loaded onto two factors and/or had low regression coefficients, the item was retained with the factor where it loaded best, per the factor structure correlations, or based on theoretical relevance.

The following results detail factor loading and correlation values for three different models: a three-factor model with oblique rotation (final chosen model) and, additionally, a two-factor model with oblique rotation and a three-factor model with orthogonal rotation, performed as sensitivity analyses to support and validate of results of the final selected three-factor model with oblique rotation. Final model communality estimates ranged from 0.355-0.857, indicating each item’s variance was moderately- to well-explained by the three factors.

Eight benefit items, 5-6 and 13-18, loaded onto Factor 1 in this factor solution, with loadings ranging from 0.403-0.863 and Factor 1 item correlations ranging from 0.555-0.862. This factor clearly showcased a logistics-related benefit element, aligning with the intentions of the tool, and the research team labeled Factor 1 as Logistics.

Five benefit items, 1-3 and 9-10, loaded onto Factor 2, with loadings ranging from 0.334-0.943 and Factor 2 item correlations ranging from 0.515-0.903. Given the a priori decision to preserve all items, despite low standardized regression coefficients, Items 2 and 3 were retained. Item 2 was retained with Factor 2 based on the highest loading correlation. Given very similar loading correlations of Item 3 with two factors, the research team members decided Item 3 fit best with Factor 2 based on theoretical relevance. Factor 2 was labeled by the research team as Efficacy.

Finally, five benefit items, 4,8,7, and 11-12, loaded onto Factor 3, with loadings ranging from 0.338-0.873 and Factor 3 item correlations ranging from 0.539-0.911. Despite low standardized regression coefficients, Items 8 and 12 were retained with Factor 3 based on the highest loading correlations. Benefit 11 loaded onto Factors 1 and 3 but was retained with Factor 3 based on the highest loading correlation. Factor 3 was labeled [Patient] Understanding.

See Table 2 for results of the final chosen three-factor model with oblique (promax) rotation and Figure 2 for a visual of the final model path diagram, representing all factors with standardized regression coefficients > 0.40.

Comparing the two-factor model with oblique rotation to the final chosen three-factor model, the same eight benefits (5-6, 13-18) loaded on Factor 1 in both the two- and three-factor models, with extremely similar item loading and correlation values. Item 17 had a low standardized regression coefficient but loaded highest with Factor 1. Factors 2 and 3 from the final chosen three-factor model were collapsed as one single factor in the two-factor model (Factor 2), with loadings ranging from 0.379-0.873 and item correlations ranging from 0.547-0.759. Despite Items 17 and 3 having low standardized regression coefficients, they were each retained with the Factors where they loaded highest (Factor 1 and 2, respectively). Final communality estimates ranged from 0.387-0.773. See Table 3 for the two-factor model results with oblique (promax) rotation.

Comparing the two and three-factor models with oblique rotation, in the three-factor model, item correlations were generally higher, suggesting greater variance accounted for by the separated factors, and communality estimates were larger, indicating a greater proportion of each item’s variance was explained by the separated factors. Furthermore, all factors within the three-factor model had five or more strongly loading items (.50 or better), indicating all three as solid factors ¹⁰. While there is no proven or universally accepted test to select the optimal model ¹⁹, the two-factor model was ultimately deemed less interpretable compared to the three-factor model by the research team.

As an additional sensitivity analysis, although oblique rotation approach decided a priori, a three-factor EFA model with an orthogonal (varimax) approach was performed, with items loading similarly as the final three-factor model with oblique rotation, thus corroborating the final selected model approach. The three-factor model with orthogonal rotation validated the interpretation of the final selected model. See Table 4 for the three-factor model results with orthogonal (varimax) rotation.

4. Discussion

The main purpose of this study was to identify key constructs underlying benefits experienced by prenatal patients’ due to use of the piloted cost tool. This study found three common underlying factors that encompassed the interrelationships between patients’ reported experiences as measured by the 18 benefit items: logistics, efficacy and [patient] understanding [of care plan]. EFA of the benefit items from prenatal patients who used the cost tool produced a three-factor model that is consistent with emerging themes from qualitative interviews conducted during the mixed-methods study. The original goal of the cost tool was to act as a means to initiate cost conversations between patients and providers. A cost-related theme, labeled in this study as logistics, was a very clear and unwavering factor that emerged from the data. Throughout the study, other potential benefits of the pilot tool emerged: a means to know and plan for what will happen (understanding) and a means to reinforce one is doing things right, or to feel in control (efficacy). Interestingly, both of these themes were well-supported by the data as aligning with factors in this analysis.

Additionally, this study provides an analytic solution for mixed-methods studies. Specifically, this study excavated 18 target benefits from in-depth interviews in a prior phase of this study ⁷. The 18 individual benefits were then assessed among sample participants using both interview and survey data collection methods in this pilot study. After unifying the data, specifically the response option scales across data collection methods, this manuscript details the analysis process for identifying the common latent factors underlying all 18 individual benefits. One factor (logistics) aligned clearly with the original intention of the tool, somewhat validating the construct validity of the target benefits. Furthermore, this analysis allowed for additional tool benefits of efficacy and understanding to be identified.

In addition to performing this factor analysis that identified three common factors underlying the 18 benefit items across the pilot study sample, it should be noted that this research team also examined individual benefit items experienced due to use of the cost tool, as well as the identified factors, by income groups within the sample, defined by public or private insurance ²⁰. Further, this team also detailed differences in cost-related conversations, specifically with whom sample prenatal participants were having them, and cost-specific benefits by income group due to use of the tool ²¹.

The greatest utility of this analysis was that it grouped a large number of individual benefit items into three broader and more comprehensive constructs. With these broader constructs in mind as experienced benefits, we will expand our pilot study to a larger group of prenatal patients and collect data using validated and standardized measures of the identified factors to further support the benefits of this cost tool. We hope to expand the use of this tool throughout multiple learning healthcare systems. This study will continue to explore the identified factors of logistics, efficacy and [patient] understanding [of care plan] as outcomes representing the utility of this cost tool. Furthermore, given so many items loaded on the logistics factor, separating items into two types of logistics – time and financial costs – will be explored. Next, we will identify demographic- and pregnancy-related factors associated with these factors as outcomes among prenatal patients.

5. Strengths

This study provides a framework for unifying response options collected from a mixed-methods study in order to pursue quantitative analysis. This study also detailed a step-by-step analysis approach to identifying the underlying factor structure of 18 unique items, as determined by prenatal patients’ benefit item responses via interview and survey. This study merges qualitative and quantitative methods to enhance findings from a pilot study seeking to understand patients’ experiences with a pilot tool.

6. Limitations

Because there is no “gold standard” tool or any other measure with which to compare and assess validity of these benefit items to measure tool utility, this current approach and the subsequent findings should be interpreted accordingly. Also, while ideal sample size for EFA is subjective and strict rules regarding sample size for EFA have mostly disappeared ¹⁰, a general consensus is between 3-10 participants per item or a reasonable absolute minimum sample size of 50 ²². In our study, 71 participants responded to the 18 items, equating to a 3.94:1 ratio of participants per item, which is appropriate but low. Furthermore, results of this study confirmed strong data. Per Costello & Osborne (2005), the stronger the data, as defined by high communalities without many cross loadings plus several variables loading strongly (0.50 or better) onto each factor, the smaller the sample can be for accurate analysis.

Declarations

Ethics approval and consent to participate: Prior to onset, study approval was granted by the health system’s institutional review board (#20-264E) and all patient informed consent was obtained in writing.

Consent for publication: Not applicable

Availability of data and materials: The datasets generated and analyzed during the current study are not publicly available due to proprietary issues by the healthcare system but are available from the corresponding author on reasonable request.

Competing interests: The authors declare that they have no competing interests.

Funding: This study was funded by the Robert Wood Johnson Foundation.

Authors’ contributions: AR analyzed and interpreted the factor analysis data. AR was a major contributor in manuscript writing. VF and MS were minor contributors in manuscript writing. VF and KE were responsible for the conception, design and securing of funding of this project. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to thank those who supported this study, specifically Andy Marek and Chris Blumberg in Research Analytics.

References