Abstract
BACKGROUND
Veterans with posttraumatic stress disorder (PTSD) have low employment rates and the job interview presents a critical barrier for them to obtain competitive employment.
OBJECTIVE
To evaluate the acceptability and efficacy of virtual reality job interview training (VR-JIT) among veterans with PTSD via a small randomized controlled trial (n=23 VR-JIT trainees, n=10 waitlist treatment-as-usual (TAU) controls).
METHODS
VR-JIT trainees completed up to 10 hours of simulated job interviews and reviewed information and tips about job interviewing, while wait-list TAU controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews scored by blinded human resource experts and self-reported interviewing self-confidence.
RESULTS
Trainees attended 95% of lab-based VR-JIT sessions and found the intervention easy-to-use, helpful, and prepared them for future interviews. VR-JIT trainees demonstrated significantly greater improvement on role-play interviews compared with wait-list TAU controls (p=0.04) and demonstrated a large effect for within-subject change (Cohen’s d=0.76). VR-JIT performance scores increased significantly over time (R-Squared=0.76). Although VR-JIT trainees showed a moderate effect for within-subject change on self-confidence (Cohen’s d=0.58), the observed difference between conditions did not reach significance (p=0.09).
CONCLUSIONS
Results provide preliminary support that VR-JIT is acceptable to trainees and may be efficacious for improving job interview skills and self-confidence in veterans with PTSD.
Keywords: veterans, virtual reality training, job interview skills, vocational training
Go to:
1. Introduction
The symptoms of posttraumatic stress disorder (PTSD) experienced by veterans of the United States Military contribute to a disabling condition that limits their reintegration into competitive employment (Magruder et al., 2004; Smith, Schnurr, & Rosenheck, 2005). Although there is no epidemiological research on employment rates among veterans with psychiatric disorders, approximately 37% of veterans enrolled in Veterans Health Administration (VHA) vocational services find competitive employment (NEPEC, 2014). Moreover, available evidence suggests that veterans with PTSD were 19 percent less likely than veterans without PTSD to be employed after completing VHA vocational training programs (Resnick & Rosenheck, 2008).
A critical step to obtaining competitive employment is the job interview, which often becomes a barrier to employment for individuals with PTSD. Specifically, veterans with PTSD need to overcome common fears that include being asked about their military service, being questioned about gaps in job history, and needing to schedule work hours around other demands including treatment appointments. The ability to feature their military service as an employable strength would help veterans handle difficult questions that arise during a job interview about their military service or their current disability. Thus, the job interview warrants further consideration as a critical target for vocational rehabilitation services (Bell & Weinstein, 2011).
In successful job interviews, candidates discuss their experience and core knowledge (i.e., job relevant interview content) related to the position, their ability to learn new skills, and demonstrate their ability to engage socially (i.e., interviewee performance) (Huffcutt, 2011). In addition, studies suggest that greater self-confidence in interviewing contributes to more effective communication during an interview (Tay, Ang, & Van Dyne, 2006) and that low self-confidence may be a barrier to employment among individuals with psychiatric disabilities (Corbiere, Mercier, & Lesage, 2004; Provencher, Gregg, Mead, & Mueser, 2002). Thus, self-confidence is an important attribute to improve when preparing for job interviews.
Role-plays for job interviews are a common training technique used by counselors in vocational rehabilitation programs, however this method may have limited generalizability from the clinic to real world outcomes (Dilk & Bond, 1996). On the other hand, virtual reality (VR) training has been found efficacious at improving interactive behavior and social skills that may transfer to real-life conversations. For example, VR simulated role-plays have been developed to train FBI agents to interrogate suspects (Olsen, Sellers, & Phillips, 1999), family physicians to perform brief psychosocial interventions (Fleming et al., 2009), and individuals with psychiatric disabilities to use more effective social skills (Park et al., 2011; Rus-Calafell, Gutierrez-Maldonado, & Ribas-Sabate, 2014). Moreover, training with VR simulations has advantages over other learning methods (Cook et al., 2011; Issenberg, McGaghie, Petrusa, Lee Gordon, & Scalese, 2005) because it provides 1) repetitive practice with simulated conversations, 2) exercises that allow trainees to practice new skills, 3) customized training experience with each simulated interaction, 4) consistent feedback in-the-moment, 5) an interpersonally safe environment, 6) accurate representation of real-life conversations, 7) application of different skills and strategies as the level of difficulty increases (e.g., hierarchical learning), and 8) access to web-based didactic material to enhance learning (Issenberg et al., 2005). Hence, VR simulation role-play training is fundamentally different from the usual vocational specialist approach.
In the present study, we evaluated the acceptability and efficacy of a highly interactive VR job interview simulation program, that has customizable features for veterans, in a randomized controlled trial of veterans with PTSD who were enrolled in VHA services. The intervention, “Virtual Reality Job Interview Training” (VR-JIT), targets improvement of job relevant interview content and interviewee performance (Huffcutt, 2011). Based on our prior studies of VR-JIT efficacy among individuals with psychiatric disabilities and adults with autism spectrum disorders (Smith et al., 2014a, 2014b), we hypothesized that the VR-JIT sessions would be well-attended and that trainees would rate the intervention as easy-to-use, enjoyable, and helpful. We also hypothesized that completion of VR-JIT training would be related to improvements in job interview role-play performance and enhanced self-confidence in job interview skills in the VR-JIT group as compared to a wait-list treatment-as-usual (TAU) control group. We explored whether VR-JIT process measures were associated with outcomes as well as whether job interview role-play performance and self-confidence in one’s interview skills during follow-up assessments were associated with each other and with demographic characteristics, vocational history, and cognitive functioning.
Go to:
2. Methods
2.1. Participants
Participants included 33 veterans with PTSD who were recruited through advertisements and by vocational counselors affiliated with the Edward Hines Jr. Veterans Administration Hospital and the Jesse Brown Veterans Administration Medical Center in Chicago, IL. Participants were required to be U.S. Military veterans with a diagnosis of PTSD. B.S. or Ph.D.-level research staff interviewed participants using the Mini-International Neuropsychiatric Interview (MINI) to determine DSM-IV Axis I diagnoses (Sheehan et al., 1998). Participants were eligible for inclusion if they were: 1) 18–65 years old, 2) achieved at least a 6th grade reading level using the sentence comprehension subtest of the Wide Range Achievement Test-IV (WRAT-IV) (Wilkinson & Robertson, 2006), 3) agreed to be digitally recorded, 4) were unemployed or underemployed, and 5) were actively seeking employment.
Participants were excluded if they had: 1) a medical illness that significantly impaired cognition (e.g., traumatic brain injury), 2) a current diagnosis of substance abuse or dependence, or 3) an uncorrected vision or hearing problem. The Northwestern University Feinberg School of Medicine Institutional Review Board approved the study protocol, and all participants provided written informed consent. Participants were randomized after enrollment into the intervention (n=23) or the wait-list TAU control group (n=10) at an estimated ratio of 2 to 1. This ratio was chosen to allow more information about acceptability and within-subject improvement from VR-JIT.
2.2. Intervention
“Virtual Reality Job Interview Training” (VR-JIT) is a computerized training simulation designed by SIMmersion LLC (http://www.simmersion.com) to improve job interview skills. VR-JIT uses PeopleSim™ technology, learning objectives determined by a panel of vocational experts, and video recordings of an actress to create an interactive virtual character named Molly Porter. Molly is a human resources representative for Wondersmart, a large, fictional department store that virtually interviews the trainees. Additional details about the learning objectives (Bell & Weinstein, 2011), information about the development of VR-JIT (Smith et al., 2014a, 2014b), and figures of Molly and the VR-JIT interface (http://www.jobinterviewtraining.net) are available.
VR-JIT was specifically designed with customizable options for individuals with a range of backgrounds (e.g., education, employment, history, job-related skills), disabilities (e.g., psychiatric, physical), and circumstances (e.g., no prior work history, employment gaps, past substance abuse, prior criminal history). Special consideration was given for individuals who identified themselves as veterans, including military veterans with PTSD who may be prone to social anxiety (Crowson, Frueh, Beidel, & Turner, 1998). Specifically, the training allows veterans to practice and gain confidence in answering questions about their military service, featuring their service experience as an employment strength, explaining gaps in job history, handling anxiety-producing or difficulty questions about their service or disability, and negotiating for accommodations or scheduling needs.
Throughout each virtual interview, Molly selects from a database of 1000 video recorded questions to create a dynamic, variable, and individualized training experience. Molly’s questions are selected based on the answers trainees provide to the VR-JIT online job application and range from general inquiries (e.g., “Tell me about yourself?”) to specifics about military history (e.g., What skills did your period in the armed forces give you?”). This variation provides trainees with opportunities to 1) repeatedly practice job interviews until they are prepared for a real interview, 2) verbally respond to Molly’s questions using speech recognition, 3) try different approaches to answering questions, 4) develop personal answers about a specific job based on personal work history and skills, 5) gain insights from an on-screen coach who provides in-the-moment feedback, 6) practice recovering (e.g., apologizing or clarifying) from responses that hurt interviewer rapport, 7) experience more difficult interview experiences as their skill increases (e.g., at a moderate level, Molly may ask general follow-up questions like, “Can you tell me more?”, and at the advanced level, she may ask illegal questions or specific follow-up questions like, “Since you stopped working, how have you been spending your time?”), and 8) learn about other steps to finding a job (e.g., creating a resume, researching a position, types of questions to ask, selecting a job that meets their needs) by reviewing didactic electronic learning (e-learning) materials.
The on-screen coach, color-coded transcripts, and detailed post-interview feedback mechanisms allow trainees to identify mistakes and learn why some answers are better than others. For example, if Molly asked how the trainee handles pressure and the veteran answered “Pressure doesn’t really bother me,” the coach may suggest that the veteran provide more details or examples. Then, the next time Molly asks about pressure, the veteran may provide a more detailed answer, such as, “My military training really emphasized staying calm and focused under pressure. That training would help during busy times in the store.” VR-JIT allows veterans to gain exposure to an anxiety-provoking situation in a safe environment where they can exercise maximum control. Meaning, VR-JIT serves as a form of exposure therapy while providing opportunities for veterans to learn new skills and gain realistic practice with feedback.
2.3. Study procedures
Baseline measures for both groups included 1) demographic, clinical, and vocational interviews; 2) neurocognitive and social cognitive assessments; and 3) a pre-test self-report of self-confidence. After completing baseline, participants were randomized to the VR-JIT or TAU groups and then attended a pre-test visit to complete two job interview role-plays. Following the completion of the pre-test visit, the TAU group attended their usual outpatient services for two weeks, which may have included vocational training (noted in Table 1). The VR-JIT group completed up to 10 hours of VR-JIT simulations (approximately 20 trials) across 5 visits (within a two-week period), while participating in their usual outpatient services (which may have included vocational training, see Table 1). Both groups returned after two weeks to complete the post-test self-confidence measure (identical to the pre-test measure), the Treatment Experience Questionnaire (TEQ; VR-JIT group only), and two post-test job interview role-plays (in that order).
Table 1
Characteristics of the Study Sample
| TAU Group (n=10) |
VR-JIT Group (n=23) |
χ2 / T-Statistic |
|
| Demographics | |||
| Mean age (SD) | 50.8 (10.4) | 51.2 (11.5) | <-0.1 |
| Gender (% male) | 100% | 95.7% | 0.5 |
| Parental education, mean years (SD) | 12.2 (3.9) | 11.9 (2.4) | 0.2 |
| Race | |||
| % Caucasian | 20% | 8.7% | |
| % African-American | 80.0% | 87.0% | 1.2 |
| % other | 0.0% | 4.3% | |
| Clinical History | |||
| VA-Psychiatric or other support services | 30.0% | 34.8% | <0.1 |
| Post-Traumatic Stress Disorder | 100% | 100% | <0.1 |
| Major Depressive Disorder | 60.0% | 52.2% | 0.2 |
| Bipolar Disorder | 10.0% | 34.8% | 2.2 |
| Schizoaffective Disorder | 0.0% | 4.3% | 0.5 |
| Substance Use Disorder | 0.0% | 4.3% | 0.5 |
| Vocational History | |||
| Prior participation in vocational training program | |||
| Compensated Work Therapy | 20.0% | 26.1% | 0.1 |
| Unknown Program or seeking vocational services | 80.0% | 73.9% | |
| Months since any prior employment, mean (SD) | 38.7 (62.5) | 41.3 (55.9) | −0.1 |
| Financial Support | |||
| Current SSI or SSDI support | 0.0% | 21.7% | 2.6 |
| Receives non-military pension | 10.0% | 17.4% | 0.3 |
| Other public support | 80.0% | 52.2% | 2.3 |
| Cognitive Function | |||
| Prior participation in cognitive rehabilitation | 10.0% | 30.4% | 1.6 |
| Global Neurocognition, mean (SD) | 89.9 (4.4) | 87.9 (11.6) | 0.7 |
| Basic Social Cognition, mean (SD) | 0.63 (0.14) | 0.66 (0.12) | −0.6 |
| Advanced Social Cognition, mean (SD) | 0.79 (0.10) | 0.77 (0.11) | 0.5 |
Open in a separate window
Note: TAU, treatment-as-usual participants; VR-JIT, virtual reality-job interview training trainees.
Staff encouraged the VR-JIT trainees to review e-learning materials prior to each simulation, but referencing the e-learning component was optional. To promote hierarchical learning, trainees progressed through three difficulty levels. They were required to complete at least three ‘easy’ interviews. One score of 80 or higher was required on ‘easy’ to advance to the ‘medium’ level. Trainees automatically advanced to medium if they did not score at least 80 prior to completing 5 virtual interviews. This process was repeated for trainees at the ‘medium’ level before advancing to the ‘hard’ level. Remaining trials were completed on the ‘hard’ level. Staff reviewed the transcript with trainees for approximately 15 minutes after each completed virtual interview.
2.4. VR-JIT training fidelity
Research staff were trained to administer VR-JIT using a checklist that covered: navigating the graphic user interface, creating a user profile, completing a job application, accessing e-learning materials, starting the virtual interview, reading transcripts, using in-the-moment feedback and help modules, reviewing transcripts, and reviewing summarized interview performance. Staff were required to administer VR-JIT to mock participants using the checklist during practice sessions. Subsequently, staff used the checklist to conduct one-on-one training sessions with participants (30–45 minutes) to ensure they were able to independently and successfully navigate the intervention.
2.5. Measures
2.5.1. Demographic characteristics and vocational history
Demographic characteristics (e.g., age, gender, race) and vocational history (e.g., months since prior employment, prior vocational training) were obtained via a self-report interview.
2.5.2. Neuocognitive and social cognitive measures
The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) (Randolph, Tierney, Mohr, & Chase, 1998) was administered to assess participants’ neurocognitive functioning. The total score of the RBANS reflects the following neurocognitive functions: immediate memory (i.e., list learning, story memory), visuospatial capacity (i.e., figure copy, line orientation), language (i.e., picture naming, semantic fluency), attention (i.e., digit span, coding), and delayed memory (list learning free recall, list learning recognition, story memory free recall, figure free recall).
The Bell-Lysaker Emotion Recognition Task (BLERT) (Bell, Bryson, & Lysaker, 1997) was used to measure basic social cognition. Participants viewed 21 video-recorded vignettes of an affective monologue and identified the emotion prominently displayed. An accuracy rating was computed based on the number of correct responses. The emotional perspective-taking (EPT) task measured advanced social cognition. Participants observed 58 scenes of 2 actors engaged in social interactions where the face of one actor was covered with a mask. Participants were instructed to select which of two facial expressions best reflected the feelings of the masked character (Smith et al., 2014c). An accuracy rating was computed based on the number of correct responses.
2.5.3. Acceptability assessments
VR-JIT trainees were invited to attend five training sessions during which they could spend up to two hours receiving VR-JIT. Attendance was recorded as the rate of session attendance and the number of minutes spent completing VR-JIT trials were collected (out of a possible 600 minutes of training). The Training Experience Questionnaire (TEQ) (Bell & Weinstein, 2011) required trainees to rate their experience with VR-JIT on 5 items (ease-of-use, enjoyment, helpfulness, instilling confidence in interviewing, and preparing them for interviews). Items were rated on a 7-point Likert scale with higher scores reflecting more positive views of VR-JIT.
2.5.4. Primary efficacy assessments
2.5.4.1 Job interview role-plays
All participants completed two pre-test role-plays and two post-test role-plays (approximately 20 minutes each). Prior to the role-play, participants selected two of eight job scenarios (e.g. data entry specialist, mail clerk, inventory manager, sales associate) and completed a paper-based job application to guide the role-play. Participants were given directions stating, “You are interviewing for part-time work, particularly because you need to have Thursdays off for personal reasons. You will need to negotiate for a schedule that will accommodate for Thursdays off.”
Standardized actors (SAs) posing as human resources representatives led the role-plays and were trained to ask 13 standardized questions and 3–4 random questions from a list of 70+ questions, in a naturalistic way. The job scenarios were developed by the research team and vetted through a panel of vocational rehabilitation experts. All role-plays were video-recorded for scoring purposes.
Role-plays were scored on nine communication skills: 1) sounding like a hard worker (dependable), 2) sounding easy to work with (teamwork), 3) behaving professionally, 4) negotiation skills (asking for Thursdays off), 5) sharing things in a positive way, 6) sounding honest, 7) sounding interested in the position, 8) trainee comfort level, and 9) establishing overall rapport with the interviewer. Role-play videos were randomly assigned to two raters with expertise in human resources and blinded to treatment group status. Both raters completed 10 practice videos to establish reliability before independently rating the study videos. The raters established reliability by double scoring approximately 20% of the videos and attained a high degree of reliability (ICC=0.84). To prevent rater drift, both raters met with the research team every 20 videos to review two videos and discuss inconsistencies and reach a consensus score. A total score was computed from nine domains (range of 1–5 per domain, with higher scores reflecting better performance) for each of the two baseline role-plays. The two total scores were averaged to generate a single score representing baseline role-play performance. A single post-test role-play score was computed using the same method.
2.5.4.2. Job interview self-confidence
Participants rated their self-confidence at performing job interviews using a 7-point Likert scale to answer nine questions, with higher scores reflecting more positive views (e.g., “How comfortable are you going on a job interview?”). Total scores at pre-test and post-test were computed, and each had strong internal consistency (α=0.91 and α=0.85, respectively).
2.5.5. Process measures
Participants’ VR-JIT performance scores and the number of completed virtual interview trials were recorded in the lab. Each virtual interview trial generated a total score from 0–100 using an algorithm programmed into the software based on the appropriateness of their responses that achieve the 8 learning objectives that guide VR-JIT.
2.6. Data analysis
Between-group differences for demographics, vocational history, global neurocognition, and social cognition were assessed with an analysis of variance (ANOVA) and chi-square analyses. Acceptability was determined with descriptive statistics of session attendance, training time, and TEQ scores. A time-by-group interaction from a repeated measures analysis of variance (RM-ANOVA) determined whether the primary outcome measures (i.e., role-play performance and job interview self-confidence) for the VR-JIT group significantly improved between pre-test and post-test as compared to the TAU group. Cohen’s d effect sizes were generated to characterize the within-subject differences in scores between pretest and post-test as well as between-group differences in post-test scores. Previous reports on the efficacy of VR-JIT in other disability groups allowed us to have a directional hypothesis for this small study (Smith et al., 2014a, 2014b), and therefore our between-groups analyses are one-tailed with alpha set at 0.05.
We evaluated VR-JIT performance score improvement across trials as a process measure by computing linear regression slopes for each trainee based on the regression of their performance scores on the log of trial number. The group-level performance average for each successive VR-JIT trial was plotted with a report of the R-squared from the regression of average performance on the log of trial number. Also, we generated the mean number of completed virtual interviews.
We computed partial correlations in an effort to explore whether post-test role-play and self-confidence scores and VR-JIT performance slopes were associated with each other as well as with age, gender, months since prior employment, global neurocognition, and basic and advanced social cognition (when co-varying for pre-test outcome scores).
Data were normally distributed and no transformations were necessary. Although participants were instructed to negotiate for Thursdays off during each role-play, they forgot during 29.5% of the role-plays despite prompting from the SA. The mean value of the other scores for this item was imputed for the missing variable (Myers, 2000). No other role-play ratings were missing. Data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at Northwestern University Feinberg School of Medicine (Harris et al., 2009).
Go to:
3. Results
3.1. Between-group characteristics
The VR-JIT and TAU groups did not differ with respect to age, gender, race, parental educational attainment, and neurocognitive and social cognitive functioning, clinical history, the number of months since prior employment, prior participation in cognitive or vocational rehabilitation programs, and financial support (all p>0.10) (Table 1).
3.2. VR-JIT acceptability
The VR-JIT sessions were well-attended, and participants reported that VR-JIT was easy-to-use, enjoyable, helpful, increased their self-confidence in job-interview skills, and improved their readiness for interviewing (Table 2).
Table 2
Mean Characteristics of VR-JIT Acceptability (SD)
| Attendance Measures | |
| % Session Attendance | 94.8% (10.8) |
| Elapsed Simulation Time (min) | 549.1 (74.8) |
| TEQ Items | |
| Ease of use | 5.8 (1.0) |
| Enjoyable | 6.5 (0.7) |
| Helpful | 6.8 (0.4) |
| Instilled confidence | 6.8 (0.4) |
| Prepared for interviews | 6.5 (0.6) |
Note. Scale for TEQ, 1=Extremely Unhelpful to 7=Extremely Helpful
3.3. VR-JIT efficacy
Primary outcome RM-ANOVA analyses are presented in Table 3. The RM-ANOVA revealed that the group-by-time interaction was significant (F1,31=3.4, p=0.04). There was no group effect at baseline (p>0.10). This significant interaction was characterized by a large effect size improvement between pre-test and post-test (d=0.76) for the VR-JIT group, and a small difference for the TAU group (d=0.04). The RM-ANOVA assessing self-confidence revealed that the group-by-time interaction did not reach significance (F1,31=2.0, p=0.09). There was no group effect at baseline (p>0.10). This trend-level finding was characterized by a medium effect size improvement in the VR-JIT group (d=0.58), and a small difference for the TAU group (d=0.20).
Table 3
Primary Outcome Measures
| TAU Group | VR-JIT Group | |||||
| Pretest mean (SD) |
Posttest mean (SD) |
Cohen’s d | Pretest mean (SD) |
Posttest mean (SD) |
Cohen’s d | |
| Role-Play Performance Total Score | 34.8 (4.0) | 34.9 (4.1) | 0.04 | 34.0 (2.7) | 35.8 (2.7) | 0.76 |
| Job interview self-confidence rating | 49.7 (5.0) | 51.0 (6.3) | 0.20 | 44.9 (9.1) | 51.2 (6.1) | 0.58 |
Note: TAU, treatment-as-usual participants; VR-JIT, virtual reality-job interview training trainees.
3.4 Process measures
RM-ANOVA revealed trainees completed a mean of 15.0 trials (sd=3.1). VR-JIT performance scores improved across the number of completed trials. Specifically, the slope (mean=2.9, sd=2.6) suggests that performance improves 2.9 points for every 1.0 point increase in the natural log of the trial number (Figure 1; R-Squared= 0.76).
Fig. 1
VR-JIT Learning Curve in U.S. Military Veterans with Posttraumatic Stress Disorder. This figure plots the average score for each successive VR-JIT simulated interview trial. Trials 1-3 at easy, trials 4-6 at medium, and trials 7-20 at hard. Initial trials at each difficulty level (trials 1, 4, and 7) are lower than subsequent trials at that level. Overall model fit, R2=.76.
3.5 Exploratory correlations
Exploratory analyses revealed positive correlations between baseline neurocognition (r=0.49, p=0.02) and advanced social cognition (r=0.42, p=0.049) with post-test self-confidence when co-varying for pre-test self-confidence. The number of completed VR-JIT trials was correlated with post-test role-play performance while co-varying for pre-test role-play performance (r=0.39, p=0.08), although not reaching statistical significance. Remaining pairwise correlations with post-test outcome scores were not significant (all p>0.10, see Table 4).
Table 4
Exploratory Pearson Correlations
| Post-Test Role-Play Score |
Post-Test Self- Confidence |
|
| Age | 0.08 | −0.20 |
| Gender | 0.09 | 0.19 |
| Months since prior employment | 0.22 | −0.22 |
| Global Neurocognition | −0.11 | 0.49* |
| Basic Social Cognition | 0.32 | 0.10 |
| Advanced Social Cognition | <0.01 | 0.42* |
| VR-JIT Performance Slope | 0.01 | −0.18 |
| Completed VR-JIT Trials | 0.39+ | 0.16 |
| Post-Test Role Play Score | — | −0.15 |
| Post-Test Self-Confidence | −0.15 | — |
+p<.10,
*p<.05.
Go to:
4. Discussion
The findings support our hypotheses concerning the acceptability of VR-JIT among veterans with PTSD. Specifically, veterans completed >95% of training sessions and >550 minutes of training (out of a maximum of 600 minutes), and they reported high scores indicating that VR-JIT was easy-to-use, enjoyable, helpful, instilled them with confidence, and made them feel well-prepared for future interviews. Results are consistent with our evaluation of VR-JIT among individuals with severe mental illness (SMI) or autism spectrum disorder (ASD) (Smith et al., 2014a, 2014b).
VR-JIT trainees had a large effect size improvement in pre-to-post role-play job interview performance, which was significantly greater compared to individuals in the wait-list TAU group. Effect sizes of the within-group changes in the VR-JIT and wait-list TAU groups were consistent with the effects observed among individuals with SMI or ASD who used VR-JIT training (Smith et al., 2014a, 2014b) and that of other studies that used animated avatars to improve social skills for individuals with psychiatric disabilities (Park et al., 2011; Rus-Calafell et al., 2014; Tsang & Man, 2013; Zawadzki et al., 2013).
During the VR-JIT simulations, trainees learned how to improve their interview skills as they transitioned between difficulty levels and across the total number of trials showing a highly significant relationship between scores and log of trial number. This finding suggests that VR-JIT trainees improved their interviewing skills between the pre-test and post-test measures. Our exploratory analysis of the data suggested that when accounting for pre-test role-play performance, the VR-JIT group scored higher on their post-test role-plays as the number of completed simulated interviews increased. Although this relationship was at the trend-level, it provides some support that the amount of exposure to VR-JIT is related to improved performance in live role-play interviews.
The VR-JIT trainees reported greater improvement in their self-confidence than the wait-list TAU participants. The observed difference was characterized by a medium effect size, but did not reach statistical significance. This pattern of observed differences is similar to our work in prior studies of individuals SMI or ASD (Smith et al., 2014a, 2014b). The limited significance observed might be explained by limited statistical power from a small sample size. Although prior studies suggest that higher self-confidence may be related to better interview performance (Corbiere et al., 2004; Tay et al., 2006), we did not observe this relationship in our analysis of the data.
There are several limitations to this study. First, we evaluated VR-JIT in a small group of veterans with limited statistical power. Second, the sample was older so the result may not generalize to younger veterans who receive VR-JIT. Although the intervention was designed to be implemented in a practical setting (e.g., home, service provider), the study was conducted in a laboratory setting. Future studies are needed to evaluate its effectiveness under the usual conditions in which vocational rehabilitation services are offered to veterans and with a wider age-range of veterans. Third, psychiatric symptoms, pharmacological treatment, motivation, and length of time seeking employment were not considered as moderating variables. Finally, it was beyond the scope of this small study to examine effects on employment outcomes. Future research is needed to determine whether VR-JIT leads to more job interviews, more job offers and more rapid employment for veterans with PTSD who are not enrolled in vocational services or whether VR-JIT enhances the existing vocational services received by veterans with PTSD.