1. Introduction

T Research methods courses are part of the regular curriculum in undergraduate psychology programs around the world. Although the specifics of the class offerings and the methods of assessing student performance in these courses vary, it has become increasingly important to provide evidence of increased student research skills outcomes upon completion of these types of courses to meet university assessment and accreditation requirements ^[1]. We offer the current investigation as an exercise in assessment other professors may choose to use in their own research methods classes, or as part of departmental assessment activities.

Undergraduate research methods courses, and working with faculty members on research for these courses, are a very powerful instructional tool and an effective way to increase research skills. However, research methods courses are typically labor intensive to teach and also require a high amount of faculty’s time and commitment as well as a sizeable amount of funding for materials and other departmental resources. Therefore, it is important to determine the actual value of research methods courses ^[2]. Specifically, which research skills are improved through undergraduate research experiences and why are these particular skills important for departmental goals and learning objectives?

Studies have demonstrated the usefulness and success of such research courses and how to improve and expand students’ research skills. Kardash studied undergraduates who completed a summer or semester long internship under the supervision of a faculty mentor ^[2]. Students worked directly with faculty advisors on research projects. At the end of the internship, the students presented their research at conferences and made posters for presentation at their respective campuses. As part of the investigation, students completed a questionnaire designed to assess their research skills at the beginning of the internship as well as at the end of the internship. Research skills improved from the beginning to end of the internship based on the students’ perceptions of their research skills. This study shows support and evidence of the usefulness and effectiveness of participating in a research internship experience ^[2].

The purpose of the current investigation was to examine how student perceptions of their own research skills would change upon completion of a senior level research methods course and to share our assessment methodology and the selected instrument we used with interested instructors. Previous researchers have not used the current research skills measure to investigate changes in student research skills in a structured classroom. We predicted that students would indicate improvements in their research skills from the beginning to the end of the course. We also predicted positive relationships between an increase in research skills and self-reported enjoyment of the course, effort in the course, and knowledge gained in the course. Furthermore, students who obtained significant results in their research projects and students who anticipated earning As in the course were predicted to report more improvement in research skills due to their positive experiences with research.

2. Method

Seventy-one students from a medium sized, public university in the southeastern United States enrolled in six sections of a required senior research course participated in the current investigation. The racial distribution of the sample included 81.7% Caucasians, 15.5% African-Americans, 1.4% Hispanics, and 1.4% Asian Americans. The average age of the participants was 22.68 years (SD = 2.86), and all of the participants were psychology majors. Most of the participants enrolled in the course were women (87.3%). The majority of the participants were seniors (90.1%), and the remaining 9.9% were juniors. Approximately half of the participants planned to graduate the semester during which they were enrolled in the course (49.3%). The majority of students planned to attend graduate school (84.5%), especially in clinical/counseling areas.

Applied Research in Psychology (PSYC 497) is a psychology major requirement which students generally complete during their senior year of college. The course is a “research experience in which students are required to develop a research project, conduct a literature review, gather and analyze data, prepare a research paper in accord with the standards of the American Psychological Association (APA) and present their research.” The instructor supervises all projects and leads group instruction on topics related to research.

Students participating in the current investigation completed a research skills assessment instrument ^[2] at the beginning and end of the 15 week semester course in a repeated measures design. The instrument comprised 14 Likert-style questions related to specific research skills typically developed during a research experience. Example research skill areas included understanding concepts in the field, using primary scientific research literature, formulating hypotheses, collecting data, statistically analyzing results, and communicating results. Participants rated their skills on each scale item using a 5-point Likert scale (1 = not at all, 5 = a great deal).

Participants also reported their anticipated course grade, whether they obtained statistically significant results in their research projects, and whether they planned to follow-up on their research study by conducting more research. They also rated their course effort, enjoyment of the course, and whether the course increased their research knowledge on 10-point Likert scales (1=minimum amount, 10=maximum amount).

Participant age, sex, class rank, and major were collected on a demographic questionnaire. Students were also surveyed about their plans to attend graduate school and the type of graduate program and degree sought.

3. Results

Students taught by the different professors scored similarly in their assessments of course outcomes and changes in research skills, therefore sections were combined for further data analysis. Students reported positive course outcomes regarding their course effort (M = 8.92, SD = 1.10), course enjoyment (M = 7.69, SD = 1.62), and research knowledge gained from completing the course (M = 9.24, SD = .98). As predicted, the composite score of research skill ratings increased from the beginning to the end of the class, t(70) = 9.11, p < .001, d = 1.56. Separately, each of the 14 research skill areas was rated higher at the end of the class than at the beginning, all ps < .01. Specific research skill area results are provided in Figure 1.

Figure 1. Mean pre and post research skills assessment scores by question

Download as

PowerPoint Slide

Larger image(png format)

Figures index

Veiw figure

View current figure in a new window

We were also interested in how student’s self-perceived effort, enjoyment, and knowledge gained was related to changes in research skill ratings. Difference scores were calculated to examine student research skill rating changes from the beginning of the course to the end. Overall, there was a positive relationship between increased research skills and course effort, r(69) = .29, p = .04, enjoyment, r(69) = .24, p=.04, and knowledge gained, r(69) = .25, p = .04.

Sixty-two percent of the participants’ research projects did not yield statistically significant results. Fifty-two percent of the participants indicated their intention to follow up on their research projects. Forty-four percent anticipated a course letter grade of A, 52% B, and 4% C. Interestingly, students who obtained significant results for their research and students who anticipated earning As in the course reported greater increases in research skill ratings scores than those who did not, but these differences were not statistically significant.

4. Discussion

As predicted, research skill ratings increased, from the beginning to the end of the class, for all 14 areas assessed. Students felt more confident in their research skill abilities after participating in a course which required them to conduct their own original piece of research. Our results show that all students reported increased research skills, not just those students who earned high grades in the class or students who obtained significant research results. Similar studies have been conducted to examine students’ perceived research skills as well as their faculty mentor’s views of the students’ research skills ^[2]. Results revealed that mentors’ and students’ scores were very similar for the majority of specific research skills measured on this particular scale. While we did not obtain ratings for our own student researchers in the current investigation, this could be an area for further study. The 14 research skill areas provided in the current measure ^[2] could serve as areas for faculty raters to assess.

Limitations of this research include a small sample size of predominately women and the research skill assessment instrument was a self-report exercise. As previously mentioned, student assessments of their own abilities is useful, but additional verification of skill acquisition from professors would confirm the increase in students research skills ^[2]. Another interesting direction would be to have students assess other students based on attending their research presentations and reading their research papers for peer assessment.

Most undergraduate psychology programs require research methods courses and our selected research skills instrument ^[2] may be useful for assessment purposes. Psychology programs could measure changes in research skills at multiple times as students complete coursework to follow growth and learning in students and to address program assessment. Specific skill areas may also be identified and emphasized as part of continuous quality improvement measures and curriculum adjustments.

Our results suggest students enrolled in our senior research course believed their research skills improved after completing the course. Future investigations may also include professor assessments of student research skills, in addition to student self-assessments, as part of the course assessment. As more departments focus on student learning and assessment, the current measure of research skills ^[2] may be a convenient yet informative measure to consider.

Author Note

Portions of this research were presented at the 24th Annual Association for Psychological Science Convention in Chicago, Illinois, USA.

References

[1]	APA. The assessment cyberguide for learning goals and outcomes (2nd edition), American Psychological Association, Board of Educational Affairs, Washington, DC, 2009.
	In article
[2]	Kardash, CM, “Evaluation of an undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors,” Journal of Educational Psychology, 92(1). 191-201. Mar. 2000.
	In article		CrossRef