Training Advisors to Conduct Research

Hurt, R. L., & State, C. (2012). An Applied Introduction to Qualitative Research Methods in Academic Advising. NACADA Journal, 32(2010).

This article presents approaches for conducting academic advising research. Presents qualitative and quantitative approaches, however, emphasizes the use of qualitative approaches. Advising as a form of teaching can be evaluated. The writers suggest reluctance on advisors’ part to conduct research due a lack of training in statistical analysis. The writers present frameworks to provide validity to qualitative research.

 Qualitative Research

             The article includes a description of qualitative research. It begins by outlining four characteristics of valid qualitative research: targeted to address a specific question, have qualities which can be measures, seeking to understand the factors influencing behavior, ensure the researcher is genuinely interested in the people he/she is researching so that a deep connection can be made.  Four areas to be mindful of when conducting qualitative research: may last a long span of time, be aware to ensure the story of the data is told and bias is reduced in interpretation, review possible journals that might publish a qualitative (as opposed to quantitative) study, and to possibly incorporate quantitative research (where appropriate).

Three Approaches to Quantitative Research

The writers presented three types of quantitative research approaches and provided examples of how each type can be used. Ethnography is “rooted in cultural anthropology and sociology” (pg. 65). In academic advising, we are trying to describe the student experience. That is an example of an ethnographic study. In my research, I’m interested in describing the learning students may do as a result of advising interactions. This would be supported through this type of research framework. Appreciative Inquiry, is the second type presented. It is described as a form of small group discussion, which leads to the production of the most effective form of “x”. Discussion is guided and attempts to cover four phases: Discovery, Dream, Design, and Destiny. These are described further as identifying the current status of “x”, considering what could be, designing what could be, and finally, creating what could be for “x”. The third type presented is Case Studies. In this type, varying groups are selected by the researcher to discuss a topic. Perhaps this would be different years of students within a major (i.e. Freshman, Sophomores, Juniors, or Seniors) and are guided with open-ended questions to discuss a specific topic. The responses are collected and common themes or further research topics emerge.

 Article Critique

            This article presented very little literature review. It’s based upon a supposition that academic advisors are reluctant to do quantitative research due to lack of understanding of statistics. That idea lacks any reference. The authors suggest identifying journals that will publish qualitative research, but did not recommend any. That would have been helpful, too. In addition, each of the three types of qualitative research presented did not include significant literature support for the explanation or definition.   It could be considered an introductory piece for an advisor beginning to consider research questions. However, further inquiry would need to be done before crafting a research methodology.

Application to My Research

            Well-developed research could build more validity into the field of advising. As I have been doing my literature review, I would agree that much of what I am finding is qualitative research. However, I have seen a transition within the day-to-day execution of advising, as we have been integration technology based tools with our practice. We track and record interactions, use data sets to identify which students to reach out, and have fully online tools available. This is developing data sets that will be available (with the appropriate approvals) for quantitative research. As this is evolving, I am hopeful I can access this information for my research.

I’ve administered surveys and led focus groups in the past, with varying degrees of success. The results from the surveys were presented with very basic statistical analysis along with the focus group comments. The focus groups comments helped to put the data from the surveys into context. In my current role, I’ve been working towards the implementation of quantitative survey assessments. The first two I am considering employing include post appointment surveys and pre and post surveys to test knowledge of a specific academic tool. I hope to publish or present findings in the future. I’m hoping that by collecting data from these surveys, I can open a dialogue with students and advisors to find out more about the factors, which might be contributing to what is being measured in the surveys. This article was helpful in that it will present a starting point for how to conduct the qualitative methodologies, which I need to consider. However, I’ll need to conduct a more in-depth literature review before moving forward.

What drew me to the article was that it helped me understand the context in which my research will be presented.  If the authors are correct in their assumption about advisors’ capacities for statistical analysis and they are the primary audience for my research, I need to understand how to convey my findings in a way that is accepted and understood.

Assessing Advisor Practices with the Student Perspective

Teasley, M. L., & Buchanan, E. M. (2013). Capturing the student perspective: a new instrument for measuring advising satisfaction. NACADA Journal, 33(2), 4–15. doi:10.12930/NACADA-12-132

Summary: This article discusses the rationale for assessing student perceptions of advisor interaction in the context of the application of advising practice theory. It proposes a survey instrument, which institutions can use to assess the interactions. The article seeks to identify any connection between certain student development theories and the students’ perceptions of the experience (i.e. does a certain type of advising theory more significantly influence the experience). The purpose of this research is to more effectively quantify how advisor interactions influence student retention (Kuh, 2008). A survey instrument was developed and deployed three times. Each time it was administered, statistical analysis was conducted and modifications were made for the next iteration.

O’Banion (1971,1994, 2009) identified the 5 major dimensions of the advising experience. These are meant to outline the types of discussions advisors lead with students. This type of advising is called developmental advising. This theory suggests that advisors support the development of understanding and skills, which support a successful college experience. Another type of advising is prescriptive advising. That type of advising includes course scheduling, discussing graduation requirements, etc. It is meant to outline next steps for students. Each type of advising has a place however advisors are encouraged to leverage developmental advising practices as it is most beneficial to a student’s growth.

Teasley and Buchanan (2013) developed a survey “originally designed to measure satisfaction with prescriptive functions…developmental functions…and overall advisor traits” (pg. 6).  Three factors were initially considered in the survey: prescriptive, developmental, and advisor functions. Those three factors correlate to student developmental advising theories. Those three factors were considered in the first two versions of the survey. The third version of the survey eliminated the third factor related to advisor functions.  The article concluded with support for the validity of the survey and the encouragement of its adoption within institutions.

Organization: The article was organized well. It began with a review of key literature and effectively demonstrated the need for assessment of advising. The literature review included key pieces in student development theory and general assessment for the purposes of increasing student retention. It included a discussion of the limitations of the research. The research data and collection methodology were conveyed. It concluded with appropriate recommendations and possible future steps.

Methodology: Exploratory Factor Analysis was used with the first two iterations. As previously discussed, the designers were interested in learning about three factors, which influenced the students’ perceptions. The first and second surveys were administered to the undergraduate research pool. Based upon the analysis, modifications were made to the questions in survey 1 and survey 2.

The analysis suggested that only two factors were contributing to the students’ perceptions: general advising concerns and outreach functions. Students did not distinguish between the use of development or prescriptive advising.

As a result, Confirmatory Factor Analysis was employed with the third survey.  The participants involved in the third survey came from the university research pool.

(Since I am not as familiar with these two statistical analysis options, I researched them on Wikipedia. The links above were explanations of them and based upon that information, it appeared the data analysis techniques used were appropriate and the guidelines for reliability and validity were followed.)

Limitations:  Initial discussion of the source of the questions was not included. The origin of those questions would have been beneficial in terms of replication.

The undergraduate research pool was utilized. Students could have participated in both the first and second survey. It would have been helpful to understand how many students did participate again and if they made an impact on the findings.

The third iteration of the survey targeted only two factors, but very little information was shared or discussed regarding the second factor related to advisor outreach. It would have been helpful to understand or convey next steps with those findings. The third survey was intended to further examine the validity of the instrument.

Reflections: My research interests are directed towards the integration of student learning outcomes and advisor performance.  White and Shulenberg (2012) highlight and define the value of student learning outcomes:

“The challenge of coming to grips with the questions about learning outcomes is twofold: (1) each institution needs to accept advising as an educational endeavor and identify the relevant learning outcomes and (2) reliable and valid methods to determine if these outcomes that have been met need to be developed”(pg. 14)

I need to be mindful that assessment of advising is not the same as assessment of student learning outcomes. There is a lot of literature in the field about assessing advising. However, student learning outcomes aim to measure the student’s learning as a result of advising interactions. Advisor performance, knowledge, and training are certainly components of that, however, student learning outcomes look at specific interactions to induce learning.

The article references other sources for possible assessment tools. I certainly want to research and learn more about those. Those other tools were described as instruments created within the institutions themselves and perhaps were not statistically tested for validity and reliability. This annotated bibliography on the NACADA website includes valuable resources for learning more about assessment.

An additional aspect for me to consider with my research area includes that of the inclusion of advisor training and application of student development theory. I had been considering the application of assessment to influence training, development and performance management.

References:

Kuh, G. D. (2008). Advising for student success. In V. N. Gordon, W. R. Habley, & T. J. Grites (Eds.), Academic advising: A comprehensive handbook (2nd ed.) (pp. 68–84). San Francisco, CA: Jossey-Bass.

King, M. C. (2005). Developmental academic advising. Retrieved from NACADA Clearinghouse of Academic Advising Resources Web site: http://www.nacada.ksu.edu/Resources/Clearinghouse/View-Articles/Developmental-Academic-Advising.aspx

White, E., & Schulenberg, J. (2012). Academic advising-a focus on learning. About Campus, 16(6), 11–17.