What is the relationship between instructors' perceptions of data privacy and their adoption of the learning analytics technology?

Research Approach

Methodology: Designed and distributed surveys incorporating vignettes—hypothetical teaching scenarios—to elicit instructors' reactions concerning privacy and usefulness.

Participants: University instructors who teach at least one course across various disciplines, ensuring a representative sample of the academic teaching community.

Research Process

Secondary Research

I examined existing research and discovered that a few studies have identified challenges for instructors' adoption of learning analytics applications for teaching and learning, but no study has explored instructors perceived usefulness, data privacy concerns, and their impact on adoption.

Methodology

I selected a survey methodology to capture a broad range of instructors' perspectives on data privacy and learning analytics. To make the survey engaging and contextually relevant, I incorporated vignettes—brief, hypothetical scenarios in teaching—which are an effective and efficient way to elicit user perspectives in realistic situations. Surveys provided a fast and cost-effective method for examining relationships between concepts, enabling timely insights without the need for complex experimental setups. The chosen analysis approach further facilitated a rapid exploration of these relationships, making it ideal for this study.

Challenge

Ensured vignette scenarios were realistic and relatable to accurately capture instructors' perceptions.

Literature Review to identify issues has been examined about privacy and adoption of learning analytics applications among instructors.

Survey to discover a large group of users thoughts on privacy and adoption in a short period of time.

Vignettes to evoke reactions from users in a series of different situations.

Example Vignettes

The following are some examples of vignettes we used in this study. The vignettes are either outcome oriented or socially oriented scenarios of use cases of analytics in the teaching and learning context. Users are asked to rate these vignettes on expected usefulness, privacy concerns, and adoption likelihood.

Poll: While prepping for tomorrow’s CHEM 101 class, Taylor pulls up a Canvas visualization of last year’s responses to a clicker/Tophat poll on the same topic. He sees that 350 of his 395 students chose the same wrong answer in the poll, and decides to add a second clarifying example to the lecture materials. (STEM, large, in-class activity, class prep)

Social: During his 30-student mechanical engineering design class, Bayley monitors team dynamics using a social behavior analytics dashboard (e.g., classification of facial expressions, gestures, body positions). When he sees Team 3 slowing down, he heads over for a check in. (STEM, small, social interactions, class management)

Prereqs: Alex is developing a third-year course in machine learning for the new data sciences major. He reviews assignment and test grades for current majors during their first two years, and he can see that about 30% perform poorly on programming intensive assignments. He expands his early review and extra resources relating to core programming skills. (STEM, small, grades, new course development)

Collaboration: Priya is about to have a coaching session with one of her industrial engineering teams. To prepare, she reviews team communication analytics that show how frequently members communicate and update shared documents, as well as which members are texting and emailing others. This gives her insight into the team dynamics. (STEM, small, social interactions, mentoring)

Findings

Themes:

Privacy concerns, perceived usefulness, and their impact on technology adoption in educational settings.

Key Insights:

• Identified a negative correlation between privacy concerns and perceived usefulness; higher privacy concerns led to lower adoption likelihood.

• Specific use cases, such as monitoring team dynamics through analytics, raised significant privacy apprehensions among instructors.

Impact

Recommendation:

• Design learning analytics tools with transparent data usage policies.

• Incorporate customizable privacy settings to empower instructors.

• Provide clear value propositions to mitigate privacy concerns and enhance perceived usefulness.

Outcomes:

Developed actionable design implications aimed at increasing instructor adoption of learning analytics by addressing privacy concerns.

Reflection

Lessons Learned:

• The importance of considering user privacy perceptions in technology adoption.

• The effectiveness of using vignettes to gather nuanced user insights.

Growth:

Enhanced proficiency in survey design, data analysis, and understanding of ethical considerations in educational technology.