By Professor Dr. Dirk Ifenthaler, University of Mannheim

Learning analytics uses dynamic information about learners and learning environments, assessing, eliciting and analysing it for real-time modelling, prediction and optimisation of learning processes, learning environments and educational decision-making.

With the increased availability of vast administrative systems, academic and personal information within educational settings, and educational data management analysis, interpretation is becoming complex. Several concepts closely linked to processing such educational information are educational data mining, academic analytics and learning analytics. However, these concepts are often confused and lack universally agreed as well as applied definitions. Educational data mining (EDM) refers to the process of extracting useful information out of a large collection of complex educational datasets. Academic analytics (AA) is the identification of meaningful patterns in educational data in order to inform academic issues (e.g., retention, success rates) and produce actionable strategies (e.g., budgeting, human resources). Learning analytics (LA) emphasises insights and responses to real-time learning processes based on educational information from digital learning environments, administrative systems and social platforms. Such dynamic educational information is used for real-time interpretation, modelling, prediction and optimisation of learning processes, learning environments and educational decision-making.

Benefits of Learning Analytics

The benefits of learning analytics can be associated with four levels of stakeholders (see Figure 1): mega-level (governance), macro-level (institution), meso-level (curriculum, teacher/tutor), and micro-level (learner, OLE). An essential prerequisite for LA benefits, however, is the real-time access, analysis and modelling of relevant educational information.

The mega-level facilitates cross-institutional analytics by incorporating data from all levels of the learning analytics framework. Such rich datasets enable the identification and validation of patterns within and across institutions and therefore provide valuable insights for informing educational policymaking.

The macro-level enables institution-wide analytics for better understanding learner cohorts for optimising associated processes and allocating critical resources for reducing dropout and increasing retention as well as success rates.

The meso-level supports the curriculum and learning design as well as providing detailed insights about learning processes for course facilitators (i.e., teachers, tutors). This information can be used for improving the overall quality of courses (e.g., sequencing of learning processes, alignment with higher level outcomes) as well as enhancing learning materials (e.g., their alignment to anticipated learning outcomes and associated assessments).

The micro-level analytics supports the learner through recommendations and help functions implemented in the OLE. Learners benefit from such personalised and adaptive scaffolds and are expected to be more successful in reaching the learning outcomes.

Another critical component for improving the benefits of LA is information from the physical environment (e.g., learner’s current emotional state) which is not directly linked with the educational data. Accordingly, data may be collected within the OLE through reactive prompts and linked with the available educational information.

Figure 1. Learning analytics associated with stakeholder levels

Challenges

More educational data does not always make better educational data. Hence, LA has its obvious limitations and data collected from various educational sources can have multiple meanings. Therefore, serious concerns and challenges are associated with the application of LA:

(1) Not all educational data is relevant and equivalent. Therefore, the validity of data and its analyses is critical for generating useful summative, real-time and predictive insights. This generates a new interdisciplinary research area for cognitive psychology, educational technology, learning design, psychometrics, data management, artificial intelligence, web development and statistics. The challenges are to investigate the complex processes within LA frameworks and to understand their immediate and long-term effects on learning and teaching processes.

(2) Ethical issues are associated with the use of educational data for LA. That implies how personal data is collected and stored as well as how it is analysed and presented to different stakeholders. Hence, procedures regulating access and usage of educational data need to come into operation before LA frameworks are implemented. This will also include transparency of applied algorithms and weighting of educational data for predictive modelling. Storing and processing anonymised personal data is only a small step towards a more comprehensive educational data governance structure for LA.

(3) Limited access to educational data generates disadvantages for involved stakeholders. For example, invalid forecasts may lead to inefficient decisions and unforseen problems. A misalignment of prior knowledge, learning pathways and learning outcomes could increase churn and the late identification of learners at risk may create dropouts. A definition of threshold standards for LA could prevent vast gaps between educational institutions and provide equal opportunities for all stakeholders.

(4) The preparation of stakeholders for applying insights from LA in a meaningful way is vital. Professional development for stakeholders ensures that issues are identified and benefits are transformed into meaningful action. Hence, the increased application of LA requires a new generation of experts with unique interdisciplinary competences. This will also require new infrastructures for administration and research in order to accelerate the understanding of LA.

(5) Information from distributed networks and unstructured data cannot be directly linked to educational data collected within an institution’s environment. An aggregation of such data and uncontrolled relations to existing educational data increases the chance of critical biases as well as invalid analysis, predictions and decisions. The challenge is to develop mechanisms to filter biased information and warn stakeholders accordingly.

(6) An optimal sequence of data collection and economic response times (seconds, minutes, hours, days, weeks) of LA have yet to be determined. This includes the minimum requirements for making valid predictions and creating meaningful interventions. Missing data is a critical challenge for future LA algorithms.

(7) Besides the analysis of numerical data (e.g., click streams), a qualitative analysis of semantic rich data (e.g., content of discussion forums, responses to open-ended assessments) enables a better understanding of learners’ knowledge and needs. An obvious requirement is the development of automated natural language processing (NLP) capabilities. The major challenge besides the development of real-time NLP is the validation of such algorithms and the link to quantitative educational data.

Future Directions

Future applications of LA presuppose a seamless and system-inherent analysis of learner’s progression in order to continuously adapt the learning environment. LA provides the pedagogical and technological background for producing real-time interventions at all times during the learning process. It is expected that the availability of such personalised, dynamic and timely feedback supports the learner’s self-regulated learning as well as increases their motivation and success. However, such automated systems may also hinder the development of competences such as critical thinking and autonomous learning.

It is expected that a holistic LA framework will provide evidence for increasing the overall quality of learning environments and facilitate the reform of learning and teaching in the 21st Century.

Selected References

Bellin-Mularski, N., & Ifenthaler, D. (2014). Learning Analytics: Datenanalyse zur Untestützung von Lehren und Lernen in der Schule. SchulVerwaltung NRW, 25(11), 300-303.

Ifenthaler, D. (2014). Toward automated computer-based visualization and assessment of team-based performance. Journal of Educational Psychology, 106(3), 651-665. doi: 10.1037/a0035505

Ifenthaler, D., Adcock, A. B., Erlandson, B. E., Gosper, M., Greiff, S., & Pirnay-Dummer, P. (2014). Challenges for education in a connected world: Digital learning, data rich environments, and computer-based assessment – Introduction to the inaugural special issue of technology, knowledge and learning. Technology, Knowledge and Learning, 19(1-2), 121-126. doi: 10.1007/s10758-014-9228-2

Ifenthaler, D., & Widanapathirana, C. (2014). Development and validation of a learning analytics framework: Two case studies using support vector machines. Technology, Knowledge and Learning, 19(1-2), 221-240. doi: 10.1007/s10758-014-9226-4

Ifenthaler, D. (2015). Learning analytics. In J. M. Spector (Ed.), Encyclopedia of educational technology (Vol. 2, pp. 447–451). Thousand Oaks, CA: Sage.

Ifenthaler, D., Bellin-Mularski, N., & Mah, D.-K. (2015). Internet: Its impact and its potential for learning and instruction. In J. M. Spector (Ed.), Encyclopedia of educational technology (Vol. 1, pp. 416–422). Thousand Oaks, CA: Sage.

Prof. Dirk Ifenthaler

Professor Dr. Dirk Ifenthaler, University of Mannheim

The Council on Business & Society Global Alliance is an ongoing international dialogue between six of the world’s leading business schools and an organiser of Forums focusing on issues at the crossroads of business and society – The Council Community helps bring together business leaders, academics, students and journalists from around the world. #CouncilonBusinessandSociety