🎓

Breakthrough Innovation in Education: EdTech Startups and their Learning Analytics Technologies

Learning analytics is an emerging subject discussed by many authors in the literature. By utilizing computational tools and statistical models on student data underlying patterns can be identified to derive knowledge. In addition, learning analytics provides a positive impact on learning engagement, personalization and supporting students emotional mindset. Many universities already identified the benefits of learning analytics and therefore developed tools individually. Nonetheless, startups also identified the potential of learning analytics and started to create their own products. This thesis discusses three startups (Knewton Alta, Acrobatiq, Civitas Learning) that focus on learning analytics in higher education and examines their provided analytic tools. In particular, the thesis evaluates their proposed solutions to tackle challenges mentioned in the literature. In addition, a learning analytic tool (Bookshelf CoachMe) is examined and evaluated in regards to their degree of improving learning engagement and personalisation.

📋 Key facts

💡

Research question 1: Analyze the current technologies applied by learning analytics startups. What methods are used to increase learning engagement / personalization? What challenges are mentioned in the literature and how do the startups cope with these challenges?

💡

Research question 2: Assess a learning analytic tool in regards to the degree of learning engagement and personalization.

👩🏼‍💻 Review of Learning Analytics Startups

‣

Knewton Alta and Civitas Learning belong to the first companies founded, which focus on learning analytics.

They both provide visual dashboards giving instructors further insight into students course completion rate or learning progress.
In the context of predictive assessment, Knetwon Alta went one step further and predicted the learning target and possible knowledge gaps based on student feedback and input data to personalize courses and tasks. Civitas learning alerts tutors, if students face difficulty to pass courses.
In the area of engagement both platforms encourage social and collaborative engagement due to offering chat systems where students can exchange learning's and seek for help actively.

‣

Acrobatiq is another startup which was aquired by Vitalsource.

Preventive measures are taken to determine if students are missing learning objectives and are therefore at higher risk of failing courses.
In the area of engagement Acrobatiq also focuses on the teaching perspective. Courses can be shared between tutors to encourage developing digital courses collaboratively. Other tutors can therefore learn from courses created by their colleagues and customize them according to the learning subject.

‣

Challenges

Several challenges that learning analytics tools have to overcome have been pointed out in the literature. In summary, all assessed startups recognized most of them and developed appropriate solutions.
For example, research and use cases have been carried out in collaboration with universities to measure how much failure rates are reduced or student retention is increased by employing learning analytics platforms. The results were promising and showed further application possibilities.
Also data regulation and data security measurements were introduced to prevent unauthorized access and keep student data secure.
Nonetheless the assessment of the startups also showed their shortcomings. The startups did not address ethical topics to make sure that the data is used ethically.
In addition, all startups did not take students emotions into consideration. Other authors from the literature identified, that emotions like anxiety, motivation or frustration have a significant impact on learning progression. Nevertheless, it can be stated that the platforms are widely deployed across multiple universities and support the learning and teaching process significantly.

💡 Results of research question 1:

Predictive algorithms reduce students course failure rate and increase student persistence
Startups focus on increasing student engagement and personalization by providing tasks adapting to students knowledge gaps
Visual dashboards show further statistical insights to help tutors adjust courses and approach students optimaly
Ethical impact and emotional support are not covered by the startups

👩🏼‍💻 Case Study Bookshelf CoachMe by Vitalsource

‣

Within the thesis the learning analytic tool provided by Vitalsource named Bookshelf CoachMe was assessed directly by the author. The learning analytic feature can be considered as an add-on to digital books, which automatically generates questions related to the learning content. Answering the questions helps the user to check their own level of knowledge and to deepen the learning objectives. In order to be able to assess the feature, a case study is conducted measuring the level of learning engagement and personalization. In particular, learning improvement, engagement and personalization in learning are investigated on the one hand with the textbook itself and on the other hand with the the learning analytics tool Bookshelf CoachMe.

During the experiment, the assessment is conducted on a business economic topic and the book "Fundamentals of Corporate Finance" is selected.
The developers of the feature Bookshelf CoachMe created the tool by utilizing natural language processing algorithms. First, they identified learning objectives of each chapter. Afterwards the objectives were structured according to the books content. Finally, questions are generated automatically.
The automatically generated questions through Bookshelf CoachMe are fill-in-the-box and matching questions. Both questions are not distributed equally within the feature, while former mentioned questions represented the majority (82 %) while later mentioned the minority (18 %) respectively.
The results of the case study in regards to engagement show a clear trend. In total 72 % of the fill-in-the box questions and 86 % of all matching questions from the learning analytics tool were answered, while only 53 % of questions inside the textbook were not skipped.
Furthermore, a questionnaire was used to measure the level of learning engagement. The questions within the questionnaire focus on measuring the cognitive and behavioral engagement.
The findings of the questionnaire concerning cognitive engagement are in line with the previously established results. However, the findings discussing behavioral engagement show no distinction between learning with and without the learning analytics feature.
During the case study, the questions in the textbook engaged the least, while the automatically generated matching questions were answered the most. In the area of personalization the tool provided a functionality which generated new fill-in-the-box questions based of the users input and learning gaps. In the context of the case study, this helped to answer further questions (about 20 %).

💡 Results of research question 2:

The feature lead to an increased engagement by answering 26 % more questions.
Personalized questions based on the users input lead to an increase of 20 % of more answered questions.
In conclusion it can be stated, that bookshelf CoachMe supports the teaching and learning process similarly to the assessed startups. It also has to be pointed out, that no clear statement was possible within the thesis to determine, if the learning analytics platforms directly improved learning outcomes.

🖼️ Thesis Infographic

📈 Outlook

The results of the use case carried out supports the statement, that learning analytics improves learning and teaching. However, it should be noted that the tool Bookshelf CoachMe was only evaluated with n=1 participants and on one textbook/digital book. In the future, the results should be evaluated with a larger number of participants as well as a variety of books to obtain statistical significance and to reduce bias.
After reviewing existing learning analytics startups, one can conclude that the companies provide emotional support only sparsely. Therefore, startups should evaluate emotionally supporting students by employing questionnaires or deep-learning methods based of video material to be more aware of students emotions.
One challenge that has been recognized in the literature concerns the ethical compliance of learning analytics methods. In the future the tools should by evaluated in regards to the ethical background. In the future, tools should be evaluated in terms of ethical background.
For the purposes of this work, the evaluation focused mainly on assessing learning analytics tools in regards to their level of engagement and personalization. Several authors mentioned that the use of certain learning approaches coupled with analytics methods improves learning speed by up to six times. In the future, the actual learning improvement in regards to the learning speed should be examined.