Siddata Study Assistant

The Siddata project (Studienindividualisierung durch digitale, datengestützte Assistenten) was an interdisciplinary research project investigating how students can be supported in pursuing their individual educational goals through a data-driven study assistant. It was developed at the universities of Osnabrück, Hannover, and Bremen, together with the HIS Institute for Higher Education Development, and funded by the German Federal Ministry of Education and Research (BMBF).

The idea was to combine information from different sources, such as university course catalogs, learning management systems, and students’ individual interests, to provide personalized recommendations. Instead of simply following the standard curriculum, students could discover courses outside their own study program, find relevant learning materials, connect with other students, or get support in planning their studies.

The assistant was integrated into Stud.IP, the learning management system used by all three universities, and was tested by more than 3,000 students during the project.

The Study Assistant

The assistant consists of a Stud.IP plugin providing the user interface and a central Python/Django backend, communicating through a REST API. The backend collects, processes, and connects data from the different universities, manages individual study goals, and generates personalized recommendations.

Its functionality is organized into independent recommender modules, which students can activate depending on their interests and needs. These include, among others:

• Fachliche Interessen: Students describe their interests in natural language and receive recommendations for relevant courses, including those at other universities, as well as MOOCs and Open Educational Resources (OERs). Machine learning is used to match resources based on their content rather than simply searching for keywords.
• Get Together: Connects students with similar interests, courses, or plans, for example to find study partners or exchange experiences about studying abroad.
• Auslandsaufenthalt: Supports students in planning a semester abroad, including information and reminders for the different stages of the application process.
• Meine Studienziele: Allows students to define, organize, and keep track of their individual study goals.
• Persönlichkeit: Uses psychological questionnaires to encourage self-reflection and provide information about individual learning strategies.

An important part of the concept was giving students control over which modules they wanted to use and which personal data they wanted to share. The backend operates on pseudonymized user data, while the individual recommenders use different approaches, ranging from simple rules and questionnaires to machine-learning-based semantic matching.

One example of the latter is SidBERT, a BERT-based NLP model that categorizes educational resources into knowledge domains based on their titles and descriptions. For this, it was trained using the Dewey Decimal Classification (DDC), a classification system commonly used in libraries. The same model can categorize a student’s freely formulated interests, allowing the system to identify courses and learning materials from related knowledge domains.

My Contribution

I worked on Siddata as a student research assistant at the University of Osnabrück. My responsibilities included developing machine learning recommenders for suggesting university courses and other educational resources, working on the Django backend, and maintaining the infrastructure required to operate the system.

However, a substantial part of my work eventually became turning the existing research prototype into properly maintainable, tested, and deployable software.

As is often the case with research software, the initial focus had been on implementing features and experimenting with different approaches, rather than long-term maintainability. Over time, the project had grown into a fairly complex system with numerous dependencies, several interacting services, and an increasingly difficult setup process.

Deploying a new instance could take around three days, involving manually installing dependencies, configuring services, and getting the different components to work together. This also made it difficult for new developers to get started or to reproduce the same environment on another machine.

I therefore substantially restructured the backend and its infrastructure, focusing on code quality, reproducibility, testing, and deployment. In particular, I worked on:

• Machine Learning: I developed and implemented machine-learning-based recommenders to suggest relevant courses and other educational resources to students, and worked on the infrastructure for running and serving these models.
• Containerization and Deployment: I created a Docker Compose setup covering the Django backend, PostgreSQL database, TensorFlow Serving, and the web server. Separate configurations allow the same project to run in development or production environments, without manually setting up the individual services.
• Backend Restructuring: I reorganized the Python project, introduced separate settings for development, production, and automated testing, and improved the code structure and maintainability.
• Automated Testing and Code Quality: I introduced code formatting and linting using Black and Flake8, enforced through pre-commit hooks and GitHub Actions. The CI pipeline automatically builds the Docker environment and performs smoke tests and health checks, including database connectivity and migrations.
• Data Analysis and Visualization: I developed a Plotly-based analytics dashboard for researchers to inspect aggregated project data, including the usage of individual recommenders, student participation, and characteristics of the user base.
• Documentation: I wrote installation and developer documentation covering the different environments, development workflows, and the containerized infrastructure.

The production setup uses nginx and uWSGI to serve the Django application, with PostgreSQL and TensorFlow Serving running as separate containers. Docker Compose manages these services and their dependencies, while the CI pipeline verifies that the application starts correctly and its essential components are working.

The result was that setting up the complete backend on a fresh server went from a process that could take days to essentially a single Docker Compose command, once Docker and the necessary configuration were available. No more manually installing and configuring the entire software stack!

Research and Master’s Thesis

Besides the development work, I also contributed to the scientific research and writing associated with the project. In particular, I co-authored an early manuscript describing the software architecture, its modular recommender design, and the analytics infrastructure, in which I was also listed as an author. A later version was published as A Web-Based Recommendation System for Higher Education: SIDDATA - History, Architecture and Future of a Digital Data-Driven Study Assistant in 2022.

I also conducted my master’s thesis on Explainable Course Recommendation within the scope of the Siddata project.

The motivation behind it was closely related to the challenges of the study assistant: How can we recommend educational resources based on their content, while also making the recommendations understandable to students?

Traditional embedding models can represent the semantic similarity between texts, but the individual dimensions of their vector spaces are generally not interpretable. For my thesis, I investigated a different approach based on Conceptual Spaces, in which the dimensions represent meaningful semantic properties.

Using descriptions of university courses, I developed a pipeline to derive such a space directly from the data, with the aim of enabling explainable recommendations. Instead of simply recommending courses because they are similar to something a student already likes, the system could potentially explain in what way they are similar, or allow students to explicitly express preferences along dimensions such as how mathematical or theoretical a course should be.

The project involved a scalable machine learning pipeline that I implemented using Python and Snakemake, running on the university’s computing cluster. The results indicated that the generated representations captured interpretable semantic features of the courses, providing a promising basis for more transparent recommendation systems.

More about the methodology, implementation, and results can be found in the dedicated portfolio entry for my master’s thesis.

Links

• Siddata Project - The official project website.
• Siddata Backend - The open-source Django backend, including the recommender modules and analytics dashboard.
• My Docker and Infrastructure Implementation - My containerized backend setup, automated testing infrastructure, and documentation.
• Stud.IP Frontend - The frontend plugin integrating the assistant into Stud.IP.
• My Master’s Thesis: Explainable Course Recommendation - Research on interpretable embeddings of educational resources for explainable recommendation.
• A Web-Based Recommendation System for Higher Education: SIDDATA - Scientific publication describing the project’s architecture and functionality.