ChatTutor pilot study

ChatTutor pilot study#

During this course, we will conduct a pilot study on using Large Language Models (LLMs) as a study aid. You can access chattutor at chattutor.dk.

Warning

ChatTutor is not mandatory. Participation will not influence your evaluation. You can get help without using ChatTutor.

ChatTutor allows you to engage in conversation with a LLM (ChatGPT) on a site that contains our book, the slides, and other material. We will attempt to base the answers in the study material, and the platform will also allow you to escalate any unanswered questions, or questions that weren’t answered sufficiently by the AI, to a human teaching assistant.

Early build of ChatTutor

The aim of the study is to:

  • Get feedback on the interface and how we can improve it

  • Get information on how it impacts your learning

  • Hopefully get a better understanding on how DTU can use AI in the future

If you wish to participate in the study look out for further information on DTU Learn.

For other researchers#

Note

Any study will obviously be conducted in a voluntary basis and after appropriate anonymization.

We are looking for collaboration with researchers with an interest in the use of AI and higher education.

It is our experience that most research into how students interact with AI has been based on observing limited and highly structured interactions followed by interviews and questionnaires.

ChatTutor offer a complementary window into how learners engage with AI over longer time and based on their actual goal: to master a subject. We believe that augmented with questionnaires/interviews, this can offer insight into how AI is best applied in education. If you are interested in a collaboration please contact us at

Important note on privacy#

Large language models in general offer a number of privacy challenges and these carry over to ChatTutor. It is important you are aware of them before signing up.

  • Your answers are generated by a commercial AI (ChatGPT). OpenAI, who own ChatGPT, will most likely store your data and use it for training and other purposes.

  • We will also store your conversations, information about when you log in, and other common metadata.

  • One of the main ideas in ChatTutor is that questions you ask to TAs should be public so that others can benefit from the answers, that is, visible to students who are already logged in.

    • Only the questions/chat escalated to a human TA will be visible by other students

    • However, this is an application that is in development, which creates a higher risk of hacking or data leakage between accounts. You should not input information into ChatTutor which could harm you in any way if made public through accident or malice.

  • As part of the evaluation, we may use examples of your past chats (anonymized to not contain identifying information) as examples of interactions. We will also store information such as the number of logins etc. to monitor participation.

  • Please ask if you have any questions about what data will be collected or how it will be used.

Contents