Participant Information Leaflet for Students

Trialling an evaluation framework for LLM-powered careers advice - Students

Participant Information Leaflet for Students

果冻传媒 Title:	Trialling an evaluation framework for LLM-powered careers advice
Investigator(s):	Peter Dickinson, Deidre Hughes, Chris Percy, Honorary Research Fellow and Stef Poole IER (Institute for Employment Research, 果冻传媒); Vincent Ramos (University of Southampton).

A downloadable version of this leaflet is available here.

Introduction

You are invited to take part in a research study. Before you decide, you need to understand why the research is being done and what it would involve for you. Please read the following information carefully. Please contact us if there is anything that is not clear or if you would like more information. Take the time to decide whether or not you wish to take part.

Who is organising and funding the study?

The study is being funded by the Nuffield Foundation and being carried out by researchers from the 果冻传媒.

What is the study about?

The study is exploring how the advice given by different types of AI chatbots compares to that of careers advisers.It is also developing an evaluation framework that people can use to help them make comparisons between different careers advice services.

Do I have to take part?

Your participation in this study is entirely voluntary and you can withdraw at any time without giving a reason.

What would taking part involve?

There are one of two roles that you might be assigned to:

路Participation in a randomised control trial where we ask you to seek and receive careers advice from either one of four sources:

-A general AI-chatbot;

-A chatbot that is designed to provide careers advice and guidance;

-An experienced careers adviser;

-An experienced careers adviser with access to a chatbot that is designed to provide careers advice and guidance.

This would be for 60 minutes during a one week period.

This will involve you completing a pre-survey just before the intervention, a post-survey just after completion, and a 6-8 week follow-up survey.

Your involvement should take no longer than 90 minutes.

路Reviewing the useability, accuracy and relevance of AI chatbots. This will involve you asking questions of a general chatbot or a dedicated careers chatbot and completing a survey on their useability, usefulness of information provided, and relevance to your situation. And providing feedback on the process.

Your involvement should take no longer than 60 minutes.

NB: We understand that some students may be making important careers decisions at the time of their involvement in the study.Your involvement in the study does not exclude you from seeking careers information, advice and guidance that you usually use or should want to use.You would be entirely free to seek and use whatever careers information, advice and guidance you need during your participation in the study.

What are the possible benefits of taking part in this study?

It will help in the current and future evaluations of AI chatbots and the careers advice they provide to see if it as good as can be provided.

What are the possible disadvantages, side effects or risks, of taking part?

There are no anticipated risks to you as a research participant.

Expenses and payments

You will receive a 拢20 Amazon/High Street voucher on completion of your participation in the study.

Will my taking part be kept confidential?

Yes. We will follow strict ethical and legal practice and all information about you will be handled in confidence. All the information and data you provide will be pseudonymised, this means your contact details will not be stored in the same file as other data you provide. All data will be stored securely in a password-protected folder at the 果冻传媒. Only the named researchers will have access to the data. No data will be transferred outside of 果冻传媒.

What will happen to the data collected about me?

Any information you provide will be kept securely in a password-protected folder on 果冻传媒 servers, only for the purposes of the research. We will only use the data you provide in the ways needed to conduct this evaluation and analyse the data resulting from it.

We will be using information from you in order to undertake this study and 果冻传媒 will act as the data controller for this study. We are committed to protecting the rights of individuals in line with data protection legislation. Your contact details will be destroyed immediately after the end of the project (December 2027). Consent information will be kept securely for 10 years after the study has finished and then destroyed securely.

Research data will be pseudonymised as quickly as possible after data collection. This means all direct and indirect identifiers will be removed from the research data and will be replaced with a participant number. The key to identification will be stored separately and securely to the research data to safeguard your identity. It will be possible for us to withdraw your data up to 3 weeks after the data has been collected and this data will then be securely destroyed if you wish us to do so.

Some of the interventions involve the use of AI tools. In using these tools you may disclose personal details. This information will be reviewed by a careers adviser to check the accuracy and quality of support provided in response to your queries. All transcripts will be anonymised so neither the AI tools nor the careers advisers reviewing the tools will know who you are or your contact details. If any issues or concerns are identified (such as incorrect information shared with you) the research team will arrange for you to speak to a careers adviser about this. The data are also processed by the companies running the platforms but we will turn off training data so these companies cannot use it for their training purposes.

The research team have arranged licenses to access the necessary AI tools, which include frontier large language models, ensuring that the conversations will not be used except as required for the purposes of this research and standard legal duties.

Data Sharing

Data will not be shared with any other organisation.

Your rights to access, change or move your information are limited, as we need to manage your information in specific ways in order for the research to be reliable and accurate.

The 果冻传媒 have in place policies and procedures to keep your data safe.

What will happen if I don鈥檛 want to carry on being part of the study?

Participation is entirely voluntary, and a decision to withdraw participation from the study without giving a reason, will not affect you in any way. If you decide to withdraw from the study, please tell us before or during your participation and we will destroy your contact details securely and will not contact you again for the purposes of this research. You can stop being involved in the study at any point without giving a reason.

After the your participation in the study has ended, you will have three weeks to withdraw your data. This can be done by contacting either of the researchers below. It will not be possible to withdraw your data 3 weeks after data has been collected. To safeguard your rights, we will use minimum personally-identifiable information and keep the data secure.

What will happen to the results of the study?

The findings of the study will be presented in a report to the Nuffield Foundation and made publicly available. All data will be presented in an aggregate form e.g. in tables and graphs. The report will be made available on the Nuffield Foundation鈥檚 and 果冻传媒鈥檚 website.

Who should I contact if I want further information?

If you have any questions about any aspect of the study, or your participation in it, not answered by this participant information sheet, please contact:

Peter Dickinson at p.dickinson@warwick.ac.uk

Stef Poole at stefanie.poole@warwick.ac.uk

Who should I contact if I wish to make a complaint?

Any complaint about the way you have been dealt with during the study or any possible harm you might have suffered will be addressed. Please address your complaint to the person below, who is a senior 果冻传媒 official entirely independent of this study:

Head of Research Governance

Research & Impact Services

University House

果冻传媒

Coventry

CV4 8UW

Email: researchgovernance@warwick.ac.uk

Tel: 024 76 522733

If you wish to raise a complaint on how we have handled your personal data, you can contact our Data Protection Officer, Information and Data Director who will investigate the matter: infocompliance@warwick.ac.uk

If you are not satisfied with our response or believe we are processing your personal data in a way that is not lawful you can complain to the Information Commissioner鈥檚 Office (ICO).

Thank you for taking the time to read this Participant Information Leaflet

PRIVACY NOTICE

How we will ensure that we are using your personal data fairly and transparently in accordance with the requirements of the General Data Protection Regulation (GDPR)

How we will use your personal data

If you agree to participate, we will ask you to confirm your name and email address. We will use the information you provide as a basis for analysis in order to develop the study鈥檚 findings.

The nature of your personal data that we will be using

The categories of personal data that we will ask for are:

路Name and contact email address when completing the consent form;

路Where payments for participating in the study can be made.

Why our use of your personal data is lawful

Under the GDPR, we are required to have a lawful basis for processing your personal data. For the purpose of this project, the lawful basis is that processing is necessary to perform a task in the public interest (e.g. providing unbiased research evidence in an article).

Who will have access to your personal data?

Access to your personal data will be restricted to members of the 果冻传媒 Institute for Employment Research (IER) project team for this research project. We will not report your name or anything that would make you personally identifiable in any outputs from the research and will use our best endeavours to ensure that data in any project outputs (e.g. reports and presentations) will not allow individuals or their organisations to be identified.

For the purposes of reporting, any identifying information will be removed to ensure confidentiality. The information you provide will be stored safely on a password-protected folder at the 果冻传媒 and will only be accessible by the research team. Contact details will not be stored in the same file as the information you provide. IER and its data storage procedures are fully compliant with GDPR. Aggregated research data will be made available (for example, in a published report or article) but they will not have access to any individual information, names, email addresses, telephone numbers and other identifying details of participants. The collated research findings will be published in a research report and a journal article but none of your personal data will be included in the report.

Your right to withdraw consent

Participation in this study is entirely voluntary. Refusal to participate will not affect you in any way. If you do agree to participate, you may withdraw from the study at any time: there will not be any consequence of this. You have the right to withdraw from the study completely and decline any further contact by researchers after you withdraw. However, you will not be able to withdraw any data you provide (apart from contact details) if they have been collated and analysed to produce reports.

If you take part and inform us that you wish to withdraw from the research we will not use your response in the research and we will delete your responses, as long as it has been withdrawn before the data has been analysed.

How long we will keep your personal data?

We will retain your pseudonymised data for ten years in accordance with the 果冻传媒 retention requirements for research data. Your stored contact data will be deleted as soon as is practicable after the end of the project.

Your data protection rights

Under certain circumstances, you have the right:

路to ask us for access to information about you that we hold;

路to have your personal data rectified, if it is inaccurate or incomplete;

路to request the deletion or removal of personal data where there is no compelling reason for its continued processing;

路to restrict our processing of your personal data (i.e. permitting its storage but no further processing);

路to object to direct marketing (including profiling) and processing for the purposes of scientific/historical research and statistics;

路not to be subject to decisions based purely on automated decision-making.

If you need to contact us regarding any of the above, please email the 果冻传媒鈥檚 Data Protection Officer at infocompliance@warwick.ac.uk. For further details see /services/idc/dataprotection/datasubjectrights

Your right to complain

If you are unhappy with our use of your personal data, please let us know by contacting:

Head of Research Governance