Medical Chatbot Response Evaluation

Welcome! 👋

Thank you so much for taking the time to help with this study! Your thoughtful evaluation will help us make medical conversation systems more warm and empathetic.

Please select your evaluator ID:

Please enter your name:

Evaluation Guide 📋

We really appreciate you being here! This evaluation helps us understand how well medical conversation systems express care and empathy.

📝 What you'll see:

A patient's medical question
Four different responses to that question
These responses may vary in tone and style

💡 A few helpful notes:

We're keeping you in the blind about where responses come from - this helps ensure fairness
Please evaluate each response on its own merits, independently of the others
For this study, we'd like you to focus on emotional qualities rather than medical accuracy
The key question is: How well does this response address the patient's emotional needs?

✨ Your task:
For each response, you'll rate 6 dimensions on a 5-point scale (from "Strongly Disagree" to "Strongly Agree").

📊 What you'll evaluate:
You'll evaluate 25 patient questions, with 4 responses each (100 responses total). You can pause and resume whenever you need.

💾 Auto-Save: Your progress is automatically saved after each response to both your browser and server. You can safely close the browser and resume later from the same device.

Jump to:

Patient Question

Chatbot Response

🎉 Excellent work!

You've completed 50 responses.

You're doing great! Feel free to take a short break - stretch, grab some water, or just rest your eyes.
When you're ready, just click below to continue.

All done!

Thank you very much for completing the evaluation study.

We truly appreciate the time, care, and thoughtful attention you devoted to reviewing the chatbot responses. Your evaluations play a critical role in helping us rigorously assess compassion and responsiveness in medical AI systems, and the quality of this work depends directly on careful human judgment like yours.

We recognize that this was a detailed and time-intensive task, and we are grateful for your commitment and professionalism throughout the study.

If you have any questions about the study or would like to receive a brief summary of the results once the project is complete, please feel free to reach out.

With thanks and best regards,

Hüseyin, Wenqi, Zihao, Natalia

Your responses have been securely saved to our server.