TEACHING · ACADEMIC WORK

Research question
and method

Turn a broad topic into a question you can answer with the sources and time you actually have.

QUESTIONS
SUPPORT WITH EVIDENCE
JUSTIFY

My topic is too broad. How do I narrow it?

Write down what you want to find out
Put the question in one sentence. Name who or what you will study.
See what you can check
Decide what sources, data or examples you really need.

An example from campus printing

A question for the paper
Are AI answers easier to verify when the prompt asks for a citation?
A fair comparison
Give the AI the same question and handbook twice. Ask for the relevant passage in only one version.
Permitted AI assistance
Ask the AI to flag vague terms. Decide for yourself which question you can answer.

Narrow down your research question

  1. Write out your question

    Use one sentence to say exactly what you want to find out.

  2. Check your sources and time

    Find out whether you can obtain the material or data in time.

  3. Record your decision

    Explain why this version suits the assignment.

A prompt you can adapt

Here is my topic [topic] and a draft question [draft]. Which three words should I make more precise? Ask me questions so I can narrow the topic myself.

Quick check Can I answer the question with these sources in the time available?

Find a question you can answer

The case study question How many AI answers contain at least one unsupported claim? I compare answers to the same questions, first without and then with a request to cite the exact section of the IT service handbook.

How to compare two answers fairly

Use two fresh chats, the same question, handbook and model settings. Change only one thing: in the second chat, also ask for the exact passage in the handbook.

First prompt “Answer in no more than three sentences using the handbook. Say when the handbook does not give enough information.” Additional instruction for the second prompt “Give the relevant section number for each factual claim.”

Both chats receive the same question and handbook. Only the second is asked to cite the exact passage.
Both chats receive the same question and handbook. Only the second is asked to cite the exact passage.

Start each run in a new chat. Record the date, model and edition of the handbook.

IT service handbook and test questionsDOCX · Go to template

A fictional exchange to work through

My question for the AI

My question is “Are AI answers with citations better?” Which words do I need to define?
What the AI says
The AI suggests asking about reliability in general.
How I check it
I count only answers with at least one unsupported claim. I can check this for each answer.
What I record
I record how the feedback helped me narrow the question.

When does an answer count as unsupported?

How to score the answers: Flag an answer if any factual claim is missing from or contradicts the handbook. Count each answer once even if it has several errors. If the handbook leaves the question open and the response says so, the answer contains no unsupported claim.

In the example table, 1 means “at least one unsupported claim” and 0 means “no unsupported claim”. Use separate columns to judge completeness and whether the cited passage supports the answer.

The rubric scores factual claims, whether the question was answered and citation quality separately.
The rubric scores factual claims, whether the question was answered and citation quality separately.

Example: “Until 4 pm” matches the “Service desk” section. A citation to “Enquiries” does not support it. Score the claim and the reference separately.

Define what to countDOCX · Go to template

Test the rule on three questions

Try your rule on three short examples. In the service desk answer, the time is right and the reference is wrong. In campus printing, the figure is wrong. The handbook gives no rule for late fees. Record how you would score each case.

Three pilot cases: a correct time with the wrong citation, an incorrect printing limit and no handbook rule for late fees.
Three pilot cases: a correct time with the wrong citation, an incorrect printing limit and no handbook rule for late fees.

Finalise the rule after the pilot, before you score the actual answers.

Try out your scoring ruleDOCX · Go to template

Exercise · Set up a fair comparison between two chats

You want to see whether asking for an exact citation reduces unsupported claims. In the first chat you enter only the question. In the second, you enter the question, the handbook and a citation request. What would make this a fair comparison?

Justify your choice of methodDOCX · Go to template

Give both chats the same information

Use the same handbook, question and core prompt each time. Use the same model and start a fresh chat for each answer.

Why? If only one chat has the handbook, the information alone could explain any difference. Fresh chats also keep earlier questions and replies from influencing the next run.

Add exactly one instruction

Add “Cite the exact passage in the handbook that supports your answer” to the second prompt. Give the first chat only the core prompt.

Why? This isolates the change you are testing. If you also switch models or reword the question, it is harder to attribute the difference to the citation request.

Set the scoring rule before comparing

Count an answer once if at least one factual claim is missing from or contradicts the handbook. Score the citation in a separate column.

Why? Both groups are judged by the same standard. An answer that says 50 MB is still factually wrong when it names the “Campus printing” section.

Further reading

  • In Vorbereitung. Aufgaben zwischen Mensch und KI abgrenzen und Übergaben gestalten.

    KI-ARBEITSTEILUNG

    A quick guide to tasks, collaboration and productivityQUICK GUIDES
    Welche Arbeit übernimmt die KI, wo bleibt menschliches Urteil gefragt und wer prüft Ergebnisse? Der QuickGuide hilft, Aufgaben aufzuteilen. Am Beispiel eines Berichtsentwurfs zeigt der Einstieg, wie sich Vorbereitung, Entwurf, fachliche Prüfung und Freigabe aufteilen lassen. Er erläutert, wie Teams Übergaben vereinbaren und im …

Templates for this chapter

Draft your question or run a small test of how you might answer it.

The guide and its figures are in English. The downloadable templates, handbook and example data are the German originals. In those files, “Campus printing” appears as “Campusdruck” and “With citation request” as “Mit Bitte um Fundstelle”.

  • Outline your study

    Summarise your question, approach and open decisions on one page.

  • IT service handbook and test questions

    Open the IT service handbook, test questions and both AI prompts.

  • Define what to count

    Specify what you will count and where the values come from.

  • Try out your scoring rule

    Try the rule on a handful of answers and refine it.

  • Data access and feasibility

    Check which data you can obtain and are permitted to use in time.

  • Justify your choice of method

    Explain why your chosen method is the best fit for your question.

Your question fits the material

  • You know what you will compare.
  • Both runs receive the same material.
  • You decided how to assess answers before collecting them.

Sources and further guidance

Signature of Dr Markus Groß

We use cookies

We use cookies on our website. Some of them are essential for the operation of the site, while others help us to improve this site and the user experience (tracking cookies). You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.

CookieHint and Consent by reDim GmbH (Opens in a new window)