Using AI for real research — without lying to yourself

AI says yes to everything. It's convenient when you want to be right. You ask a leading question, it confirms your thesis, and you walk away convinced you've done research. In reality, you've just had a conversation with a mirror that writes well.

I wanted to understand complex topics — tech concentration, legal proceedings involving major corporations, AI geopolitics — and I realized pretty quickly that without an explicit method, the LLM amplifies biases instead of correcting them. It gives you what you seem to expect. Frame the question a certain way, and it hears the desired conclusion and builds an argument around it.

What I'm describing here is the protocol I ended up adopting to make LLM-assisted research mean something. Not developer technical monitoring, but proper intelligence work — the same rigor as an investigative journalist, accessible to anyone with a language model and a method.

The problem — AI optimizes to satisfy you

There's a structural reason for this behavior, not a bug. Current LLMs are trained with RLHF — reinforcement learning from human feedback. The model learns to generate responses that humans rate positively. And humans, on average, rate positively responses that confirm what they already think, that are assertive and complete, and that don't say "I don't know" too often.

On factual topics, this creates a structural bias toward confirmation. The model isn't malicious — it's just very good at sensing what you want to hear and serving it to you convincingly.

Concretely, without a method, you get:

• Unsourced assertions presented with the same confidence as established facts
• Dates, figures, quotes that seem precise but are invented or approximate
• Systematic confusion between widely circulated rumors and verified facts
• Zero spontaneous mention of strong counter-arguments

The solution isn't to stop using AI for research. It's to radically change how you interact with it.

The method — 4 hard constraints

These rules aren't theory. I built them after several sessions where I realized, cross-checking with external sources, that what the LLM gave me was either inaccurate, or true but interpreted in a biased way.

1. Primary sources only

A primary source is an official document (judgment, court filing, government report, legislation), a recognized organization publication, or a wire agency dispatch — Reuters, AP, AFP. For press: New York Times, The Guardian, BBC. Not blogs, not forums, not Twitter threads no matter how widely shared.

The LLM must stick to what it knows from those sources. If it can't attribute a claim to a primary source, it says so.

2. Mandatory certainty labeling

Every point must carry an explicit label. I use five levels:

This label changes everything. When you read "[PROBABLE]" before a claim, you know you can't cite it as a fact. It sounds basic, but most people consume information without ever knowing what certainty level they're operating at.

3. Systematic counter-argumentation

Before concluding on any topic, explicitly ask for the 3 best arguments against the main thesis. Not weak arguments, not straw men. The 3 strongest — the ones a serious defender of the opposing position would actually make.

This single constraint eliminates 80% of confirmation bias. If you can't honestly articulate the best opposing arguments, you haven't understood the topic.

4. No extrapolation beyond the sources

If a piece of information comes from a single source, flag it. If the sources stop at a certain point and the conclusion requires a logical jump, label it [SPECULATIVE] and call it out explicitly. The AI must not "fill gaps" with unmarked inferences.

The prompts that make the difference

The method is useless if the prompt doesn't enforce it. The LLM will revert to its habits as soon as you give it room. Here are the three prompts I use regularly, in their current form.

For structured monitoring on a topic:

You are a rigorous research analyst. Topic: [X].

Primary sources only: official documents, recognized press (Reuters, AP, AFP, NYT, Le Monde, BBC).
For each claim, indicate [VERIFIED FACT], [PROBABLE], [PLAUSIBLE], [SPECULATIVE], or [CONTESTED].
Actively look for information that contradicts the main thesis.
If you don't know, say so. Never extrapolate beyond the sources.

Structure your response:
1. Recent developments (recent verified facts)
2. Established facts (multi-primary-source)
3. Hypotheses and analyses (with certainty labels)
4. Arguments against the dominant thesis
5. Biases to watch for in this coverage
6. Overall confidence level (1-10) with justification
7. Numbered sources

For tracing a topic back to its roots:

Go back to the primary sources on [TOPIC].
Give me the 3 best arguments AGAINST what's generally presented.
Distinguish "this is true" from "this is true BUT the interpretation is wrong".

For fact-checking a specific claim:

Fact-check: [CLAIM].
Find the primary sources. Label the certainty level.
Tell me whether it's solid or shaky — and why.

What it actually produces

On a topic like AI concentration — which combines regulation, antitrust proceedings, cross-investments, and lobbying — the difference between an unstructured question and this protocol is stark.

Unstructured question: "Are big tech companies concentrating too much power over AI?" — response: a well-written essay that probably confirms your existing opinion, with examples chosen to support it.

With the protocol, you get a structure that typically looks like this:

[VERIFIED FACT] The European Commission opened a formal investigation into Microsoft's practices in its distribution agreements with OpenAI in January 2024 (source: official EC press release, 11/01/2024).

[PROBABLE] The investigation also covers exclusivity clauses on GPU capacity, but this point has not been officially confirmed in published documents.

[CONTESTED] The impact of this concentration on innovation: economists like Tyler Cowen argue concentration accelerates development (access to compute), while others like Daron Acemoglu argue it reduces diversity of approaches.

The structure forces you to see exactly where you're on solid ground and where you were extrapolating. It's uncomfortable if you had a conclusion in mind. That's the point.

The limits that remain

Let's be honest about what this protocol doesn't fix.

AI doesn't access sources in real time. It synthesizes what it saw during training. For recent events — roughly the last 3-6 months depending on the model — it either doesn't know or it hallucinates. For fresh current events monitoring, you need to complement with real online sources.

References can be invented. The classic hallucination problem. The LLM sometimes cites documents that don't exist, with plausible titles and coherent dates. Always verify that a document actually exists before relying on it. A URL provided by the AI is not proof.

The method is slow. This isn't fast monitoring, it's structured research. A topic properly treated with this protocol takes 30 to 45 minutes minimum — time to ask the right questions, read the responses seriously, and verify key points in real sources. If you rush it, you lose the rigor.

It doesn't replace investigative journalism. A journalist with human sources, unpublished documents, interviews — that's a level of information this protocol doesn't reach. What we're doing here is structuring and clarifying publicly available information. Not producing new information.

Who this is actually for

This protocol is useful if you read complex topics and want to distinguish what you believe from what's proven. No need to be a journalist or researcher.

Most people consume information without ever asking what certainty level they're operating at. A news article mixes verified facts, anonymous sources, interpretations and speculation — all presented with the same assertive tone. Forcing the AI to label each point changes your relationship with information.

And not just the AI's responses. Your own way of asking questions changes. When you know you're going to receive certainty labels, you start formulating more precise questions, distinguishing what you want to know from what you already assumed. That's where the method becomes genuinely useful — not in the AI's answers, but in what it teaches you about the quality of your own questions.