I used AI as a tax advisor.
Not literally, but I used it heavily while preparing yearly tax documents for stock exchange transactions.
Quick context: taxes in Poland are not always straightforward. If you only submit forms from a full-time job, it is simple. But once you invest through different brokers, instruments, and markets, things get messy quickly.
So I decided to use AI assistants during the entire process.
I did not treat responses as ground truth. AI was an extra tool to reason through decisions. Sometimes it helped a lot. Sometimes it made things worse.
Here is what I learned and how I would approach a similar task now.
1. Use AI with research capabilities
For anything related to regulations, taxes, or law, the model should be able to search the web. Static knowledge is not enough.
Even better if the system is agentic, meaning it can search, verify, and iterate on answers.
Without that, you’re basically guessing.
2. Context length matters more than you think
Too little context → the model misses important details.
Too much context → the model starts confusing concepts.
Tax terminology is especially tricky because many things sound similar but mean something completely different.
For example:
- PIT-36
- PIT-38
- different settlement rules for instruments
When everything sits in one long conversation, the model can start mixing them up.
3. Do not ask controversial follow-ups in the same long session
This was a surprising one.
When the conversation gets long and concepts are similar, clarifying questions can make the model more confused, especially after a questionable answer.
Better approach:
- start a fresh conversation
- paste only the necessary context
- ask the question again
- use search / research
4. Ask for sources and citations
A simple prompt that helped a lot: “Show the source of this interpretation.”
When you ask for citations, the model usually becomes more careful and grounded in actual regulations.
It also lets you verify things yourself.
5. Use multiple models
Never trust a single model with something important.
Use a small consortium of LLMs:
- ask the same question in multiple models
- compare reasoning
- look for inconsistencies
If two models disagree, that’s usually where the real learning starts.
6. Turn AI into an annoying auditor
Instead of asking AI for the answer immediately, first write down your full process.
Describe what you did step by step. Turn your reasoning into a clear checklist or mini tutorial.
Then give it to AI and ask it to analyze everything critically.
Its job:
- look for logical errors,
- find missing steps,
- point out questionable assumptions,
- highlight places where concepts might be mixed up,
- be annoying and without mercy.
This works surprisingly well.
You not only catch mistakes earlier, but you also end up with a repeatable process you can reuse next year.
7. Branching chats are powerful
Instead of one long conversation, create branches for different hypotheses and paths.
You keep the same core context without cluttering one thread.
8. Which LLM was the best?
Across all tools I tested, Grok was the best assistant for this specific job because of its strong research skills.
Did AI get everything right? No. There were moments when it confidently gave wrong answers, and I caught them only because I verified everything manually.
That is the key point: AI is a thinking partner, not a tax advisor.
Kamil Kwapisz