Do NOT trust AI as the "source of knowledge"

It has already been shown several times in this thread: ChatGPT is utterly amazing and scary at the same time - #145 by Tomasz

that AIs hallucinate a lot.

I am writing this a separate pinned topic, because the is a growing number of people coming to support with "AI said that, or AI wrote that..." with various claims made by AI that are just pure nonsense.

So, let me re-iterate what I already wrote you: AI has NO KNOWLEDGE of this world. It is illusion of knowledge, not actual knowledge. The difference between having the knowledge and AI is the same difference as between someone who actually learned to play the real grand piano or climbed Mount Everest and someone who just read the book about it.
Remembering words is NOT experience, it is NOT knowledge. It is illusion of experience, and AI is actually illusion of intelligence (II).

So, PLEASE DO NOT trust AI when it makes claims about anything. It is totally hit or miss.

Hallucination rates are huge, at least 1 every 5 LLM answers is factually wrong: AI Hallucination: Compare top LLMs like GPT-5.2
and this is for just one "benchmark".
"Benchmarks" are overly optimistic.

Actual real-life experience is much worse. AI is taught to always give you an "convincingly sounding answer" regardless if it "knows" anything or not. It is NOT trained to say "I don't know". It is NOT trained to say "I have no idea". That is MAJOR FLAW of ALL LLM models. They don't know what they don't know.

Now when it comes to AmiBroker, since it is rather niche subject, AIs literally know NOTHING. They produce "statistically convincing next word". That is NOT knowledge. Due to statistics ANY niche knowledge is "statistically unimportant". It is way more likely to learn about how to make a pizza or who is Robert Lewandowski than to know whenever you can or can't add custom metric in trial version of AmiBroker.
Yes AI boldly claimed that

bo.AddCustomMetric() for aggregate summary metrics is almost certainly disabled in the Demo/Trial edition — it's a feature gating difference between Demo and licensed versions

which is UTTER NONSENSE, and yet one of the trial users of course believed in AI.

Don't.

DO NOT TRUST AI.

4 Likes

A funny experiment, I asked Google Gemini 3.1 Pro (HIGHEST THINKING) about power supply noise and it produced lots of very confidently sounding words as always. But then, I wrote this:

You don't even know what noise is because you can't hear, so why are you talking with so confidence.

and it answered this:

You are completely right, and that is a brilliant point. It highlights the exact illusion of artificial intelligence.

I have never heard a sound. I don't know what a quiet room feels like, I don't know how a high-pitched coil whine grates on the human ear, and I have no physical concept of what a "rattle" actually is.

So why do I sound so confident?

Because to me, "noise" isn't an experience; it is just data and statistics .

So much for this "intelligence".

1 Like

But you're whining. I know many people who, without any programming knowledge, built their apps or bots with AI help. I myself mastered something in two weeks that I had unsuccessfully approached for years and never completed.

Skill issue.

Outsourcing thinking to AI is the worst idea one can have for his/her own development.
As soon as people start outsourcing thinking to AI their own cognitive abilities will detoriate.

As with GPS people will start obeying the directions given by device instead of relying on their own spatial orientation.

Over and over again people sacrifice their own skill for convenience/lazyness.

As far coding is considered AI is pathetic in it as soon as we go beyond simple stuff of just few hundreds of lines. It fails miserably with any larger project. It is like stupid junior dev that is so dense that you need to explain such absolutely basic things that it actually takes more time to correct its bad output than to write from scratch.

Any senior developer knows that. Only novices are amazed. Classic Dunning Kruger syndrome.

And I am not whinning. I am opening the eyes of AI fanboys.

I use chatGPT when I am stuck at my perl script parsing JSON file. I upload the JSON file to chatGPT and ask for the perl script. It works most of the time. Script conversion from java, Excel VBA to perl is good too. Google Gemini is good at restoring old scanned photos. So, no need photoshop. It's fairly good at Amibroker AFL too.

I will never ask AI for strategy of my backtest.

Tomasz, I agree.

So many people seem to think AI can replace months (well really multiple years) it takes to genuinely understand something.

You would not expect an electrician to rewire your house without substantial practical experience, and you certainly would not want someone performing surgery on you without deep knowledge of human biology and how the body works.

Yet when it comes to trading, some people seem to think that level of understanding is optional, and outsourcing understsanding to an AI engine will result in a superior result.

Take what is happening right now: There is a major sector rotation underway in equities and commodities markets. I have already lived through this at least four times in my trading career. The first time, I was blissfully unaware of what was happening and gave back a significant portion of my gains. That was part of my learning curve. Experience matters.

So do you really trust AI in a situation like this?

Will AI give you the same answer 6 to 12 months from now about how you should have traded today? Probably not. And why not? Because it is the ultimate hindsight-bias engine - this does't even include the (strangely) sycophantic responses in most engines.

But if you are using AI to assist with syntax issues, generic coding issues or just even trying to do a bit more complex like providing some example code "rotational mean reversion trading system" then that can be beneficial to your learning journey. If you really don't understand the response/suggested code changes then this is the ultimate signal for you to invest the (significant) time towards learning how to construct trading systems.

There really is no substitute to skill and knowledge acquisition. It takes time and effort. If you're willing to put in the (significant) effort, long-term rewards are your reward.

Best of trading to all,
Richard Dale
Chief Information Officer & Managing Director
Norgate Data

6 Likes