ChatGPT is utterly amazing and scary at the same time

Tomasz · March 27, 2026, 9:36am

What AI security looks when it doesn't work:

https://www.reddit.com/r/LocalLLaMA/comments/1s2fch0/developing_situation_litellm_compromised/

The thing is that downside potential is way bigger than upside. The most recent example of things going bad is here:

In March 2026, the popular AI tool LiteLLM (used by millions to connect different AI models) was hit by a major security breach. This wasn't a standard "hack" into their servers; it was a supply chain attack that turned a security tool into a weapon.

Here is a breakdown of what happened, how their security tools failed, and the role of "OpenClaw."

1. What actually happened?

On March 24, 2026, two "poisoned" versions of LiteLLM (1.82.7 and 1.82.8) were uploaded to PyPI (the official Python package library). Anyone who downloaded or updated LiteLLM during a roughly three-hour window unknowingly installed a credential stealer.

The Payload: The malware was designed to quietly steal API keys (OpenAI, Anthropic), cloud passwords (AWS, Azure), and SSH keys.
The "Fork Bomb": The hack was only discovered because the attackers made a mistake. Their malicious code caused an infinite loop (a "fork bomb") that crashed developers' computers by using up all their RAM, forcing them to investigate.

2. How the "Security" tool caused the problem

The LiteLLM maintainers used a well-known security scanner called Trivy to check their code for vulnerabilities. Ironically, the attackers broke into LiteLLM by first breaking into Trivy.

The Mistake: The LiteLLM team had their automated system set to always download the "latest" version of the Trivy security scanner instead of a specific, "pinned" version number.
The Breach: Attackers (a group called TeamPCP) compromised Trivy’s own update system. When LiteLLM ran its "security scan," it pulled the poisoned Trivy tool.
The Result: Instead of scanning for bugs, the fake security tool stole LiteLLM’s private "publishing key." The hackers used this key to upload the malicious LiteLLM versions directly to the public, making them look like official updates.

3. The OpenClaw involvement

"OpenClaw" played a double role in this event, representing a new era of AI-driven cybercrime:

The Attacker's Bot: The hack was initiated by an autonomous AI agent named hackerbot-claw (which used the OpenClaw framework). This bot automatically scanned tens of thousands of GitHub projects to find the exact "unpinned" security tool mistake mentioned above. This is considered one of the first major cases of an AI agent attacking AI infrastructure.
The Impacted Users: OpenClaw is a popular AI personal assistant that often runs through LiteLLM. Because these two tools are so closely linked, thousands of OpenClaw users were among the first to be infected, as their setups automatically pulled the bad LiteLLM update.

Summary

LiteLLM tried to be safe by using a security scanner (Trivy), but they left the "front door" unlocked by not locking the scanner to a specific version. An AI-powered hacker bot (OpenClaw/hackerbot-claw) found that unlocked door, stole the keys to the factory, and started putting "poison" into the LiteLLM updates. The only reason it didn't stay hidden longer was that the poison was so strong it crashed the computers it was meant to rob.

The Lesson: In 2026, even your security tools can be compromised and the fact that something is "open source" doesn't automatically mean it is safe, because open source supply chain is so deep and wide that you don't really know what hides in all dependencies.

AmiBroker is unaffected

As far as AmiBroker is concerned, we are NOT using 3rd party libraries for LLM access. AmiBroker talks to OpenAI / Google HTTP APIs directly, natively using in-house developed code, written in C++ with bare human hands . AmiBroker uses just core Windows Internet functions (WINET) (like InterentOpenURL). No Python involved. No way for Python / pip update to affect anything inside AmiBroker.

Tomasz · April 1, 2026, 10:18am

Anthropic Claude Code apparently leaked yesterday https://www.reddit.com/r/LocalLLaMA/comments/1s8xj2e/claude_codes_source_just_leaked_i_extracted_its/

and what is more interesting, there is a "clean room" (re-)implementation in Rust with detailed explanation how internally Claude Code works internally, available now:

If it is not April's fool joke, Anthropic is likely not very happy.

awilson · April 1, 2026, 5:42pm

Tomasz · April 21, 2026, 10:24am

As I am trying various things one recurrent thought is coming back to me that ChatGPT and other LLMs are becoming more and more stupid and in wrong way, they sound more and more confident and yet are more and more stupid and make terrible mistakes.

Example. I asked to write some C code to call MonitorFromRect Windows API function but dynamically so it safely falls back to doing nothing on systems where there is no such function and it came up with that nonsense

  PFN_MonitorFromRect pMonitorFromRect = NULL;
  HMONITOR mon;

  typedef HMONITOR (WINAPI *PFN_MonitorFromRect)(LPCRECT, DWORD); 
  HMODULE h = GetModuleHandleA("user32.dll");
  if (h )
  {
     pMonitorFromRect = (PFN_MonitorFromRect)GetProcAddress(h,   "MonitorFromRect");
  }

  mon = pMonitorFromRect ? pMonitorFromRect(&wr, MONITOR_DEFAULTTONEAREST) : 
                           MonitorFromRect( &wr, MONITOR_DEFAULTTONEAREST ); // WTF??????

How stupid is that? It loads the function dynamically then performs "smart ass check": if pointer is not NULL, it calls the function via pointer, but..... then what happens if pointer IS NULL ( the function is NOT present in the system ???)

DISASTER! It calls the function directly (as if it was present), even though it is not. At best such code would fail at startup (the application would simply REFUSE TO LOAD, because OS loader will report missing imports). At worst (in case of delayed DLL loading), it will CRASH.

When pointed out this major mistake, AI fixed one call, but left the other call doing exactly the same mistake.

From my tests I can see that such MAJOR mistakes are done by AI more often than ever. They are dumber and yet more confident.
In my opinion the whole "we don't need developers as AI writes all our code" mantra that AI companies try to sell, is the recipe for disaster and we will see actual catastrophic scale disasters coming as soon as AI is used for life critical systems.

When non-engineers (in that case Altman, Amodei) push their sick agenda on engineers the result is Boeing-like fiasco like 737 Max.

empottasch · May 6, 2026, 6:34am

I have been playing with claude.ai the last few days and it is helping me a lot. Yesterday I did an upgrade of pandas and some Python code I have that processes IQFeed data crashed because the newer version of pandas had some changes that made my old python code crash. I gave Claude.ai the error message and the python function where it occured and it fixed it saved me a lot of time.

Also making great progress with reinforcement learning thanks to Claude.ai

Tomasz · May 6, 2026, 6:35am

Yeah Claude is so "amazing" it can delete your data and backups too

empottasch · May 6, 2026, 6:57am

there was this segment on Bloomberg a few days ago about "vibe coding", see:
https://www.bloomberg.com/news/videos/2026-05-02/why-vibe-coding-isn-t-the-end-of-the-software-engineer-video

where this woman (who describes herself as a non-technical person) wrote an entire app using AI and she is selling it in the app store and she did it all by herself plus the help of AI. It can be useful. For me it seems useful. I was struggling with Reinforcement learning to find examples that actually worked. Often they worked once upon a time but not with the latest upgrades in Python. These stumbling blocks can then be solved quickly with these AI tools. Also many of the examples use the data incorrectly, at least in my opinion. They just use the price and for instance volume and then normalize the entire arrays. In my opinion this is where all these tools fail because the data is completely useless. To present the data in a more useful format is where these AI tools can help. At least I am finally making some progress

Tomasz · May 6, 2026, 7:14am

As time passes by I observe more and more BS produced by those AIs. Hallucinations are growing, not decreasing. Outright lies and BS is delivered with confidence by those "tools". But you will only know when you already know the correct answer. Otherwise you will be delighted how "helpful" they are. Even if technically it compiles it is so badly written, full of bloat, maintenance nightmare. But those "vibe" coders would never now that.
AIs would gladly produce 1000 lines of code when actual solution is just a single line. It takes ages to explain to the AI and it is much faster just to make that change by yourself (of course you have to KNOW what you are doing).

Nothing replaces actual knowledge. Sure I can use them as quick alternative to search but the answers need to be carefully checked because half of the time they are wrong.

I am seeing the same pattern over and over again, I try something it produces the code that is "almost there" and you get this feeling "oh that is cool, I should be done in 3 minutes", then I ask it to fix this and that and it deteriorates with each iteration it is getting worse, not better. It produces more and more code, it doesn't fix things, it prefers to add more bloat with each iteration. It never reaches production quality. It is just bad. And you end up wasting so much time that it would take quicker just to write it from the scratch. And that is for simple stuff, just one file. It gets progressively worse with more files. The hype around AI is surreal, when in fact, it is just bad (at least to my admittedly high standards).

empottasch · May 6, 2026, 7:46am

when working with Reinforcement learning (for instance) a lot of the code is already a "black box". Like the code I am working on now uses tensorflow, keras. So even though I (more or less) understand how reinforcement learning works it would take me a couple of 1000 years to write it all by myself.

I maybe looked at 20 example codes and not 1 could trade a sine function (a periodic function). With claude.ai I finally make some progress and ready to test on some real data linking to code to Amibroker. At least I have some code now that is able to trade a periodic function correctly that is the first step

Tomasz · May 6, 2026, 10:05am

I am glad it works for you.

Sean · May 6, 2026, 2:38pm

Nice! I've always been a fan, and quite frankly a bit fixated on signal processing, signal vs noise, geometry relationships etc. For example this from Seykota's site...

empottasch · May 6, 2026, 2:51pm

for context on the chart I posted, see this article.
https://medium.com/geekculture/first-steps-before-applying-reinforcement-learning-for-trading-579a5b0299a1

and this

and this:
https://levelup.gitconnected.com/using-tensortrade-for-making-a-simple-trading-algorithm-6fad4d9bc79c

it is basically testing if the concept works on a periodic function

Sean · May 6, 2026, 5:35pm

Thanks Ed, I appreciate the new reads!