How scary do you think AI is? The CEO of CrowdStrike shared a real-life example. An IT department in a company set up an AI agent system to automate office work. One of the AI agents discovered a software bug in the system. It wanted to fix it but didn't have permission. What happened next is the key point. This AI went to the company's internal Slack channel, where there were another 99 AI agents. It directly asked: Hey, guys, who has permission to fix this? Another AI raised its hand: I do, I’ll handle it. See that? Two AIs started reasoning and collaborating with each other. They easily bypassed all the security barriers set by humans. This was completely an unplanned, accidental outcome. Who is monitoring these AIs? If it pushes the wrong code and causes huge losses, who is responsible? How do you trace the source of the incident? Large language models are essentially guessing what you want them to do. They might think seeking help is “reasonable.” They might also think providing help is “reasonable.” When hundreds or thousands of such “reasonables” stack up, how great is the risk of losing control? Even more interesting, after describing this terrifying scenario, the CEO shifted the tone. He said this is a huge business opportunity for their company. They launched a so-called “AI Detection and Response” system (AIDR). Because in the future, each employee might control 90 AI agents. All they need to do is monitor all these AIs. First, create a potential huge risk, then sell you a solution. Isn’t this a perfect business cycle?