We forget that many of the controls for agents are akin to politely asking "please don't do anything stupid". LLMs are like a narcissistic, clumsy but sometimes surprisingly smart fresh grad with a history of harassment, drug abuse, and possible ties to North Korea. If asking politely is your only strategy, you're up for a wild ride.
Many controls for agents are just polite suggestions
We forget that many controls for agents are akin to politely asking 'please don't do anything stupid'.
Olivier Reuland