Caveman mode: 75% token reduction, 3x latency drop
You try. You save token. Time. Money. You thank later. 75% token reduction, 3x latency drop, no loss of accuracy.
You try. You save token. Time. Money. You thank later. Joke aside, it's surprising how much faster this goes, in addition to saving time reading overly long sentences and paying more for the privilege. I didn't measure my output, but the numbers given by Julius Brussee feel plausible:
- 75% token reduction
- 3x latency drop
- no loss of accuracy
When they said AI would bring us back to the Stone Age, I didn't have this in mind 😆
Olivier Reuland