Market Data 
Box CEO Aaron Levie says he isn’t worried about how many tokens his company’s engineers are using.
“For me right now, I’m like, ‘Yeah, we should probably waste a lot of tokens because that means that we’re trying new things,'” Levie said during a recent episode of the “a16z Show.”
Levie’s views echo where much of Silicon Valley is right now when it comes agentic AI usage. Nvidia CEO Jensen Huang recently said he would be “deeply alarmed” if an engineer making $500,000 didn’t use the equivalent of $250,000 worth of tokens. Some companies, including Meta and OpenAI, are encouraging “Tokenmaxxing” by displaying leaderboards of the of the biggest users.
Leview has previously spoken about how companies need to realize that token usage won’t just be limited to their engineering teams.
Tokens are how large language models break down words into numerical inputs and outputs. AI providers, including OpenAI, charge based on the number of tokens used.
Engineers and companies are facing big questions over how to deploy AI agents, which by their very design run more sophisticated prompts and operate for a longer-period of time, driving up token usage. Levie said every engineer is grappling with this issue.
“You have to decide, do you want that to be a long-running prompt? Do you have to be a long-running agent?” he said. “Do you want to parallelize that? What is your comfort level of wasted tokens?”
Levie said these questions won’t be solved “until we can actually find a way to build data center capacity.” At that point, AI providers might charge less for tokens because they won’t be as worried about stretching their finite amount of compute too thin. Already, AI companies like Anthropic have implemented policies to try to grapple with peak usage.
In the meantime, agentic AI isn’t just about token usage, Levie said. He said CFOs and CIOs are “running around with their hair on fire trying to figure out” whether their companies’ current IT and integration policies work in an agentic AI age.
“How do you coordinate, you know, possibly the fact that you might be hitting this system like 10,000 times an hour or something, not from a performance standpoint,” he said, ‘but just how do you make sure that people didn’t move a file from one thing, accidentally, from one folder to another folder, while the other person was trying to do a write operation, and somebody else was trying to delete something because you just have these agents running wild.”
Read the full article here
