AI economy relies on opaque "tokens"
The "tokens" used by AI engines, small language units processed by models, are causing confusion for users struggling to understand their value and cost.
Three years into the AI boom, users are still grappling with understanding the true value of AI tokens. These tokens are fundamental units that AI systems use to process, reason, and communicate. They can represent individual characters, words, or parts of words.
For everyday consumers using services like ChatGPT or Claude, understanding what a token is and what it costs can be opaque, despite estimates of how many tokens a task consumes. Unexpectedly hitting limits can be frustrating.
Users frequently report encountering token limitations, which can affect the depth of conversations or cause interruptions during peak demand. While more complex tasks require more tokens, there is significant variation in pricing between companies, and personalized models can further influence consumption.
Different types of tokens exist, including input tokens for user data, output tokens for model responses, and cached tokens for reusing information. Research from Stanford University has shown that different models can consume up to 30 times more tokens for the same task. Furthermore, models tend to underestimate their own token usage, making it difficult to predict actual costs.
The opaque nature of token units reflects a broader concern that consumers are being asked to adopt AI technology without fully understanding its terms. While tokens are just one of many opaque currencies in today's economy, the confusion surrounding them highlights the need for clarity in AI usage pricing and limitations.