I worked with 20+ large and global enterprises involving AI first, enterprise business applications in the supply chain ecosystem. The following are token economic observations, behaviors, and takeaways.
1. Enterprise Token Budget:
- Current Belief: No historical precedent exists for token budgets within these non high technology enterprises. Functional organizations within an enterprise (manufacturing, procurement, merchandising, sales, marketing etc.) expect the IT/AI organization to pay for token usage because of historical budgeting precedence.
- Implementation Reality: The actual AI training and operational token costs for supply chain related materials and products are an eye opener for all executives across the board. Non IT/AI organizations have a long list of AI “must-haves” that are typically token expensive and do not consider token economics. No internal business ROI is baselined to warrant all of the business “must-haves” versus “nice-to-haves” when token costs enter the equation. Business “must haves” require ruthlessly prioritizing AI driven recommendations and actions into smaller and discrete outcomes. Ranking and prioritizing these discrete outcomes by ROI properly educates business decision makers on “must haves” versus “nice to haves”.
2. Subdividing Prioritized Business Must Haves:
- Current Belief: The top 1 or 2 critical business “must haves” with demonstrable ROI must include all data training for production. Training and operationalizing all data for large and global enterprises for the top 1 or 2 “must haves” many times involve tokens used for processing 25 plus petabytes of structured and unstructured data.
- Implementation Reality: Token training and operationalizing cost for even one product cohort “must have” can easily exceed 50X the total proposed monthly token budget for all combined products. Not all products have the same ROI. Not all product cohorts have the same ROI. Explore specific products or product cohorts, perform respective token ROI calculations, and use the outcomes to quickly eliminate products and cohorts with moderate to poor token ROI. Some products or product families do not warrant any token usage at all; the intrinsic product profit margin is too low to justify any token consumption. Present only top “must have” business use cases using discrete product token ROI for explainability.
3. Consider Traditional Methods To Substitute Away From GPU AI Token Consumption:
- Current Belief: AI first business applications and use cases must all use AI in order to hit the check box stating we are an AI first company or an emerging AI non technology enterprise. This belief is the top culprit driving negative token ROI in many business applications and business use cases.
- Implementation Reality: Many business decisions involve slow moving dimensions, such as KPIs and metrics. This class of slowing moving KPIs and metrics do not require GPU token computing in many business use cases yet are computational expensive. For slow moving dimensions, consider traditional CPUs and possibly minimize or eliminate token consumption altogether. Use Small Language Models and respective agent tokens to display these slow moving, CPU derived dimensions. A forward looking portfolio of CPU and GPU/token computing approaches can quickly turn use case ROI positive in many applications and use cases. Consider other slow moving dimensions and business use case calculations that do not require token based computing or that can use a portfolio of CPU and GPU/token computing.