the health strategist
institute for
health transformation
and digital health strategy
Joaquim Cardoso MSc.
Chief Research and Strategy Officer (CRSO),
Chief Editor and Senior Advisor
September 4, 2023
What is the message?
The article underscores the rising costs associated with developing and maintaining generative AI systems, driven in part by the expensive CPUs and infrastructure required for training and operation.
While generative AI offers substantial economic opportunities and productivity gains, concerns arise regarding the scalability of these systems and the financial implications for businesses.
Microsoft’s Bing chatbot, for instance, reportedly required a total infrastructure investment of $4 billion to serve daily users, including expenses related to graphical processing units (GPUs).
The article suggests that despite the initial high costs, early investment in AI technology is crucial, but businesses must also consider long-term sustainability and environmental impacts.
One page summary:
The article titled “The Costs of Building Generative AI Platforms Are Racking Up” by David Howell explores the financial implications and challenges associated with the development and maintenance of generative artificial intelligence (AI) systems. In the context of businesses pursuing digital transformation projects post-COVID-19, generative AI has gained significant attention and investment.
One major cost factor highlighted in the article is the expense of central processing units (CPUs), specifically Nvidia A100 chips, which can cost as much as $10,000 each.
These chips power many large AI systems, such as ChatGPT, and the training of AI models often requires thousands of them. Additionally, there are broader compute architecture costs and expenses associated with training models to produce meaningful results for user queries.
Microsoft’s Bing chatbot, for instance, reportedly required a total infrastructure investment of $4 billion to serve daily users, including expenses related to graphical processing units (GPUs).
Enterprises looking to build their own AI models may not spend as much, but the costs can still add up, with some estimating it costs up to $0.15 per query to run a system like ChatGPT.
The article raises questions about whether enterprises fully understand the financial implications and potential environmental impacts of using generative AI.
While AI systems offer significant opportunities, such as contributing $15 trillion to the global economy by 2030 and a predicted global spending of over $300 billion on AI by 2026, concerns revolve around the scalability of these systems.
Generative AI models tend to grow in dataset size and compute power exponentially with each generation, leading to substantial costs.
OpenAI’s GPT-3, for example, reportedly cost $3 million to develop and train, while DeepMind’s AlphaGo required $35 million for training alone. This raises the challenge of balancing the need for larger models, increased data, and compute power with budget constraints and efficiency.
Despite the initial high costs, experts argue that investments in AI technology tend to pay off over time. They anticipate that as technology advances, development, training, and infrastructure costs will decrease. Cloud costs are expected to follow a similar pattern.
In conclusion, the article suggests that while the upfront costs of generative AI development and maintenance may be substantial, businesses that seize the opportunity early and build capacity for AI integration stand to benefit from this global transformation.
However, it also highlights concerns about the scalability and potential environmental impacts of AI, urging businesses to consider these factors as they navigate the AI landscape.
Ultimately, generative AI presents a significant opportunity, but its long-term costs and impact on the environment should not be underestimated.
DEEP DIVE
This summary was written based on the article “The costs of building generative AI platforms are racking up”, published by ITPro and written by David Howell, on August 24, 2023.
To read the full article, click here.
