Every company wants to win with AI. Yet most still struggle to move past small pilots. That is where an AI center of excellence comes in. It gives your teams one place to share skills, set rules, and scale what works. So let’s walk through how to build one that helps instead of slowing things down.
Why an AI Center of Excellence Matters Now
AI use keeps spreading. McKinsey’s latest global survey found that 88% of organizations now use AI in at least one business function, up from 78% a year earlier (McKinsey & Company, 2025). However, most of those companies are still stuck in pilot mode. Only 39% report any impact on enterprise earnings. In other words, adoption is wide but shallow.
Similarly, Forrester found that over 70% of firms have generative or predictive AI in production. Still, few of them measure its financial impact (Forrester, 2025). That gap is the problem a central hub solves. Without one, each department buys its own tools. Then nobody compares results. As a result, good ideas stay trapped in one corner of the business. A focused team changes that by connecting the dots.
Start an AI Center of Excellence With a Clear Mission
First, decide what your hub exists to do. Some teams focus on governance. Others focus on speed and delivery. Most need a bit of both. So write a short mission statement that leadership will sign. Keep it to two or three sentences.
Next, tie that mission to business goals. McKinsey research shows high performers don’t stop at efficiency. Instead, they also set growth and innovation as goals for their AI work (McKinsey & Company, 2025). Moreover, these leaders are nearly three times as likely to redesign their workflows. Therefore, your mission should push beyond cost-cutting. For example, you might aim to launch two revenue-driving use cases in the first year. That kind of target gives the team focus. It also gives executives a reason to keep funding it.
Pick the Right Structure for Your AI Center of Excellence
After that, choose a structure. There are three common shapes. A centralized model puts all AI talent in one group. It works well for smaller firms or early stages. A federated model places AI experts inside each business unit. Meanwhile, a hub-and-spoke model blends the two. A small core team sets standards, and embedded experts deliver projects.
Deloitte warns that many hubs fail because they sit apart from the business. In those cases, the vision never turns into results (Deloitte, n.d.). So whatever shape you choose, stay close to the people doing the work. Hub-and-spoke is often the safest bet. It keeps standards consistent. At the same time, it lets each unit move at its own pace.
Staff the Team With Mixed Skills
Now think about people. A strong hub needs more than data scientists. It also needs product managers, engineers, and security experts. Also include someone from legal or compliance. Change management skills matter too, because adoption is a human challenge.
Start small. Five to eight people is plenty at first. Then grow as demand rises. Also, give the team an executive sponsor with budget authority. McKinsey found that high performers have senior leaders who actively drive AI adoption and even role model its use (McKinsey & Company, 2025). That visible support makes a huge difference. Without it, the hub becomes an advisory group that teams can easily ignore.
Set Guardrails and Measure Results
Next, build simple guardrails. Create an approved tools list. Write a short policy on data use and privacy. Then set up a light review process for new use cases. Keep it fast, though. If approvals take months, teams will route around you.
Governance is quickly becoming standard practice. Flexera’s 2026 cloud research found that 47% of large enterprises have created dedicated AI governance teams or roles (IT Brief UK, 2026). Your hub can play that role.
Finally, track outcomes from day one. Measure time saved, revenue gained, and adoption rates. Share wins in short monthly updates. As a result, leaders see progress, and skeptics start to come around.
Keep It Evolving Over Time
Your hub should not stay frozen. As skills spread, its job shifts. Early on, it builds most solutions itself. Later, it trains others and steps back. Eventually, it may focus mostly on standards, shared platforms, and new bets like AI agents.
So review the charter every six months. Ask which services teams still need. Then drop the ones they no longer use. Above all, keep the focus on value. A well-run AI center of excellence turns scattered experiments into steady business results. And that is the whole point.
References
Deloitte. (n.d.). Is your AI center of excellence still a center of experimentation? https://www.Deloitte.com/content/dam/Deloitte/us/Documents/consulting/us-is-your-ai-center-of-excellence-still-center-of-experimentation.pdf
Forrester. (2025). The state of AI, 2025. https://www.forrester.com/report/the-state-of-ai-2025/RES189955
IT Brief UK. (2026). AI drives wasted cloud spend higher despite FinOps gains. https://itbrief.co.uk/story/ai-drives-wasted-cloud-spend-higher-despite-finops-gains
McKinsey & Company. (2025). The state of AI in 2025: Agents, innovation, and transformation. https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/the-state-of-ai-2025
