Organizational Design for AI: From Functional Silos to Integrated Systems
Part 5 of the series "The AI Operating Model Gap"
The Agent That Couldn’t Cross the Hallway
A large consumer goods company deployed an AI agent in its marketing function to analyze campaign performance and recommend budget reallocation. The agent was good at its job. It identified underperforming channels, flagged creative fatigue, and surfaced optimization opportunities that the marketing team had missed. But the highest-value insight the agent could have produced, connecting marketing spend to supply chain capacity and inventory position, was inaccessible. The supply chain data lived in a different system, owned by a different function, governed by a different team. The agent could see half the picture. The org chart prevented it from seeing the other half.
This is the organizational design problem. The traditional functional organization, built around specialized departments with clear boundaries, is structurally incompatible with how AI creates value. AI's economic potential does not concentrate within functions. It concentrates at the intersections between them: customer operations that span service, sales, and product; marketing-to-sales handoffs where lead quality and conversion depend on shared data; product-to-engineering workflows where design decisions ripple into manufacturing constraints; finance-to-operations feedback loops where cost data informs operational decisions in near-real time.
McKinsey's research on generative AI's economic potential found that roughly three-quarters of the value concentrates in four cross-functional areas: customer operations, marketing and sales, software engineering, and R&D. These are not functions. They are workflows that cross functional boundaries. An organizational design that walls them off captures a fraction of the value.
The Silo Problem in 2026
The data on organizational barriers is consistent across surveys. PwC's 2026 Digital Trends in Operations Survey found that 83% of respondents say AI agents and automation will accelerate the breakdown of traditional functional silos, yet only 27% have fully embedded an AI strategy across business units. 58% of operations leaders report their departments still operate in isolated silos. 94% of those with siloed or partially integrated operating structures expect to shift toward a more horizontal, networked model, but only 41% operate that way today.
The agent deployment data tells the same story from a different angle. The average enterprise now runs 12 or more AI agents, according to Salesforce's 2026 Connectivity Benchmark survey of 1,050 IT leaders. Half of those agents operate in isolated silos without central coordination. 96% of IT leaders say agent success depends on seamless data integration. The technology is ready to work across boundaries. The organization is not.
The gap between ambition and readiness is striking. Celonis's 2026 survey found that 85% of organizations want to become "agentic enterprises" within three years, yet 76% admit their business processes are not ready. OutSystems reports that 96% of organizations are using AI agents in some capacity and 97% are exploring system-wide agentic AI strategies, but only 24% have deployed anything that qualifies as a true agent with genuine autonomy and cross-system reach, according to a ChapsVision/Sinequa survey. Only 10% have true agentic AI capabilities. The gap is not technological. It is organizational. And the consequences of leaving it unaddressed are becoming clearer: Gartner projects that more than 40% of agentic AI projects will be canceled by the end of 2027, largely because organizations will fail to build the structural and governance foundations those projects require.
The Center of Excellence Evolution
The AI Center of Excellence was the right organizational response in 2023 and 2024. When organizations were running early pilots and building foundational AI capabilities, a centralized team that could concentrate expertise, manage vendor relationships, and develop standards made sense. It was the starting structure for a technology that few people in the organization understood.
In 2026, the CoE is necessary but insufficient. The mandate has expanded beyond what a centralized team can deliver. A modern AI CoE is expected to handle strategy aligned to business goals, governance and standards for increasingly autonomous agents, capability development and AI literacy across the workforce, platform management and infrastructure, measurement of value, risk, adoption, and change management, and security and compliance for a rapidly expanding AI surface area. No centralized team can do all of this and simultaneously enable dozens of business units to build AI-native workflows at the pace the market demands.
The evolution path is from centralized team to hub-and-spoke enablement function. The hub sets enterprise-wide standards, provides shared platforms and infrastructure, maintains governance frameworks, and concentrates specialized expertise that individual business units cannot sustain on their own. The spokes are embedded AI teams or AI-skilled individuals within business units who understand the domain, own the use cases, and build workflows within the hub's guardrails.
This is not a new organizational concept. It mirrors how IT governance, security, and data management evolved from centralized control to federated models as those capabilities became enterprise-wide rather than specialist functions. AI is following the same trajectory, only faster.
The critical design question is what belongs in the hub versus the spokes. The hub should own what must be consistent across the enterprise: model evaluation and selection standards, security and compliance frameworks, data governance policies, vendor management, and the platform infrastructure that every business unit builds on. The spokes should own what requires domain expertise: use case identification, workflow design, agent configuration, performance measurement against business outcomes, and the organizational change management that adoption requires. The boundary between hub and spoke is not fixed. It moves as organizational maturity increases and as business units develop deeper AI capabilities of their own.
Three Organizational Models
Organizations scaling AI face a choice among three structural models, each with distinct trade-offs.
The Centralized Model concentrates all AI capability in a single team, typically reporting to the CTO, CIO, or Chief AI Officer. The centralized team owns model development, deployment, and governance. Business units submit requests and receive AI capabilities as a service.
The advantage is governance clarity. Standards are consistent, security is centralized, and the organization avoids the fragmentation that comes with distributed AI development. The disadvantage is speed. The centralized team becomes a bottleneck. Business units wait in queue. Use cases that require deep domain knowledge suffer because the centralized team lacks the context to build effective solutions. Grant Thornton's 2026 research on C-suite AI alignment found that AI conversations often default to the technology function, with implementation, vendor outreach, and strategy all flowing through the CIO first, regardless of where the business value sits.
The Federated Model distributes AI capability across business units. Each function builds and deploys its own AI solutions with minimal central coordination. The advantage is speed and domain relevance. Teams build what they need, informed by their own expertise. The disadvantage is fragmentation. Without central standards, organizations end up with inconsistent governance, duplicated infrastructure, incompatible tools, and the silo problem described above. Only 21% of organizations have a mature governance model for AI agents, according to Deloitte's 2026 State of AI in the Enterprise survey. In a federated model, the governance gap compounds as each business unit improvises its own approach.
The Hub-and-Spoke Model combines centralized governance with distributed execution. The hub provides standards, platforms, and specialized expertise. The spokes build and deploy AI solutions within their functions, following the hub's guardrails. The evidence increasingly favors this model for organizations scaling beyond pilots. Organizations with formalized AI CoEs operating in a hub-and-spoke structure report materially higher returns on AI investment than those with purely decentralized approaches. The structure gives business units the autonomy to move quickly while the hub prevents the fragmentation that makes cross-functional AI workflows impossible.
Organizational Models With AI
The hub-and-spoke model is also the most complex to manage. It requires clear delineation of responsibilities: what the hub decides versus what the spokes decide, how conflicts are resolved, and how the hub's standards evolve as business unit needs change. This is an organizational design challenge, not a technology challenge.
Cross-Functional AI Workflows
The organizational design question becomes concrete when you look at specific cross-functional workflows. Consider a customer churn prediction and intervention workflow. The data inputs come from customer service (interaction history, sentiment), sales (relationship health, contract status), product (usage patterns, feature adoption), and finance (payment history, revenue trajectory). The actions span marketing (targeted retention campaigns), sales (relationship outreach), product (feature recommendations or concessions), and customer success (escalation and recovery protocols).
No single function owns this workflow. In a siloed organization, each function runs its own analysis and takes its own actions, often without visibility into what the other functions are seeing or doing. The customer receives disconnected communications. The organization misses the integrated signal.
The same pattern applies across other high-value workflows. Supply chain optimization requires inputs from procurement, manufacturing, logistics, and sales forecasting. Product development benefits from integrating customer feedback (support), market signals (sales and marketing), engineering constraints, and financial analysis. Revenue operations spans marketing lead generation, sales pipeline management, customer success retention, and finance revenue recognition. In each case, the AI application that crosses functional boundaries is more valuable than the sum of the function-specific applications. But crossing those boundaries requires organizational permission, data access, governance coordination, and workflow redesign that the functional org chart is not designed to provide.
Harvard Business Review research by Ferreira and Tong, published in the September-October 2026 issue, addresses this directly. Their analysis argues that the most consequential decisions are not made within a single task. They emerge from linking together many narrowly scoped tasks across organizational boundaries. The authors propose a layered design for agentic AI orchestration: specialized agents that handle domain-specific tasks, connectors that verify results and pass data between agents, an orchestration layer that coordinates the overall workflow, and humans who retain context, set guardrails, and make final decisions.
This layered architecture maps directly to the three-layer orchestration model described in Part 2 of the "Orchestrating the Hybrid Workforce" series. The structural insight is the same: AI workflows that create the most value are inherently cross-functional, and the organizational design must enable them to cross boundaries without losing governance or accountability.
The Data Architecture Dependency
Organizational design for AI cannot be separated from data architecture. Every cross-functional AI workflow depends on the ability to access and integrate data across business units. If the data is siloed in functional systems without governed cross-functional access, the org chart redesign is irrelevant. The agents cannot reach the data they need to deliver value.
The Dual Maturity Framework, described in a separate Arion Research article, treats federated data access as a defining characteristic of organizational maturity. At lower maturity levels, data is locked in departmental systems. At higher maturity levels, governed data services make data available across the enterprise, with access controls, quality standards, and lineage tracking that enable cross-functional use without compromising security or compliance.
The Salesforce Connectivity Benchmark data underscores the point. 96% of IT leaders say agent success depends on seamless data integration. 35% cite siloed apps and data integration as a barrier to agent deployment, behind risk and compliance concerns (42%) and a lack of internal agent design expertise (41%). The data architecture is the infrastructure layer without which organizational redesign cannot deliver.
This dependency creates a sequencing challenge. Organizations can’t fully realize cross-functional AI workflows without both organizational and data architecture changes. But they can’t justify the investment in either without demonstrating value. The practical approach is to select one or two high-value cross-functional workflows, build the data integration and organizational coordination for those specific workflows, demonstrate value, and then use the results to justify broader restructuring. This is the pilot-then-scale approach that the Dual Maturity Framework recommends for organizations at Level 2 (Managed) maturity moving toward Level 3 (Systemic).
Governance as Organizational Infrastructure
The operating model series has a consistent thesis: governance is not a layer applied on top of the operating model. It is built into the operating model's structure. This principle applies directly to organizational design.
In a hub-and-spoke model, governance responsibilities distribute across three levels. At the enterprise level, the hub establishes standards for agent deployment, data access, decision authority, security, and compliance. These are the non-negotiable guardrails that apply everywhere. At the business unit level, the spokes develop function-specific policies that operate within the hub's framework. A financial services business unit may have stricter agent autonomy limits than a marketing business unit, reflecting different regulatory environments and risk profiles. At the workflow level, governance is embedded in individual human-agent workflows through the decision authority framework described in Part 3: which decisions agents can make, which require human review, and which are human-only.
This three-level governance structure is what the Arion Research framework calls proportional governance: different governance intensity for different autonomy levels and risk profiles. It’s an organizational design decision, not a compliance exercise. The governance structure is part of the organization's architecture, not an afterthought.
The governance challenge intensifies as agent autonomy increases. A simple AI assistant that summarizes documents needs minimal governance. An agent that can approve purchase orders, modify customer accounts, or reallocate budget requires governance that is specific, enforceable, and auditable. The decision authority framework from Part 3 of this series maps directly: Tier 1 decisions (agent acts freely) need automated monitoring and exception logging. Tier 2 decisions (AI recommends, human decides) need clear escalation paths and response time expectations. Tier 3 decisions (human only) need hard blocks that prevent agents from acting, not just guidelines that depend on the agent's own judgment.
Projects that deploy agents without embedded governance structures create risk that eventually forces the organization to pull them back. Building governance into the organizational design from the start is cheaper and more effective than retrofitting it after an incident.
The Mid-Market Advantage
One often-overlooked dimension of the organizational design challenge is scale. Large enterprises with deep functional silos, complex data architectures, and entrenched organizational politics face the hardest redesign challenge. Mid-market companies, discussed in the Arion Research "AI-Powered Mid-Market" series, often have a structural advantage. Fewer silos to navigate. Shorter decision paths. Greater willingness to reorganize around outcomes rather than functions. The organizational design challenge is real for mid-market companies, but the degree of difficulty is lower, and the speed of transformation is often faster.
The Competitive Implication
Organizational design is the structural dimension of the AI operating model. Without it, AI investments remain trapped in functional silos, delivering point solutions rather than enterprise value. The organizations that redesign their structures to enable cross-functional AI workflows, build hub-and-spoke governance that balances speed and control, and align their data architectures to support integrated workflows will capture a disproportionate share of the value that AI can create.
The organizations that leave their functional structures unchanged will find that their AI investments deliver incremental productivity within each function but fail to create the cross-functional value that drives competitive advantage. The operating model gap will persist, not because the technology is inadequate, but because the organizational design prevents the technology from reaching its potential.
The final article in this series, "The Operating Model as Competitive Differentiator," brings all 6 dimensions together and addresses the strategic implications for enterprise leaders who recognize that closing the operating model gap is not a technology initiative. It is an organizational transformation.
Strategy Playbook
1. Cross-Functional Value Map. Identify the top 5 cross-functional workflows where AI could create the most value. For each, map the current organizational boundaries it crosses, the data it requires from each function, the decision rights involved, and the governance mechanisms in place. The gap between the value potential and the current organizational barriers is the redesign target. Start with the workflow where the value is highest and the organizational barriers are most addressable.
2. CoE Evolution Assessment. Evaluate your current AI organizational model against the 3 models (centralized, federated, hub-and-spoke). If you are still centralized, define the spoke-enablement model: what standards, platforms, and governance the hub provides, and what autonomy and accountability the spokes have. If you are federated without strong central coordination, the priority is establishing the hub: enterprise-wide standards, shared platforms, and governance frameworks that prevent fragmentation while preserving business unit agility.
3. Data Architecture Alignment. For each priority cross-functional workflow, assess whether the data architecture supports it. Can agents in one function access the data they need from other functions? Are there governed data services that enable cross-functional access without compromising security? If data remains siloed in functional systems without governed cross-functional access, the organizational redesign will fail regardless of the org chart changes. The data architecture and the organizational design must advance together.
4. Governance Integration. Redesign governance as part of the organizational structure, not as an overlay. Define governance responsibilities at 3 levels: the hub (enterprise-wide standards, monitoring, and compliance), the spokes (business-unit-specific policies and implementation), and the workflow level (embedded governance in individual human-agent workflows, using the decision authority framework from Part 3 of this series). Proportional governance, where governance intensity matches the risk and autonomy level of each workflow, is the design principle.
Arion Research advises enterprise leaders on AI strategy and the shift to a digital workforce.
