Your AI Workforce Will Help Run the Business. Who Will Control It?
Exploring this for your team? Start a CrmWare conversation →
When a tool becomes operating capacity
AI agents can use organisational knowledge, access business records, and perform approved actions through connected systems. This ability to act is why identity, authorisation, and interoperability are central to current agent standards work from the US National Institute of Standards and Technology (NIST).[1]
But an agent does not need to resemble a human employee to become part of a digital workforce. What matters is organisational reliance.
When employees, customers, or business processes begin depending on an agent's continued availability and behaviour, it is no longer merely an experimental tool. It has become part of the organisation's operating capacity.
Dependency develops quietly
An assistant begins by answering routine questions. It is connected to trusted company knowledge, then permitted to retrieve customer information, update records, initiate approved actions, or coordinate work across a process.
Employees adapt. Processes change. Customers begin expecting the capability to be available. Each step can be reasonable on its own. Together, they can create an operational dependency.
Dependency becomes strategically important when the organisation can no longer confidently supervise, change, or continue an AI-enabled process without the incumbent system or supplier.
Human oversight must remain meaningful
Research on human interaction with automation shows that people can over-rely on incorrect automated recommendations. They can also become less effective at recognising problems and intervening after moving from active work to passive supervision.
A 2025 systematic review found evidence of automation bias across human-AI decision environments, while emphasising that reliance varies with the task, risk, expertise, expectations, and system design.[2] Earlier human-factors research describes a related out-of-the-loop problem: passive supervision can affect situation awareness, vigilance, and intervention performance.[3]
Meaningful oversight therefore requires more than a final approval button. People need sufficient competence, information, authority, and practical opportunities to challenge or interrupt the system. The OECD AI Principles similarly connect human oversight with the ability to override, repair, or safely decommission systems when necessary.[4]
Responsibility does not disappear into the supply chain
AI governance is shared across providers, implementers, deployers, and users. Each has different responsibilities and different abilities to act.
Obtaining AI from a third party does not remove the organisation's responsibility for how it is authorised and used within its operations.
The NIST AI Risk Management Framework calls for internal policies addressing third-party AI risks, defined human-oversight processes, and contingency arrangements for failures involving higher-risk third-party systems.[5] The OECD's work on AI accountability similarly treats accountability as a lifecycle concern distributed across the AI supply chain.[6]
A supplier can provide controls, monitoring, and specialist expertise. But practical organisational control still requires internal authority, knowledge, and operating competence. Responsibility without those capabilities leaves the organisation in an unstable position.
The deeper risk is capability dependency
Conventional vendor lock-in makes software difficult or expensive to replace. Capability dependency goes further. Changing suppliers may require the organisation to reconstruct the knowledge, permissions, evaluation methods, integrations, and operating practices through which the work is performed.
The European Commission identifies weak interoperability, lengthy procedures, and the risk of losing data or application functionality as obstacles to changing data-processing providers. Its Data Act guidance promotes open interfaces and machine-readable exports to make switching more practical.[7]
Portability helps, but exporting data alone does not preserve an operating capability. The objective is not to eliminate every external dependency. Salesforce, AI models, infrastructure providers, and specialist partners can all deliver substantial value.
Rent the technology. Retain the capability.
Ownership does not require self-hosting every model or employing every technical specialist internally.
The organisation needs enough authority and competence to direct, evaluate, challenge, and evolve the capability, even when others provide the underlying technology.
Salesforce can be where the workforce operates without becoming the owner of the organisation's capability. CrmWare uses 'own your digital workforce' to describe this relationship: rent the technology where it makes sense, but retain control of what that technology enables.
What would remain yours?
- Can we decide what the AI worker may do?
- Can competent people challenge and override it?
- Can we reconstruct its important actions?
- Can we test, approve, and reverse changes?
- Could the capability survive a supplier or technology change?
If these answers are unclear, the organisation may use an AI workforce without yet controlling it. That gap should be addressed before further operational responsibility is delegated.
AI transformation should leave the organisation better able to direct, govern, and evolve its operations. If essential knowledge and control remain exclusively with a platform or consultancy, the project may have delivered automation without transferring capability.
References
[1] NIST. AI Agent Standards Initiative. https://www.nist.gov/artificial-intelligence/ai-agent-standards-initiative
[2] Springer Nature, AI & Society. Exploring automation bias in human-AI collaboration: a review and implications for explainable AI (2025). https://link.springer.com/article/10.1007/s00146-025-02422-7
[3] Kaber, D. B., and Endsley, M. R. Out-of-the-Loop Performance Problems and the Use of Intermediate Levels of Automation. https://maritimesafetyinnovationlab.org/wp-content/uploads/2020/08/Kaber-Endsley-Out-of-the-Loop-Performance-Problems-and-the-Use-of-Intermediate-Levels-of-Automation-for-Improved-Control-System-Functioning-and-Safety.pdf
[4] OECD. AI Principles. https://www.oecd.org/en/topics/sub-issues/ai-principles.html
[5] NIST. Artificial Intelligence Risk Management Framework 1.0. https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf
[6] OECD. Advancing Accountability in AI (2023). https://www.oecd.org/content/dam/oecd/en/publications/reports/2023/02/advancing-accountability-in-ai_753bf8c8/2448f04b-en.pdf
[7] European Commission. Data Act explained. https://digital-strategy.ec.europa.eu/en/factpages/data-act-explained