Director of AI Operations & Governance (R5464)
Remote
Enterprise Operations Division – Enterprise Operations / Full Time Employee / Remote
apply for this job
Job Description:
Shield AI is seeking a Director of AI Operations & Governance to operationalize and govern our workplace AI ecosystem across all AI initiatives. Reporting to the VP of Workplace AI, this role will own license and platform operations, AI governance, security posture, and ongoing lifecycle management of AI tools that support Shield AI's business functions. The role will be the central owner of "post–dev-ops" for AI, ensuring systems are reliable, compliant, secure, and continuously improving in line with production usage and business needs, while building and leading the team responsible for AI sustainment.
This role requires significant hands-on technical capability across the machine learning and generative AI lifecycle — not just program oversight. The Director must be able to credibly evaluate, tune, and troubleshoot models and AI systems at a technical level in order to govern them effectively, partner with engineering, and make sound tradeoffs between reliability, performance, cost, and risk.
What you'll do:
Technical AI/ML Ownership
- Evaluate, benchmark, fine-tune, and adjust configurations of ML and generative AI models (including LLMs) in production, applying working knowledge of model training, tuning, and evaluation methodologies rather than relying solely on vendor documentation.
- Apply applied AI/ML research and emerging techniques to inform build-vs-buy decisions, model selection, and architecture choices across the AI portfolio.
- Partner directly with data science and ML engineering teams on model performance issues, drift detection, and retraining or reconfiguration needs, contributing technical judgment rather than acting purely as a liaison.
- Maintain technical fluency in prompt engineering, retrieval-augmented generation, agentic/orchestration frameworks, and the practical distinctions between generative AI and traditional ML systems, and translate those distinctions into governance and staffing decisions.
AI Sustainment & Governance
- Own AI sustainment and governance for workplace AI tools across enablement, bought solutions, and custom builds, acting as the central "run" function for the AI strategy, and building the team and processes to scale it.
- Manage all AI-related licenses and entitlements: monitor usage, optimize allocations, drive reallocation, and partner with Finance for cost visibility and optimization.
- Monitor production usage patterns and performance to recommend roadmap items, enhancements, and deprecations based on real-world dynamics in production.
- Own and triage support tickets for workplace AI tools and platforms, driving resolution across vendors, internal engineering, and security partners.
- Lead continuous evaluations of AI tools and models, including monitoring drift, benchmarking performance, and ensuring tools remain effective and aligned with business KPIs.
- Maintain the updates and security outlook for AI platforms, coordinating patches, version upgrades, vulnerability remediation, and compliance with Shield AI security policies.
- Orchestrate model swaps and configuration changes in production, including rollout planning, risk assessment, change control, and post-deployment monitoring.
- Design, maintain, and govern shared prompt libraries, including standards for prompt quality, reuse, versioning, and training for end-users and builders.
- Own management of secrets (API keys, credentials, tokens) used by AI tools and orchestrators, ensuring secure storage, rotation, and access control in partnership with Security and IT.
- Define and maintain connectors and extensions (e.g., integrations into SaaS systems, data sources, and workflow tools) to ensure reliable, secure data access for AI workflows.
- Establish and operate auditability frameworks for AI tools, including logging, traceability of AI-assisted actions, and reporting for compliance and risk management.
- Lead AI governance practices for workplace AI (policies, guardrails, usage standards, approval workflows, exception processes) in partnership with Security, Legal, and HR.
- Partner with business solution and build teams to ensure their deliverables meet sustainment, observability, and governance requirements before moving to production.
- Define operational playbooks, SLAs, and incident response procedures for AI systems, including on-call patterns supported by contractors and platform specialists.
Leadership & Strategy
- Build, lead, and develop a team of AI operations professionals, contractors, and platform specialists, establishing career paths and scaling the function as the organization matures.
- Set the strategic direction for AI operations and governance, translating enterprise priorities into a multi-quarter roadmap and budget owned by this role.
- Provide regular status and risk updates to the VP of Workplace AI and other senior leadership, including adoption metrics, reliability indicators, governance findings, and cost trends.
Required qualifications:
- 15+ years in platform operations, ML/AI operations, DevOps, or SaaS sustainment roles, including significant experience in leadership/people management, with a track record of running production systems in a high-stakes environment (defense, aerospace, enterprise SaaS, or similar).
- Direct, hands-on experience developing, training, fine-tuning, or evaluating machine learning models or generative AI systems — this is a core requirement, not a nice-to-have. Candidates should be able to speak credibly to model architecture, training/tuning approaches, and evaluation methodology.
- Working knowledge of AI/ML research practices and the ability to apply current research to production decision-making.
- Software engineering or data science background sufficient to engage deeply with technical teams on model behavior, integration issues, and system design tradeoffs.
- Direct experience with AI platforms or orchestration tools (e.g., LLM providers, RPA/workflow tools like n8n, enterprise SaaS integrations) and their operational management, including at an organizational or strategic level.
- Demonstrated understanding of the distinct technical and operational challenges of generative AI versus traditional ML — including differing skill requirements, risk profiles, and market/salary dynamics — and ability to apply that distinction to team design and hiring.
- Strong background in governance, compliance, or security in the context of data-driven or AI systems, including familiarity with audit, logging, and access control best practices.
- Demonstrated ability to manage licenses and cost optimization for SaaS or AI tools at scale, including working with Finance and procurement stakeholders, and managing significant budgets.
- Hands-on experience with monitoring and observability stacks (logs, metrics, alerts) and using those signals to shape product roadmaps and operational improvements.
- Strong technical fluency across APIs, connectors, and integrations; able to work closely with engineering and vendors to design and maintain extensions.
- Proven track record building and leading high-performing teams, including hiring, mentoring, and developing talent across a mix of core staff and contractors.
- Excellent executive communication skills, with ability to translate production dynamics and risk into clear recommendations for senior business and technical leaders, including executive stakeholders.
- Experience operating in a hybrid environment of contractors and core team members, with the ability to define processes and standards that scale as the team matures.
$280,000 - $420,000 a year
#LF
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
apply for this job
