Program Recovery: Turning Around a $2M AI Initiative

A mid-market healthcare technology company invested $2M in an AI initiative to automate prior authorization processing—a notoriously labor-intensive workflow causing patient care delays and operational costs.

Eighteen months into development, the program was in distress. Original six-month timeline had tripled. Budget was 40% over initial allocation with no production deployment in sight. The VP of Engineering who sponsored the initiative had left the company. Team morale was low, and stakeholder confidence had evaporated.

The new CTO called us in to assess whether the program should be salvaged or shut down.

We conducted a rapid two-week diagnostic to understand program status and root causes of failure:

Stakeholder Interviews

Interviews with 15 stakeholders (executives, product managers, engineers, operations staff, and clinicians) revealed fragmented understanding of program objectives and conflicting expectations:

• Engineering team believed they were building a general-purpose ML platform for future healthcare AI applications
• Operations team expected a turnkey automation solution to eliminate 80% of manual prior authorization work within three months
• Clinical stakeholders assumed the AI would handle routine cases while flagging complex scenarios for human review
• Compliance team was unaware of the initiative until recently and raised concerns about HIPAA implications

No single source of truth existed for what "done" looked like or how success would be measured.

Technical Review

Code review and architecture analysis uncovered significant technical issues:

• Model performance: Accuracy on test data was 73%, well below the 95% threshold required for production deployment
• Training data quality: Dataset included inconsistent labeling, missing fields, and unrepresentative samples (skewed toward simple cases)
• Technical debt: Code base lacked automated tests, documentation, or monitoring infrastructure
• Architecture scalability: System designed for batch processing, incompatible with real-time prior authorization workflows

Governance Review

Program governance was essentially absent:

• No formal decision-making authority after sponsor departure
• Informal, ad hoc requirements management (Slack messages and verbal requests)
• No risk assessment or mitigation planning
• Budget and timeline managed reactively with no forecasting

Root Cause Analysis

The assessment identified five primary root causes of program failure:

1. Ambiguous requirements: Program started with vague vision ("AI-powered prior authorization") that different stakeholders interpreted differently
2. Scope creep without governance: Absence of formal change control allowed scope to expand unchecked
3. Technical optimism: Team underestimated ML development complexity and overestimated initial model performance
4. Leadership gap: Original sponsor departure left power vacuum with no one accountable for program success
5. Stakeholder misalignment: Different groups pursued conflicting goals without forum for resolution

Based on diagnostic findings, we developed a phased recovery plan with three horizons:

Immediate Stabilization (Weeks 3-6)

Objective: Stop the bleeding and restore credibility.

Actions:

• Establish interim program governance: Executive sponsor (CTO), product owner (VP of Operations), technical lead (Senior ML Engineer)
• Freeze scope and halt new feature development
• Conduct requirements clarification workshop with all stakeholders to define MVP scope
• Quick wins: Fix three high-visibility bugs, improve model accuracy to 82% through better data cleaning

Course Correction (Weeks 7-18)

Objective: Rebuild on solid foundation with realistic expectations.

Actions:

• Redefine program objectives with measurable success criteria: Automate 50% of simple prior authorizations (defined as requests meeting specific clinical criteria) with 95% accuracy
• Restructure backlog into prioritized epics with clear acceptance criteria
• Address training data quality: Expand dataset, improve labeling consistency, balance simple/complex case distribution
• Refactor architecture for real-time processing
• Implement governance processes: Weekly sprint planning, bi-weekly stakeholder reviews, monthly executive updates
• Build testing and monitoring infrastructure

Sustainable Operations (Weeks 19-24)

Objective: Deploy MVP and establish long-term program success.

Actions:

• Pilot deployment processing 100 prior authorizations per day
• Continuous monitoring and model refinement based on production data
• Train operations staff on AI-assisted workflow
• Document lessons learned and update development practices
• Plan Phase 2 enhancements based on pilot learnings

Immediate Wins (Weeks 3-6)

Stabilization phase delivered visible improvements that rebuilt stakeholder confidence:

• Governance clarity: CTO assumed executive sponsorship, VP of Operations became product owner with decision-making authority
• Requirements alignment: Three-day workshop produced agreed-upon MVP scope and success criteria
• Quick technical wins: Model accuracy improved from 73% to 82% through better data preprocessing
• Team morale: Engineers appreciated clear direction and end of scope ambiguity

Course Correction (Weeks 7-18)

Fundamental rebuilding efforts gained traction:

• Model performance: Accuracy reached 96% on test data after dataset expansion and labeling improvements
• Architecture refactor: Real-time processing capability delivered in 8 weeks
• Technical debt reduction: Test coverage increased from 12% to 78%, documentation created for all major components
• Governance maturity: Regular sprint planning, stakeholder reviews, and executive updates became routine

Production Deployment (Weeks 19-24)

MVP deployed to production with measured success:

• Automation rate: 52% of simple prior authorizations fully automated (exceeded 50% target)
• Accuracy: 97.2% accuracy in production (above 95% threshold)
• Processing time: Automated cases processed in under 2 minutes vs. 45-minute manual average
• Cost savings: $180K annual savings from labor reduction
• Patient impact: Reduced prior authorization delays from 3.2 days to same-day for automated cases

1. Diagnose Before Prescribing

The two-week diagnostic investment was critical to understanding root causes, not just symptoms. Jumping directly to solutions without diagnosis would have wasted effort on wrong problems.

2. Re-establish Governance First

Technical fixes were useless without clear decision-making authority and stakeholder alignment. Governance stabilization was the foundation for everything else.

3. Quick Wins Build Credibility

Immediate stabilization phase delivered visible improvements that proved recovery was possible. Credibility enabled harder long-term work.

4. Right-Size Ambition

Original vision of general-purpose ML platform was aspirational but unrealistic. MVP focused on solving one concrete problem well, not everything poorly.

5. Address Technical Debt Early

Refusal to deploy untested, undocumented code to production—even under pressure—prevented future operational disasters. Technical debt paydown was prerequisite to sustainable operations.

Following successful MVP deployment, the organization continued AI prior authorization development:

• Phase 2: Expanded to moderate-complexity cases, automation rate increased to 68%
• Phase 3: Added explanation capability for denied authorizations, reducing appeals by 30%
• Broader impact: Governance practices and development standards from prior authorization recovery applied to three subsequent AI initiatives

Total program investment reached $2.8M (40% over original budget) but delivered annual savings of $450K and significant patient experience improvements. Most importantly, the organization built internal AI capability and governance maturity to sustain future initiatives.

Program recovery requires more than technical fixes. This case demonstrates the importance of:

• Comprehensive diagnostic to understand root causes
• Governance and stakeholder alignment before technical work
• Phased approach balancing quick wins with sustainable improvement
• Realistic scoping and measurable success criteria
• Commitment to quality over speed

Failing AI programs can be saved, but success requires honest assessment, stakeholder courage, and structured intervention—not heroic individual effort or technical silver bullets.