Local Voice Infrastructure Migration
Moving voice generation from cloud cost to local speed.
A migration from cloud-hosted voice models to local inference, keeping voice quality while improving speed, cost, and multilingual flexibility.
The Problem
The original cloud voice stack had strong quality but created cost, latency, and language constraints. It could output Chinese well, but could not flex into multilingual character performance with the same voice identity.
Role: Architecture Planner · Adapter Implementer · API Compatibility Owner · Rollout Lead
Cloud → Local
Cloud voice stack
Quality was good but locked to Chinese, with per-usage cost and network latency.
Local inference
Same voice identity across languages, faster and cheaper to run.
Contributions
What I actually did.
Planned the migration architecture from cloud to local runtime.
Adjusted interfaces and compatibility layers around the existing API contract.
Handled speaker mapping, latency constraints, quality checks, and rollout planning.
Owned the migration work outside the voice model training and behavior tuning itself.
AI collaboration governance
Picking the right model out of dozens of open-source ones wasn't me listening one by one on a hunch — it's that I designed "how to evaluate" into a pipeline the AI runs itself. This page is how I governed a large-scale model selection.
Governance cases
Open for first-hand evidence
The AI's ingrained pattern
Voice quality is inherently subjective: whether the emotional range is enough, whether the Taiwanese accent is authentic — auditioning one by one is slow and inconsistent, and across dozens of open-source models simply can't be compared reliably. Exactly where AI collaboration most easily degrades into "gut feel."
The governance mechanism I built
Orchestrated multiple sub-agents to build an automated evaluation pipeline, decomposing subjective audio judgment into cross-verifiable metrics and running a consistent comparison across dozens of models — not resting on my ears alone.
The collaboration value it unlocked
Made "preserving rich emotional range and tunability under severely limited local compute" an executable engineering process; the final model was what the pipeline surfaced, not a gut pick.
The AI's ingrained pattern
When swapping the underlying inference engine, AI tends to only care that "the new one runs," ignoring the pile of downstream dependencies on the old API contract, speaker mapping, and latency assumptions — one change and the whole chain blows up.
The governance mechanism I built
Constrained the migration around the existing API contract: added a compatibility layer, explicitly handled speaker mapping and latency limits, and planned a phased rollout, so the backend moved from cloud to local while staying invisible to the layers above.
The collaboration value it unlocked
Seven voice lines' cloud cost dropped from ~NT$90k/month to nearly zero, while the upper system didn't need a rewrite for this migration — cost cut without paying in stability.
Outcomes
Cost reduced
Local inference removed the cloud usage dependency for the voice runtime.
Latency improved
Moving generation closer to the runtime improved responsiveness for interactive character use.
Multilingual voice identity
The local model can preserve the same character voice across languages, expanding use cases beyond the original Chinese-only output.