AI Operations Analyst

<h3>This is a remote position.</h3><div><b><b><b><b>AI Operations Analyst</b></b></b></b><div>Reporting to Manager, Production Support &amp; Service Reliability, monitors AI-driven workflows in production, reviews output-quality incidents, supports fallback and human-review decisions, and partners on trust-sensitive escalations. This role helps ensure AI-enabled workflows remain reliable, supportable, and safe in live finance-oriented environments.<h3><b>Key Responsibilities</b></h3><h3>AI Workflow Monitoring</h3><ul><li>Monitor AI-driven workflows in production for abnormal behavior, trust issues, degraded outputs, and recurring exception patterns.</li> <li>Review signals related to output quality, fallback use, low-confidence patterns, and workflow anomalies.</li> <li>Escalate issues that could affect live user trust, business quality, or finance-sensitive outcomes.</li> </ul><h3>Incident Investigation</h3><ul><li>Support investigation of output-quality incidents, orchestration failures, prompt or workflow regressions, and tool-call problems in production.</li> <li>Help distinguish true software defects from model behavior issues, data-quality problems, or process misunderstandings.</li> <li>Partner closely with Engineering, QA, and business stakeholders on judgment-heavy incidents.</li> </ul><h3>Operational Controls</h3><ul><li>Support human-review patterns, fallback routes, and safe-operating responses for AI-enabled features.</li> <li>Help define and apply production review practices for higher-risk workflows.</li> <li>Document evidence and issue patterns in ways that support repeatability and better release decisions.</li> </ul><h3>Continuous Improvement</h3><ul><li>Identify recurring AI operations issues and recommend improvements to supportability, monitoring, and quality feedback loops.</li> <li>Feed production learnings back into QA, Product, and Engineering.</li> <li>Help improve operational trust in AI-enabled solutions over time.</li> </ul><br><h3>Requirements</h3><h3><b>Required Qualifications</b><ul><li>4+ years of experience in production support, AI operations, business process operations, analytics, QA, or a similar role.</li> <li>Strong analytical skills and comfort evaluating nuanced workflow or output issues.</li> <li>Ability to work across engineering and business teams in ambiguous situations.</li> <li>Experience with AI-enabled workflows, operational analytics, business process validation, or support for workflow-heavy systems.</li> <li>Strong written communication and documentation discipline.</li> </ul></h3><h3>·Bachelor's degree preferred.</h3><h3><b>Key Traits for Success</b><ul><li>Thoughtful, precise, and calm under pressure.</li> <li>Strong at spotting subtle trust or quality issues others may miss.</li> <li>Comfortable balancing evidence, judgment, and business impact.</li> <li>Focused on safe operations and clear escalation.</li> </ul><br></h3><h3>Benefits</h3><p>Base salary plus performance-based bonus.</p><p>Actual compensation packages are determined by evaluating a wide array of factors unique to each candidate, including but not limited to skill set, years and depth of experience, education, certifications, cost of labor, and internal equity.</p><br></div></div><p>Originally posted on <a href="https://himalayas.app">Himalayas</a></p>