Lead Engineer, AI Quality (Remote)
Check Your Resume Before You Apply
Upload your resume to compare your skills and keywords with this job's actual requirements.
✓ Skills That Match This Job
✗ Not Found In Your Resume
⚠️ Missing Important Resume Keywords?
Your resume may need to be optimised before you apply. Improve your resume with an ATS-friendly resume builder.
What Similar Candidates Looked Like
Illustrative only — replace with real hiring insights for this role if you have them.
Top Skills
Previously Worked As
Overview
Circle is an AI-powered, all-in-one platform designed for digital businesses, creators, coaches, educators, and communities. The platform enables users to manage discussions, courses, events, memberships, payments, live streams, and more under their own brand. Circle operates as a fully remote company with team members located across more than 30 countries.
About The Role
Circle is seeking a Lead Engineer, AI Quality to build and scale the infrastructure that measures, diagnoses, and improves the quality of AI-powered features across its platform. This role focuses on evaluation frameworks, observability tooling, AI diagnostics, and quality systems rather than machine learning research or model training. The successful candidate will lead the AI Quality Engineering team while remaining hands-on in technical implementation.
Responsibilities
- Build and maintain AI evaluation infrastructure and quality measurement systems.
- Design CI/CD pipelines, datasets, and scoring frameworks for AI agents.
- Diagnose AI quality issues across planning, execution, and tool-calling workflows.
- Develop annotation workflows and evaluation datasets.
- Run experiments across prompts, models, and agent architectures.
- Improve AI system performance, latency, and cost efficiency.
- Lead and mentor the AI Quality Engineering team.
Qualifications
Required / Minimum Qualifications
- 7+ years of experience building and shipping production software.
- Experience working with LLM-powered agents and complex AI systems.
- Strong knowledge of AI evaluation and observability infrastructure.
- Advanced proficiency in Figma and Adobe Creative Suite.
- Experience with datasets, annotation workflows, and evaluation pipelines.
- Proficiency in Ruby on Rails, Python, or the ability to learn quickly.
- Experience with frameworks such as Braintrust, LangSmith, or similar tools.
Additional / Preferred Qualifications
- Experience evaluating production AI agents at scale.
- Knowledge of open-source AI models and model benchmarking.
- Background in AI infrastructure, monitoring, and observability platforms.
⚠️ Work Authorization
WORK_AUTHORIZATION_TEXT: Not specified in the job listing — no sponsorship or country-eligibility requirement is stated.
🎯 Need Expert Help With Your Job Search?
From resume writing to LinkedIn profile optimization and interview coaching — get expert freelance help to strengthen your job application on Fiverr.
GET EXPERT HELP ON FIVERR →Ready to Apply?
Apply directly through the official Circle.so career page.
APPLY ON OFFICIAL EMPLOYER CAREER PAGE →Frequently Asked Questions
Yes, Circle operates as a fully remote company and this role is remote. "Remote (EMEA & APAC)
Ruby on Rails and Python experience is preferred.
Candidates should have 7+ years of experience building and shipping production software.
Building evaluation frameworks, observability tooling, and quality infrastructure for Circle's AI-powered products.
Europe, Middle East, Africa, and Asia-Pacific Remote (EMEA & APAC).
You May Also Like
Related Video
Share This Job
Joblyst2025 is an open-access job information and career resources website. Job details are sourced from publicly available employer career pages. Candidates should verify all job details, eligibility requirements and application status on the official employer website before applying.
Some links on this page may be affiliate links. If you purchase a product or service through an affiliate link, Joblyst2025 may receive a commission at no additional cost to you.
.jpg)
