Legal QA Lead - Remote

Overview

This is a remote, hourly contract role for a Legal Quality Assurance Lead. The position involves overseeing quality, consistency, and trainer performance across legal AI training projects. The lead will review AI-generated legal content and trainer/QA work, evaluate output quality against project guidelines, provide detailed written feedback, and ensure adherence to quality standards.

Responsibilities

• Monitor quality by spot-checking legal items, identifying issues, providing feedback, and escalating critical concerns.
• Review AI-generated legal explanations, research responses, contract analyses, policy interpretations, case summaries, compliance guidance, and issue-spotting workflows for accuracy, clarity, and appropriate caution.
• Communicate updates on item guidelines, project changes, workflow updates, quality expectations, and legal-review standards to trainers and QAs via Discord.
• Respond promptly and clearly to trainer/QA questions related to legal reasoning, jurisdiction, citations, source quality, disclaimers, contract language, compliance interpretation, and rubric application.
• Manage activation of trainers and QAs by encouraging participation, tracking follow-ups, and flagging availability issues.
• Create and maintain legal project documentation including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Schedule and conduct onboarding and training sessions to explain project expectations, workflows, rubrics, quality standards, and legal-specific review requirements.
• Ensure consistent application of legal-review guidelines among all trainers and QAs and communicate updates as projects evolve.
• Identify and flag unsafe, misleading, overconfident, or jurisdictionally inappropriate legal outputs, especially those resembling personalized legal advice.
• Propose workflow improvements and help develop scalable QA processes for legal AI training projects.

Requirements

• Bachelor’s degree, JD, LLB, LLM, or equivalent in Law, Legal Studies, Paralegal Studies, Compliance, Public Policy, or a closely related field.
• Strong command of English to follow guidelines, communicate with teams, and provide clear legal-review feedback.
• Minimum 3 years of professional experience in legal practice, legal research, contract review, compliance, regulatory analysis, paralegal work, legal operations, legal writing, legal education, or related workflows.
• Solid understanding of core legal concepts including legal reasoning, issue spotting, contract interpretation, legal research, statutory interpretation, case analysis, compliance, risk assessment, and legal writing standards.
• Ability to evaluate legal content against detailed rubrics and identify issues such as unsupported conclusions, hallucinated citations, jurisdictional mismatches, overconfident advice, missing caveats, incorrect standards, incomplete reasoning, or unclear explanations.
• Familiarity with legal workflows or tools such as contract review, legal memoranda, case summaries, regulatory guidance, citation checking, legal databases, document review, compliance checklists, or policy analysis is preferred.
• Experience leading or supporting remote teams of trainers, annotators, reviewers, lawyers, paralegals, legal writers, researchers, or QAs is strongly preferred.
• Comfortable working in fast-paced remote environments using tools like Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Highly detail-oriented and organized, capable of maintaining style guides, FAQs, trackers, onboarding materials, honeypots, calibration tasks, and other quality documentation.
• Experience with AI training, data annotation, large language models, prompt/response evaluation, legal content QA, or rubric-based LLM evaluation is a strong plus.

Compensation & Benefits

• This is an hourly, remote contractor position.
• There is no immediate project available; however, qualified candidates will be among the first contacted for relevant future opportunities.
• Access to future projects through the expert network will be provided.

Location

• Remote

Additional Information

This role requires strong legal expertise, excellent English communication skills, attention to detail, and the ability to manage quality workflows across remote legal-review teams. The work directly contributes to improving AI models by ensuring legal training data is accurate, well-reasoned, clearly explained, appropriately cautious, well-documented, and aligned with client expectations.