Legal Quality Assurance Lead (QAL)

Overview

This hourly, remote contractor role involves serving as a Legal Quality Assurance Lead (QAL) to oversee quality, consistency, and trainer performance across legal AI training projects. The role requires reviewing AI-generated legal content and trainer/QA work, evaluating output quality against project guidelines, providing precise written feedback, and ensuring adherence to expected quality standards.

Responsibilities

• Monitor quality by spot-checking legal items, identifying issues, providing ongoing feedback, and escalating critical or recurring problems.
• Review AI-generated legal explanations, research responses, contract analyses, policy interpretations, case summaries, compliance guidance, and issue-spotting workflows for accuracy, clarity, and appropriate caution.
• Communicate updates to trainers and QAs regarding item guidelines, project changes, workflow updates, quality expectations, and legal-review standards via Discord.
• Respond promptly and clearly to trainer/QA questions related to legal reasoning, jurisdiction, citations, source quality, disclaimers, contract language, compliance interpretation, and rubric application.
• Manage activation of trainers and QAs by encouraging participation, tracking follow-ups, and flagging availability issues.
• Create and maintain legal project documentation including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Schedule and conduct onboarding and training sessions to explain project expectations, workflows, rubrics, quality standards, and legal-specific review requirements.
• Ensure consistent application of legal-review guidelines among all trainers and QAs and communicate updates as projects evolve.
• Identify and flag unsafe, misleading, overconfident, or jurisdictionally inappropriate legal outputs, especially those resembling personalized legal advice.
• Propose workflow improvements and help build scalable QA processes for legal AI training projects.

Requirements

• Bachelor’s degree, JD, LLB, LLM, or equivalent in Law, Legal Studies, Paralegal Studies, Compliance, Public Policy, or a closely related field.
• Strong command of English for following project guidelines, team communication, and providing clear legal-review feedback.
• Minimum of 3 years professional experience in legal practice, legal research, contract review, compliance, regulatory analysis, paralegal work, legal operations, legal writing, legal education, or related areas.
• Solid understanding of legal reasoning, issue spotting, contract interpretation, legal research, statutory interpretation, case analysis, compliance, risk assessment, and legal writing standards.
• Ability to evaluate legal content against detailed rubrics and identify issues such as unsupported conclusions, hallucinated citations, jurisdictional mismatches, overconfident advice, missing caveats, incorrect standards, incomplete reasoning, or unclear explanations.
• Familiarity with legal workflows or tools such as contract review, legal memoranda, case summaries, regulatory guidance, citation checking, legal databases, document review, compliance checklists, or policy analysis is preferred.
• Experience leading or supporting remote teams of trainers, annotators, reviewers, lawyers, paralegals, legal writers, researchers, or QAs is strongly preferred.
• Comfortable working in fast-paced remote environments using tools like Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Highly detail-oriented and organized, capable of maintaining style guides, FAQs, trackers, onboarding materials, honeypots, calibration tasks, and other quality documentation.
• Experience with AI training, data annotation, large language models, prompt/response evaluation, legal content QA, or rubric-based LLM evaluation is a strong plus.

Desired Skills and Experience

Legal Review, Trainer Feedback, Legal Reasoning, Contract Review, Legal QA, Compliance, Documentation, Legal Research, AI Training, LLM Evaluation

Additional Information

• The selection process includes an AI interview, a domain-specific task, and an interview with a recruiter.
• There is no immediate project available for this role; however, qualified candidates will be among the first contacted for relevant future opportunities and gain access to upcoming projects through the expert network.

Location

Remote (Contractor role)