Overview
This is an hourly, remote contractor role for an Attorney / Lawyer Subject Matter Expert (SME). Work focuses on reviewing AI-generated legal responses and/or creating expert legal content. You will evaluate reasoning quality and step-by-step issue analysis, provide precise written feedback, and rate/compare multiple responses based on correctness and reasoning quality.
Please note: there is no immediate project for this role. If qualified, you will be among the first experts contacted when relevant opportunities arise, and you will have access to future projects through the organization’s expert network.
Responsibilities
- Develop AI Training Content: Create detailed prompts and gold-standard legal outputs, such as issue analyses, drafting examples, checklists, policies, and memos.
- Optimize AI Performance: Evaluate and rank AI responses to improve correctness, clarity, defensibility, and risk awareness.
- Ensure Model Integrity: Test AI outputs for inaccuracies, misleading certainty, confidentiality risks, and bias; validate reliability across legal scenarios.
- Quality Review & Feedback: Assess solutions for accuracy, clarity, completeness, and adherence to the prompt.
- Error Identification: Identify errors in legal reasoning, rule application, procedural steps, and risk/assumption handling.
- Fact-Checking & Citation Verification: Fact-check legal concepts and, when required, citations.
- Produce Explanations & Model Answers: Write high-quality explanations and model answers demonstrating correct legal analysis and clear communication.
Requirements
- JD/LLB (or equivalent law degree) and active bar membership (or eligibility) in at least one jurisdiction.
- 5+ years of professional experience practicing law (law firm, in-house, government, or legal aid).
- Strong ability to perform issue spotting and structured legal analysis (rules → application → conclusion), including identifying assumptions and uncertainty.
- Demonstrated experience in one or more practice areas, such as:
- commercial/contracts
- employment
- privacy
- corporate
- litigation
- IP
- real estate
- compliance
- Strong research skills, including locating and synthesizing primary authority (statutes, regulations, cases) and accurately characterizing holdings.
- Excellent drafting/editing skills for materials such as contracts/clauses, memos, letters, policies, and pleadings (as applicable), with attention to precision and tone.
- Exceptional attention to detail when fact-checking and identifying overconfident claims, missing exceptions, jurisdictional pitfalls, or procedural risks; minimum C1 English proficiency.
- Comfortable evaluating answers for internal consistency, client safety, and appropriateness to context (e.g., consumer vs business, HR vs litigation).
- Reliable, self-directed approach with consistent quality in an hourly, remote contractor workflow across time zones.
Preferred Qualifications
- Prior experience with AI data training/annotation, legal QA, editorial review, or knowledge-management work.
Compensation & Benefits
- Hourly contractor role. (Rate details not provided in the description.)
Location