Legal Professionals to Train AI

Overview

This fully remote role involves evaluating and improving how frontier AI models handle real legal work. Legal professionals will design realistic legal tasks drawn from their own practice, run those tasks through frontier AI agents/models, and assess the outputs against a professional standard.

Work is long-form, self-directed, and emphasizes clear written reasoning.

Responsibilities

• Design challenging, realistic legal tasks based on day-to-day practice, such as:
  • Research memos on discrete legal questions
  • Contract redlines with accompanying issues lists
  • Sections of motions or briefs
  • Compliance gap analyses
  • Due diligence summaries
  • Discovery plans
  • Responses to regulatory inquiries
• Write task prompts as you would brief a trusted colleague, including context, constraints, and expectations.
• Author realistic supporting files needed to complete the task (e.g., agreements, pleadings, correspondence, internal policies, fact summaries, exhibits).
• Run tasks through frontier AI models and evaluate the produced deliverable against the standard you would apply to a colleague’s work.
• Compare two AI model outputs generated from identical prompts and files, determine which performed better, and document where each model fell short.
• Create detailed grading rubrics defining what a correct deliverable must include (e.g., issues spotted, authorities applied, contract provisions flagged, and appropriate tone/register).
• Explain in writing why an AI response passes or fails each rubric criterion.
• Flag concrete failures with supporting evidence, including:
  • Fabricated or misread authorities
  • Incorrect legal standard applied
  • Missed issues
  • Ignored provisions
  • Conclusions not supported by the facts
  • Misinterpretations of what the assignment asked for
• Review and evaluate AI-generated responses for accuracy, clarity, and professional appropriateness; annotate and rank outputs to guide AI training.
• Contribute across litigation, transactional, compliance/regulatory, and intellectual property work.
• Propose tasks for expert review, then build accepted tasks in full and iterate based on written reviewer feedback.
• Follow detailed written instructions precisely, including formatting and file upload requirements.
• Maintain consistent quality and throughput throughout the sprint engagement.

Requirements

• Demonstrated legal experience in your field (litigation, corporate/transactional, compliance, or similar); does not need to be currently practicing.
• Full professional or native level written and spoken English.
• Strong written communication skills, including the ability to explain complex professional reasoning clearly and concisely.
• Comfort with ambiguity and high attention to detail, including the ability to verify what documents claim against underlying numbers/sources/facts.
• Ability to interpret feedback, determine what is correct, and apply it without extensive hand holding.
• Ability to ramp quickly on unfamiliar work using written material and instructions alone, including when provided materials are incomplete.
• General familiarity with AI and LLM tools; ability to judge a well-reasoned answer versus a plausible but incorrect one.
• Baseline tech literacy, including comfort with:
  • Cloud file tools (e.g., Google Workspace)
  • Managing browser profiles
  • Downloading/installing desktop apps (e.g., Claude)
  • Everyday file handling (e.g., converting between Excel and Google Sheets, zipping files for sharing)
• Available at least 10 hours per week, with no weekly maximum; consistent availability is valued, and full-time hours are available.
• Based in the United States (see also Location eligibility below).

Preferred Qualifications

• 2+ years of hands-on experience in your field.
• J.D. and bar admission not required, though preferred.

Compensation & Benefits

• Pay: $40–$125/hr USD.
• Paid via PayPal on a regular cadence.
• Contract position; fully remote.
• Minimum 10 hours per week, with no weekly maximum.
• Flexible scheduling; multi-day task timers allow work to be spread across days.
• Access to Claude and ChatGPT is provided through the project; no personal subscription required.

Annualized pay range (full-time basis): $83,000 to $260,000 (calculated on a full-time basis). Contract pay is based on multiple factors including work experience, qualifications, credentials, location, market supply/demand, and candidate preference.

Location

• Based in the United States, Canada, or the UK.
• The qualifications section also states: Based in the United States.

Application & Process (What to Expect)

• Complete a skills assessment (about 2 to 4 hours, unpaid). It tests domain fit, careful reading, and how you judge a model’s output against a professional standard.
• Submissions must be your own work; submissions produced with AI tools are rejected.
• After review, onboarding includes a short training project walking through how tasks, files, and rubrics are built.
• You will propose a task based on your professional experience (scenario, prompt, files to build, and where you expect the model to fail). An expert reviewer provides written feedback.
• Each stage acts as a gate: work must be accepted before moving to the next step.
• Support is available through platform instructions, onboarding materials, a dedicated Slack channel, and office hours.

Confidentiality

The employer will not propose selected candidates to prospective clients without prior consent.