Overview
This remote, 1099, project-based role involves hands-on legal analytical work supporting Generative AI evaluation initiatives. As a Legal Domain Expert (early-career), you will apply practical training in legal research, citation analysis, and statutory interpretation to evaluate and help train next-generation legal LLMs.
Important disclaimer: This engagement involves hands-on legal analytical work for Generative AI evaluation. It does not constitute the practice of law, creates no attorney-client relationship, and involves no client representation or legal advice.
Responsibilities
- Citation & Authority Auditing: Verify primary and secondary authority citations produced by legal AI models to detect hallucinated case law, misquoted statutory provisions, and invalid citations.
- Prompt & Scenario Engineering: Draft realistic, multi-step legal scenarios grounded in primary source documents (statutes, regulations, court opinions, contracts, and filed pleadings).
- Rubric Development & Ground Truth Creation: Create objective evaluation rubrics and author publication-grade “golden” responses adhering to precise Bluebook/local citation and legal drafting standards.
- Failure Mode & Trajectory Analysis: Red-team model outputs and document failure modes, including secondary authority confusion, jurisdiction over-generalization, and errors in multi-document synthesis or deadline computations.
- Workflow Compliance: Complete onboarding calibration, perform self-directed quality assurance checks, and incorporate peer reviewer feedback into final submissions.
Requirements
- Education & Experience: Juris Doctor (JD), LLM, or bachelor’s degree in legal studies with 1–2 years of post-graduate experience in legal research, paralegal work, litigation support, or law firm practice.
- Core Competencies: Strong command of legal research tools (Westlaw, LexisNexis, Casetext), primary source document analysis, and rigorous legal citation checking (e.g., Bluebook).
- Jurisdictional Tracks: Priority interest in US Corporate, US Securities, US Labor, or commercial litigation.
- AI Familiarity: Experience or strong interest in Generative AI evaluation, prompt testing, RLHF, data annotation, or golden-answer authoring (a plus).
Compensation & Benefits
- Engagement Type: 1099 / Project-Based (Ongoing / Recurring pipeline following initial calibration pilot)
Location