Legal Domain Expert (Remote | Freelance)

Overview

This role involves leading the development and oversight of a Deep Research Agent (DRA) Benchmark focused on the legal domain. The benchmark evaluates advanced AI agents on complex professional legal tasks that standard models cannot solve but a genuine deep research agent can. The legal benchmark spans ten specialized domains, including Commercial Litigation, Corporate Law, Contract Review & Management, Regulatory Compliance, Intellectual Property, Consumer Protection, Employment & Labor Law, Environmental Law, Legal Research & Writing, and Policy & Legislative Analysis.

The Principal Domain Expert will serve as the senior legal authority responsible for the intellectual integrity of the benchmark, overseeing taxonomy design, SME screening, and quality control of submitted prompt packages.

Responsibilities

Taxonomy Finalisation

• Review and validate selected legal domains, documenting rationale for inclusion and exclusion.
• Define clear boundaries between overlapping legal domains, such as:
  • Commercial Litigation vs. Corporate Law
  • Regulatory Compliance vs. Consumer Protection
  • Employment & Labor Law vs. Regulatory Compliance
  • Legal Research & Writing vs. Policy & Legislative Analysis
  • Contract Review & Management vs. Intellectual Property
• Develop and maintain a cross-domain persona capability matrix indicating which legal personas can author or review tasks across domains (e.g., Commercial Litigation Counsel, Corporate Counsel, Contracts Manager, Compliance Officer, IP Attorney, Employment Law Specialist, Environmental Counsel, Policy Analyst).
• Validate and correct domain tags on all reference tasks to ensure consistent labeling.

AI Interviewer Design for SME Onboarding

• Design an AI-driven screening interview to evaluate incoming legal SMEs.
• Define competency dimensions to assess, including legal technical depth, statutory and regulatory interpretation, case-law analysis, ability to identify AI reasoning failures, verifier writing skills, and legal drafting and analytical reasoning.
• Author or review the screening question bank.
• Set pass/fail thresholds for each legal sub-domain and persona level.
• Determine criteria for secondary human review of borderline candidates.
• Test and calibrate the AI interviewer against expert legal judgment.

Task Complexity Framework Calibration

• Validate and refine the Numerical Rigor Level (L0-L5) framework.
• Finalize legal prompt-type definitions and cognitive trap frameworks.
• Define complexity indicators such as multi-jurisdictional analysis, conflicting precedents, regulatory ambiguity, statutory interpretation challenges, contractual conflict resolution, litigation strategy trade-offs, and policy impact analysis.
• Ensure complexity levels reflect professional legal practice rather than simple knowledge retrieval.

Quality Control (QC) of Submitted Prompt Packages

• Review prompt packages against standards covering Prompt, Data Files, Solution Logic, and Sanity Check.
• Verify solution logic step-by-step and independently validate legal conclusions.
• Confirm cited legal authorities, statutes, regulations, precedents, and contractual provisions.
• Validate prompt-specific verifiers to ensure they are binary-checkable, legally grounded, derived from the expected solution, and non-redundant with rubric criteria.
• Issue ACCEPT, REVISE, or REJECT verdicts with written justification.
• Maintain a QC log of recurring legal reasoning and task-design issues.
• Escalate benchmark calibration issues when tasks are systematically too easy or too difficult.

Required Qualifications

• Professional legal qualification (e.g., J.D., LL.B., B.L., or equivalent recognized law degree).
• Minimum 10 years of post-qualification legal experience in law firms, corporate legal departments, regulatory bodies, consulting firms, policy organizations, or litigation practice.
• Experience working with, supervising, reviewing, or evaluating legal work products across multiple jurisdictions, with comfort assessing legal reasoning and research from both United States and Indian legal contexts.
• Demonstrable expertise in at least three of the following domains:
  • Commercial Litigation
  • Corporate Law
  • Contract Review & Management
  • Regulatory Compliance
  • Intellectual Property
  • Consumer Protection
  • Employment & Labor Law
  • Environmental Law
  • Legal Research & Writing
  • Policy & Legislative Analysis
• Strong familiarity with statutory interpretation, regulatory frameworks, contract drafting and review, legal research methodologies, judicial precedent analysis, and legal writing standards.

Preferred Qualifications

• Professional governance, corporate secretarial, compliance, or corporate governance qualifications or experience.
• Prior experience in AI evaluation, benchmark design, RLHF annotation, legal-tech products, or legal knowledge systems.
• Experience designing interviews, assessments, or competency frameworks for legal professionals.
• Cross-sector experience in litigation, corporate advisory, compliance, technology law, public policy, or regulatory affairs.

Location

Not specified.

Compensation & Benefits

Not specified.