Overview
The organization is seeking U.S. real estate attorneys to review existing AI evaluation tasks against real-world practice in their state. This work involves deciding whether the AI missed a real transaction issue, or whether the assignment, reference answer, or scoring rule needs correction.
The organization will confirm scope and hours when a matter is matched to the attorney’s state.
Responsibilities
- Read each assignment and its source documents, then write the reference answer you would expect from an associate.
- Compare your reference answer with the existing answer and explain any material differences.
- Check the governing jurisdiction and flag missing exhibits, conflicting facts, or assumptions about local closing practice.
- Compare AI-drafted work with the outcome you would accept from an associate.
- Identify the most consequential error and cite the agreement, exhibit, or applicable authority that supports your assessment.
- Review how each task is verified and apply the scoring rules to:
- your reference answer, and
- a sample AI answer.
- If a scoring rule gives credit for a material omission or rejects a defensible alternative, specify:
- which answer the rule misjudges, and
- how to fix the rule.
- Document your review so a second attorney can reproduce the score.
- Where a file leaves questions open, identify what additional fact would resolve the issue.
Example Assignment
- A mock purchase agreement, title commitment, and survey are the basis for a buyer’s closing memo.
- The AI flags a recorded easement but uses the wrong deadline to object.
- Determine what credit the memo should receive, cite the controlling provision, and revise the scoring rule so it catches the missed deadline.
Qualifications
- Active U.S. law license.
- Experience personally reviewing purchase agreements, title or survey matters, commercial leases, construction contracts, real estate financing documents, or closing packages in the state you identify.
- Ability to explain why an error would change the advice a client would receive.
Preferred Qualifications
- Prior AI experience is optional.
Compensation & Benefits
- Remote hourly contract (hourly rate and total hours to be confirmed when a matter is matched to your state).
Location
- Remote.
- Must be a U.S. real estate attorney working in the state you identify.