scoreComing soon

Content moderation

A marketplace checks new listings before they go live. Moderators can review only so many a day, so the binding limit is people’s time, not an error rate.

“How serious is the problem with this listing?”

The request · POST /v1/systemone

{
  "model": "jev-latest",
  "state": {
    "title": "…",
    "description": "…"
  },
  "questions": {
    "severity": {
      "type": "score",
      "instructions": "How serious is the problem with this listing?",
      "criteria": ["None", "Mild", "Serious", "Remove"]
    }
  }
}

The policy

noneACCEPTthe listing goes live
mild or seriousESCALATEthe moderator queue
removeBLOCKtaken down

target

The fewest errors with the review queue under [N] items a day.

Cut points are set on the cumulative probabilities of the levels.

When the truth arrives

nowLive, queued or taken down.proxy
hoursThe moderator’s decision.human_review
daysUser reports on listings that went live.downstream

In your app · Python SDK, gate() from M3

d = ts.gate("severity", answer=resp["answers"]["severity"],
            model_served=resp["model"], decision_id=ids[0])
if d.action == "ACCEPT":
    publish(listing)
elif d.action == "ESCALATE":
    moderator_queue.add(listing)
else:
    take_down(listing)

[ dashboard screenshot ] Queue size per day against errors across cut points, from the demo.

This use case is on the way. The roadmap shows where trimset is.

See the roadmap