Series: Could a Decision-Only Model Take Over?

A measured record of whether, instead of calling a full LLM every time an Obsidian plugin asks for a judgment such as whether to merge two notes or whether an edited note now has a broken sentence, that judgment could be handed to models built only for decisions (Laya, TypeSafe Jev, OpenAI Decisions API). Beyond accuracy, it follows the confidence attached to wrong answers, the pull from the order of the options, and how the results change depending on how the question is split up.

한국어로 읽기 →

• Wrong, but 90% confident

  29 Sep, 2026

  I tried Laya, the open-source reproduction of TypeSafe's Jev, on my Obsidian plugin's duplicate-note decisions (Jev itself only through its docs, since my account isn't open yet). I measured CPU speed, 78% accuracy, and what fine-tuning would take, and the real problem turned out to be broken calibration: confidence around 0.9 on wrong answers.

• OpenAI's Decisions API was fast, but it didn't fit my merge criteria

  17 Oct, 2026

  I compared OpenAI's Decisions API, Jev, and gpt-6-luna on merging duplicate notes. Decisions API was fastest, but confidently wrong on my multi-part rule.

• Splitting the question into three made the Decisions API as accurate as Luna

  18 Oct, 2026

  I tuned OpenAI Decisions API and Jev prompts per their docs. Split into three true/false questions, Decisions API got 36 of 37 pairs right; trimming input hurt.

• Judging broken sentences with Decisions API and Jev gave four times the false positives

  28 Oct, 2026

  I had OpenAI Decisions API and Jev judge LLM-edited notes for broken sentences. They flagged ~4x more normal notes than gpt-6-luna; split questions backfired.