The Obsidian plugin I’m building has two features. One takes an article and has an LLM split the concepts, people, and companies in it into separate notes saved to a wiki (I call this ingest). The other scans the accumulated notes and suggests merging duplicates or splitting notes that have grown too large (refactor). This post is about a problem where notes the LLM created together from the same article (I’ll call them “sibling notes”) contained the same paragraph duplicated across them. I had already built a test setup that drives both features remotely inside a real Obsidian app. The next question was what to feed it. For the first few rounds, I wrote six fake notes by hand each time and ran ingest, then refactor. It was convenient, since I could design whatever structure I wanted. That was exactly the problem. Made-up notes only reproduced the bugs I had already imagined. The bugs real users would hit only show up in shapes I couldn’t predict. So I changed course: from then on, I only ingested news articles and blog posts I had actually clipped.
Fabricated notes had slipped into the “real notes” pool — synthetic test data got mixed in
Even after the switch, there was one accident. Each round, I picked sources by
digging through the commit history of the repository where I record test
results, looking for articles I hadn’t used yet. Right before deploying the six
I’d picked to the raw/ folder, I reopened their frontmatter and found that
three of them had URLs like source: "https://example.com/megabank-ai-loan-a". The company was “MegaBank”,
and the people had invented names like “Jayden Park” and “Lina Okafor”. These
were synthetic test fixtures I’d added weeks earlier, before I had the
“real notes only” rule. They had stayed in the commit history and got picked
up again as if they belonged to the pool of real articles. I filtered out the
three fakes and swapped in articles with real sources (Forbes, the Korean
daily JoongAng, and publy.co, a Korean business-content site) before
continuing the round.
Two sibling notes were copying the same paragraph — duplication in LLM-generated notes
A few rounds after fully switching to real articles, I noticed something odd in the result of ingesting an interview with a Google vice president. Opening the “AI science agent” note and the “co-scientist” note side by side, I found this paragraph in both notes’ “research cases” sections, identical down to the letter:
“A research team at Imperial College London fed an antibiotic-resistance problem it had worked on for ten years into co-scientist. In just two days, co-scientist arrived at a hypothesis on par with the existing one and proposed four additional research directions.”
Both notes were classification: concept, and the intent to cover different
angles was clear: one was the broader concept of an “AI science agent”, the
other a specific tool called “co-scientist”. Yet the research case each one
used as an example was neither summarized nor reworded. It was pasted whole.
When a single ingest pulled several concept notes out of one article, the
notes weren’t dividing the content between them. Each was refilling the full
content into its own note.
Splitting into 13 notes was actually clean — the problem was distance, not count
My first guess was “the notes are too fine-grained, so they overlap.” Another ingest result from the same round proved that guess wrong right away. One development guide had been split into 13 notes (Context Engineering, Critical Verification, Problem Decomposition, Vibe Coding, Code Overload, and so on), and each note covered a clearly separate scope, connected to the others only by cross-links. Another result that split a single HBR article into 15 notes was the same. The duplication kept appearing instead when an article was split into only 2 to 4 notes that were very close sub-concepts of each other (Dense Attention ↔ Quadratic Attention Bottleneck, MathWorks ↔ MATLAB MCP Server ↔ MATLAB Agentic Toolkit, AI science agent ↔ co-scientist ↔ Yossi Matias). With more notes, the model seemed to recognize each note’s narrow scope more clearly. Among a few closely related notes, its sense of “which note does this content belong in” seemed to waver.
Company and product notes also copied paragraphs whole instead of sharing them — an article on Uber adopting Claude Code
The next round showed this wasn’t limited to concept notes. I ingested a pair of English and Korean articles about Uber adopting Claude Code. The “Uber” entity note and the “Claude Code” entity note both repeated the same early adoption figures, with little more than the word order changed: engineer usage going from 32% to 84%, a rollout to about 5,000 people, 70% of commits coming from AI tools, and 11% of backend updates written without human involvement. Two entities that are inherently intertwined, a company and the product it uses, were each telling the same “adoption story” from the beginning.
The next cleanup caught other duplicates, but not this one — embedding similarity duplicate detection
A note-split run in the same round made a similar mistake. It failed to recognize “tacit knowledge sharing”, “last line of defense”, and “taking responsibility”, which already existed as standalone notes from the original ingest, and created new notes with nearly the same content under different file names, such as “tacit knowledge sharing for team leads in the AI era”. But when I ran refactor once more right after, the non-title duplicate detection, which catches duplicates through embedding similarity even when file names are completely different, correctly flagged all three pairs as “auto-detected: merge duplicate notes” candidates. They got past the step that only looks for notes with the same title, but not past the embedding similarity check. The root cause (the split run not properly seeing the existing note list) still wasn’t fixed, but confirming that such duplicates wouldn’t stay in the vault permanently was a win.
That detection, though, asks whether two notes are the same concept as a whole. When the same paragraph is embedded inside two notes that cover different concepts, like “AI science agent” and “co-scientist”, the notes really are different at the note level, so there is nothing for it to catch. That is why the copied paragraph above slipped past every check. The problem was only fixed several rounds later, after I added a post-processing step: gather all the sibling notes ingest created from one article, find the specific facts they repeat, keep each fact in one note, and rewrite the others to link to it. That process is covered in the posts that follow.
Lingering thoughts
When testing with made-up notes, I could design the case where one article yields several concept notes. I never imagined those notes might copy the same paragraph from each other. Only after switching to real articles did this kind of bug become visible, and even then only by reading sibling notes side by side, not by checking each one against the source. I keep relearning that making test data realistic is itself part of verification.