Skip to content

fix(distance): keep Damerau-Levenshtein symmetric for repeated characters#778

Open
greymoth-jp wants to merge 1 commit into
NaturalNode:masterfrom
greymoth-jp:fix-damerau-symmetry
Open

fix(distance): keep Damerau-Levenshtein symmetric for repeated characters#778
greymoth-jp wants to merge 1 commit into
NaturalNode:masterfrom
greymoth-jp:fix-damerau-symmetry

Conversation

@greymoth-jp

Copy link
Copy Markdown

Fixes #757.

DamerauLevenshteinDistance (the default, unrestricted variant) can return a distance that depends on argument order, and even returns 0 for two different strings:

DamerauLevenshteinDistance('0,1,10,11', '0,11,110,111') // 0  (should be 3)
DamerauLevenshteinDistance('0,11,110,111', '0,1,10,11') // 3

A distance of 0 between non-equal strings, and the asymmetry, both break the metric, so the 0 is wrong.

Cause

The unrestricted path tracks lastRowMap (the da table in the standard algorithm: the last row each source character appeared in) to find candidate transpositions. The update lastRowMap[sourceElement] = row ran inside the inner column loop. Since sourceElement is constant across that loop, lastRowMap[source[row - 1]] was already set to the current row before the row finished. When the target character at some column equals the current source character, the lookup lastRowMap[targetElement] then resolved to the current row, so the transposition cost

(row - lastRowMatch - 1) * deletion_cost + ...

picked up a negative (row - lastRowMatch - 1) term and pulled the distance below its true value. Strings with repeated characters (here '0', ',', '1') trigger it.

Fix

Move the da update out of the inner loop so it runs once per row, after the columns are processed. The transposition lookup then only references earlier rows, which is what the algorithm assumes. This also fixes DamerauLevenshteinDistanceSearch, which shares the same code path.

Verification

  • Added a regression test in spec/damerau_levenshtein_spec.ts for the case in Damerau–Levenshtein Distance Asymmetry with Certain Strings #757 (both argument orders return 3).
  • The test suite passes: 859 specs, 0 failures.
  • Cross-checked the patched output against a separate brute-force Damerau-Levenshtein implementation over 20k random string pairs: no mismatches and no asymmetries.

…ters

The unrestricted Damerau-Levenshtein path updated lastRowMap (the da table)
inside the inner column loop, so lastRowMap[targetElement] could resolve to the
current row and feed a negative (row - lastRowMatch - 1) term into the
transposition cost. That produced wrong, order-dependent distances such as
DamerauLevenshteinDistance('0,1,10,11', '0,11,110,111') === 0. Update the table
once per row, after the inner loop, so the transposition lookup only references
earlier rows. Fixes NaturalNode#757.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Damerau–Levenshtein Distance Asymmetry with Certain Strings

1 participant