fix(distance): keep Damerau-Levenshtein symmetric for repeated characters#778
Open
greymoth-jp wants to merge 1 commit into
Open
fix(distance): keep Damerau-Levenshtein symmetric for repeated characters#778greymoth-jp wants to merge 1 commit into
greymoth-jp wants to merge 1 commit into
Conversation
…ters
The unrestricted Damerau-Levenshtein path updated lastRowMap (the da table)
inside the inner column loop, so lastRowMap[targetElement] could resolve to the
current row and feed a negative (row - lastRowMatch - 1) term into the
transposition cost. That produced wrong, order-dependent distances such as
DamerauLevenshteinDistance('0,1,10,11', '0,11,110,111') === 0. Update the table
once per row, after the inner loop, so the transposition lookup only references
earlier rows. Fixes NaturalNode#757.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #757.
DamerauLevenshteinDistance(the default, unrestricted variant) can return a distance that depends on argument order, and even returns 0 for two different strings:A distance of 0 between non-equal strings, and the asymmetry, both break the metric, so the
0is wrong.Cause
The unrestricted path tracks
lastRowMap(thedatable in the standard algorithm: the last row each source character appeared in) to find candidate transpositions. The updatelastRowMap[sourceElement] = rowran inside the inner column loop. SincesourceElementis constant across that loop,lastRowMap[source[row - 1]]was already set to the currentrowbefore the row finished. When the target character at some column equals the current source character, the lookuplastRowMap[targetElement]then resolved to the current row, so the transposition costpicked up a negative
(row - lastRowMatch - 1)term and pulled the distance below its true value. Strings with repeated characters (here'0',',','1') trigger it.Fix
Move the
daupdate out of the inner loop so it runs once per row, after the columns are processed. The transposition lookup then only references earlier rows, which is what the algorithm assumes. This also fixesDamerauLevenshteinDistanceSearch, which shares the same code path.Verification
spec/damerau_levenshtein_spec.tsfor the case in Damerau–Levenshtein Distance Asymmetry with Certain Strings #757 (both argument orders return 3).