I have a strange issue in the deidentification model with Spark NLP for Healthcare. #288
-
|
Hi, so I have a strange issue in the deidentification model, i'm trying to setup a whitelist so it only deidentifies dates and names, using your example I setup the NerConverter with .setWhiteList(['NAME', 'DATE']), now, I see it works because in the annotations i'm getting back only these entity types, however in the result of the DeIdentificationModel its still deidentifying IDs and replaces them the the string, any idea why this happens? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
As a workaround, you can set the the local regex rules as empty You need an empty file set as a workaround, we will have the setIgnoreRegex() param later on. |
Beta Was this translation helpful? Give feedback.
As a workaround, you can set the the local regex rules as empty
.setRegexPatternsDictionary("./empty_string.txt")\
You need an empty file set as a workaround, we will have the setIgnoreRegex() param later on.