Skip to content

Commit

Permalink
PHL-306: Renaming 'filter profile' to 'policy'.
Browse files Browse the repository at this point in the history
  • Loading branch information
jzonthemtn committed Oct 17, 2023
1 parent ed53c6c commit 7e8d9ec
Show file tree
Hide file tree
Showing 110 changed files with 1,778 additions and 1,808 deletions.
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Phileas is the underlying core of [Philter](https://philterd.ai/philter/).
* Phileas can disambiguate types of sensitive information (i.e. SSN vs. phone number).
* Phileas can deidentify text consistently ("John Smith" is replaced consistently in certain documents).
* Phileas can shift dates or replace dates with approximate representations (i.e. "3 months ago").
* Phileas is customizable using "filter profiles" that define what sensitive information to find and how to redact it.
* Phileas uses policies to define what sensitive information to find and how to redact it.

## Supported PII, PHI, and Other Sensitive Information

Expand Down Expand Up @@ -108,10 +108,10 @@ PhileasConfiguration phileasConfiguration = ConfigFactory.create(PhileasConfigur
FilterService filterService = new PhileasFilterService(phileasConfiguration);
FilterResponse response = filterService.filter(filterProfiles, context, documentId, body, MimeType.TEXT_PLAIN);
FilterResponse response = filterService.filter(policies, context, documentId, body, MimeType.TEXT_PLAIN);
```

The `filterProfiles` is a list of `FilterProfile` classes. (See below for more about Filter Profiles.) The `context` and `documentId` are arbitrary values you can use to uniquely identify the text being filtered. The `body` is the text you are filtering. Lastly, we specify that the data is plain text.
The `policies` is a list of `Policy` classes. (See below for more about Policies.) The `context` and `documentId` are arbitrary values you can use to uniquely identify the text being filtered. The `body` is the text you are filtering. Lastly, we specify that the data is plain text.

The `response` contains information about the identified sensitive information along with the filtered text.

Expand All @@ -128,20 +128,20 @@ PhileasConfiguration phileasConfiguration = ConfigFactory.create(PhileasConfigur
FilterService filterService = new PhileasFilterService(phileasConfiguration);
BinaryDocumentFilterResponse response = filterService.filter(filterProfiles, context, documentId, body, MimeType.APPLICATION_PDF, MimeType.IMAGE_JPEG);
BinaryDocumentFilterResponse response = filterService.filter(policies, context, documentId, body, MimeType.APPLICATION_PDF, MimeType.IMAGE_JPEG);
```

The `filterProfiles` is a list of `FilterProfile` classes which are created by deserializing a filter profile from JSON. (See below for more about Filter Profiles.) The `context` and `documentId` are arbitrary values you can use to uniquely identify the text being filtered. The `body` is the text you are filtering. Lastly, we specify that the data is plain text.
The `policies` is a list of `Policy` classes which are created by deserializing a policy from JSON. (See below for more about Filter Profiles.) The `context` and `documentId` are arbitrary values you can use to uniquely identify the text being filtered. The `body` is the text you are filtering. Lastly, we specify that the data is plain text.

The `response` contains a zip file of the images generated by redacting the PDF document.

### Filter Profiles
### Policies

A "filter profile" is an instance of a `FilterProfile` class that tells Phileas the types of sensitive information to identify, and what to do with the sensitive information when found. A filter profile describes the entire filtering process, from what filters to apply, terms to ignore, to everything in between. Phileas can apply one or more filter profiles when `filter()` is called. The list of filter profiles will be applied in order as they were added to the list.
A policy is an instance of a `Policy` class that tells Phileas the types of sensitive information to identify, and what to do with the sensitive information when found. A policy describes the entire filtering process, from what filters to apply, terms to ignore, to everything in between. Phileas can apply one or more policies when `filter()` is called. The list of policies will be applied in order as they were added to the list.

For examples on creating a filter profile, look at [EndToEndTestsHelper](https://github.com/philterd/phileas/blob/main/phileas-core/src/test/java/io/philterd/test/phileas/services/EndToEndTestsHelper.java). The [PhileasFilterServiceTest](https://github.com/philterd/phileas/blob/main/phileas-core/src/test/java/io/philterd/test/phileas/services/PhileasFilterServiceTest.java) and [EndToEndTests](https://github.com/philterd/phileas/blob/main/phileas-core/src/test/java/io/philterd/test/phileas/services/EndToEndTests.java) test classes have examples of how to configure Phileas and filter text.
For examples on creating a policy, look at [EndToEndTestsHelper](https://github.com/philterd/phileas/blob/main/phileas-core/src/test/java/io/philterd/test/phileas/services/EndToEndTestsHelper.java). The [PhileasFilterServiceTest](https://github.com/philterd/phileas/blob/main/phileas-core/src/test/java/io/philterd/test/phileas/services/PhileasFilterServiceTest.java) and [EndToEndTests](https://github.com/philterd/phileas/blob/main/phileas-core/src/test/java/io/philterd/test/phileas/services/EndToEndTests.java) test classes have examples of how to configure Phileas and filter text.

Filter profiles can be de/serialized to JSON. Here is a basic (but valid) filter profile that identifies and redacts ages:
Filter profiles can be de/serialized to JSON. Here is a basic (but valid) policy that identifies and redacts ages:

```
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -119,18 +119,18 @@ public interface PhileasConfiguration extends Config {
// Filter Profiles
@DefaultValue("./profiles/")
@Key("filter.profiles.directory")
String filterProfilesDirectory();
String policiesDirectory();

@Key("filter.profiles.s3.bucket")
String filterProfilesS3Bucket();
String policiesS3Bucket();

@DefaultValue("")
@Key("filter.profiles.s3.prefix")
String filterProfilesS3Prefix();
String policiesS3Prefix();

@DefaultValue("us-east-1")
@Key("filter.profiles.s3.region")
String filterProfilesS3Region();
String policiesS3Region();

// Metrics
@DefaultValue("philter")
Expand Down
Loading

0 comments on commit 7e8d9ec

Please sign in to comment.