New auto decryption (#65)

* Refactored lookups and mixins

* Added support for auto decryption without crazy managers extra sql. The old custom manager broke support for defer() and only() which this restores. This also adds support for filtering on encrypted fields with any supported built-in WITHOUT having to use annotate / aggregates.

* Added support for auto decryption of foreign key tables without additional queries or annotations / aggregations

* Added test demostrating support for get_by_natural_key(). Fixes #23

* Added tests demonstrating support for get_or_create() and update_or_create()

* Added tests demonstrating support for get_or_create() and update_or_create()

* Fix flake8 errors

* Updated CHANGELOG

* * Updated CHANGELOG to include upgrade instructions
* Removed admin, manager, aggregates
* Updated tests

* Fixes to tests

* Improved make file

* Fix flake8

* Fixes per feedback on PR #65 (tests for aggregate, distinct, annotate)

* Added test for `distinct()` and related documentation for workaround for 2.0.x and lower

* Updated CHANGELOG, fixed spelling typo

* Added additional assertions for test_aggregates()

* Fixed issue with relabel_clone calling DecryptCol which "had" a different constructor signature than Col. Extract the decyrpt sql and cast type from the target instead.

django/db/models/expressions.py", line 745, in relabeled_clone
    return self.__class__(relabels.get(self.alias, self.alias), self.target, self.output_field)

* Removed unneed use_for_related_fields = True, use_in_migrations = True

* Test for alias of get_col() for DecryptedCol

* Code cleanup

* Added PyUp safety checks

* Separated out dev/maintainers requirements into requirements_dev.txt so it simplifies what is needed for install

* * Updated packaging / setup.py to include long description
* Added AUTHORS and updated CONTRIBUTING

* Fixed makefile

* Updated TravisCI to use Xenial to gain Python 3.7 in the matrix

* Removed compatibility code for Django 1.7 and lower. The lowest version we support is Django 1.11.

* Added HTML for coverage

.travis.yml

@@ -1,13 +1,20 @@
 language: python
+sudo: required
+dist: xenial
 python:
   - 3.5
   - 3.6
-script: make test
+  - 3.7
+script: make test-coverage
 notifications:
   email: false
-install:
+services:
+  - postgresql
+before_script:
   - psql -c 'CREATE DATABASE pgcrypto_fields' -U postgres
-  - pip install -r requirements.txt
+install:
+  - pip install -e .
+  - pip install -r requirements_dev.txt
   - pip install $DJANGO
 env:
   matrix:

AUTHORS.md

@@ -0,0 +1,13 @@
+# Credits
+
+
+## Development Lead
+
+* Charlie Denton <[email protected]>
+* Kévin Etienne <[email protected]>
+* Peter J. Farrell <[email protected]>
+* Max Peterson <[email protected]>
+
+## Contributors
+
+None yet. Why not be the first?

CHANGELOG.md

@@ -1,4 +1,17 @@
-### 2.3.0
+## 2.4.0
+
+* Added auto-decryption of all encrypted fields including FK tables
+* Removed django-pgcrypto-fields `aggregates`, `PGPManager` and `PGPAdmin` as they are no longer needed
+* Added support for `get_or_create()` and `update_or_create()` (#27)
+* Added support for `get_by_natural_key()` (#23)
+* Added support for `only()` and `defer()` as they were not supported with `PGPManager`
+* Added support for `distinct()` (Django 2.1+ with workaround available for 2.0 and lower)
+* Separated out dev requirements from setup.py requirements
+* Updated packaging / setup.py to include long description
+* Added AUTHORS and updated CONTRIBUTING
+* Updated TravisCI to use Xenial to gain Python 3.7 in the matrix
+
+## 2.3.1
 
 * Added `__range` lookup for Date / DateTime fields (#59)
 * Remove compatibility for `Django 1.8, 1.9, and 1.10` (#62)
@@ -10,6 +23,10 @@
 * Updated Travis config to include Python 3.5 and 3.6
 * Refactored lookups and mixins
 
+## 2.3.0
+
+* Invalid release, bump to 2.3.1
+
 ## 2.2.0
 
 * Merge `.coveragerc` into `setup.cfg`

CONTRIBUTING.md

@@ -1,25 +1,117 @@
 # Contributing to Django-PGCrypto-Fields
 
-We welcome contributions in many forms:
+Contributions are welcome, and they are greatly appreciated! Every little bit
+helps, and credit will always be given.
+
+You can contribute in many ways:
 
 * Code patches and enhancements
 * Documentation improvements
 * Bug reports and patch reviews
 
-## Running Tests
+## Types of Contributions
+
+### Report Bugs
+
+Report bugs at https://github.com/incuna/django-pgcrypto-fields/issues
+
+If you are reporting a bug, please include:
+
+* Your operating system name and version.
+* Any details about your local setup that might be helpful in troubleshooting.
+* Detailed steps to reproduce the bug.
+
+### Fix Bugs
+
+Look through the GitHub issues for bugs. Anything tagged with "bug" and "help
+wanted" is open to whoever wants to implement it.
+
+### Implement Features
+
+Look through the GitHub issues for features. Anything tagged with "enhancement"
+and "help wanted" is open to whoever wants to implement it.
+
+### Write Documentation
+
+django-pgcrypto-fields could always use more documentation, whether as part of the
+official django-pgcrypto-fields docs, in docstrings, or even on the web in blog posts,
+articles, and such.
+
+### Submit Feedback
+
+The best way to send feedback is to file an issue at https://github.com/incuna/django-pgcrypto-fields/issues
+
+If you are proposing a feature:
+
+* Explain in detail how it would work.
+* Keep the scope as narrow as possible, to make it easier to implement.
+* Remember that this is a volunteer-driven project, and that contributions
+  are welcome :-)
+
+## Get Started!
+
+Ready to contribute? Here's how to set up `django-pgcrypto-fields` for local development.
+
+1. Fork the `django-pgcrypto-fields` repo on GitHub.
+2. Clone your fork locally:
+
+    ```bash
+    $ git clone [email protected]:your_name_here/pgcrypto.git
+    ```
+
+3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:
+
+    ```bash
+    $ mkvirtualenv django-pgcrypto-fields
+    $ cd django-pgcrypto-fields/
+    $ pip install -r requirements_dev.txt --upgrade
+    ```
+
+4. Create a branch for local development:
+
+    ```bash
+    $ git checkout -b name-of-your-bugfix-or-feature
+    ```
+
+    Now you can make your changes locally.
+
+5. When you're done making changes, check that your changes pass flake8 and the
+   tests:
+
+    ```bash
+    $ make test
+    ```
+
+   To get flake8 and tox, just pip install them into your virtualenv.
+
+6. Commit your changes and push your branch to GitHub:
+
+    ```bash
+    $ git add .
+    $ git commit -m "Your detailed description of your changes."
+    $ git push origin name-of-your-bugfix-or-feature
+    ```
+
+7. Submit a pull request through the GitHub website.
 
-* Install requirements to a virtual environment
-* Setup a local PostgreSQL server
-* Create a PostreSQL database named `pgcrypto_fields`
-* In a terminal, run `make test`
+### Pull Request Guidelines
 
+Before you submit a pull request, check that it meets these guidelines:
 
-## Releasing to PyPI
+1. The pull request should include tests.
+2. If the pull request adds functionality, the docs should be updated. Put
+   your new functionality into a function with a docstring, and add the
+   feature to the list in README.rst.
+3. The pull request should work for Python 3.4, 3.5 and 3.7. Check
+   https://travis-ci.org/incuna/django-pgcrypto-fields/pull_requests
+   and make sure that the tests pass for all supported Python versions.
 
-This section only applies to maintainers.
+### Deploying
 
-In your virtual environment, run
+A reminder for the maintainers on how to deploy.
+Make sure all your changes are committed (including an entry in CHANGELOG.md).
+Then run:
 
-* `pip install pip --upgrade`
-* `pip install setuptools wheel twine`
-* `make release`
+```bash
+$ make release
+```

Makefile

@@ -1,15 +1,55 @@
 SHELL := /bin/bash
 
+define BROWSER_PYSCRIPT
+import os, webbrowser, sys
+
+try:
+	from urllib import pathname2url
+except:
+	from urllib.request import pathname2url
+
+webbrowser.open("file://" + pathname2url(os.path.abspath(sys.argv[1])))
+endef
+export BROWSER_PYSCRIPT
+
+define PRINT_HELP_PYSCRIPT
+import re, sys
+
+for line in sys.stdin:
+	match = re.match(r'^([a-zA-Z_-]+):.*?## (.*)$$', line)
+	if match:
+		target, help = match.groups()
+		print("%-20s %s" % (target, help))
+endef
+export PRINT_HELP_PYSCRIPT
+
+BROWSER := python -c "$$BROWSER_PYSCRIPT"
+
 help:
-	@echo "Usage:"
-	@echo " make release | Release to pypi."
-	@echo " make test | Run the tests."
+	@python -c "$$PRINT_HELP_PYSCRIPT" < $(MAKEFILE_LIST)
 
-release:
+lint: ## Check style with flake8
+	@flake8 .
+
+clean-build: ## Remove build artifacts
+	rm -r -f dist/*
+	rm -r -f build/*
+	rm -fr htmlcov/
+
+build: clean-build ## Builds source and wheel package
 	python setup.py sdist bdist_wheel
+	ls -l dist
+
+release: ## Package and upload a release
 	twine upload dist/*
 
-test:
-	@coverage run ./tests/run.py
-	@coverage report
-	@flake8 .
+test: clean-build lint ## Run tests quickly with the default Python
+	./tests/run.py
+
+test-coverage: ## Check code coverage quickly with the default Python
+	coverage run ./tests/run.py
+	coverage report -m
+
+test-coverage-html: test-coverage  ## Check code coverage quickly with the default Python and show report
+	coverage html
+	$(BROWSER) htmlcov/index.html