Skip to content

Latest commit

 

History

History
74 lines (60 loc) · 6.29 KB

MIGRATION_AS_DJANGO_APP.md

File metadata and controls

74 lines (60 loc) · 6.29 KB
Raw

Notes

  • Move onadata folder to KPI under openrosa

  • Rename all imports to follow the new path under KPI

  • Identify all duplicates settings. Left comments and #FIXME

  • Commented out all useless, redundant, duplicate settings

  • Import every useful and unique setting from KoboCAT project in kobo.settings.base.py

  • Specify databases allowed to be touched by Constance Post signal (see CONSTANCES_DB in settings)

  • Alter db_routers.py to block migrations from each other (logic in allow_migrate). [GitHub]

    • ⚠️ It does not really block migrations. It does add an entry in django_migrations table for every app registered in INSTALLED_APPS but does not create / alter tables in database. For example, when running ./manage.py migrate viewer in KPI, returns this

      Running migrations:
      Applying viewer.0001_initial... OK
      Applying viewer.0002_auto_20160205_1915... OK
      Applying viewer.0003_auto_20171123_1521... OK
      Applying viewer.0004_update_meta_data_export_types... OK

      but no tables are created. Django does this in purpose to handle permissions correctly at the model level.

  • Monkey-Patch django.contrib.auth.management.create_permissions to avoid crashing because content type is wrong while creating KoboCAT model-level permissions in post-signal emitted by migrate management command when applying (kinda fake) KoboCAT migrations in KPI [GitHub 1, GitHub 2]

  • Keep return True in allow_relation in db_routers.py. It seems to help when making relationships between common models in KPI and KoboCAT. For example, If I have a User object (loaded from KPI) and I want to retrieve its related UserProfile from KoboCAT, Django is able to create SQL to load the UserProfile object right out of the box (not 100% sure , to be tested it again). Side effect, will raise a ProgrammingError when trying to join tables from two different databases

  • Create a decorator (@use_db) and utility function to specify the database alias to use in db_routers.py. Useful to force code to use KoboCAT database when the database cannot be determinated with the app label. [GitHub]
    Example: django-guardian has to be installed in KPI. All of its querysets do not use .using('database_alias'), so they are pointing to KPI database.
    With the decorator, we can make django-guardian use KoboCAT database instead

  • Create a custom User model which extends AbstractUser. Keep retro-compatibility with auth.User. Use from kobo.apps.kobo_auth.shortcuts import User everywhere instead of from django.contrib.auth.models import User.
    Needed to overload has_perm and add some logic to test perm against the correct database (i.e. with context manager use_db) [GitHub]

  • Fork django_digest to use get_user_model() instead of direct import of User. [GitHub]

  • Overload django-guardian object permission back end to avoid checking permissions at object level with guardian and KPI objects. it does a warning though

    ?: (guardian.W001) Guardian authentication backend is not hooked

    [Create utility]
    Silent warning in base.py

  • Remove unicodecsv

  • Remove djangorestframework_guardian dependency

  • Handle DRF for KC (solution: create a base Viewset which follow DRF KoboCAT settings)

  • Import KoboCAT URLs in KPI at root (e.g. ./manage.py show_urls now displays KoboCAT endpoints /<endpoint> instead of /kobocat/<endpoint> as discussed earlier. Easier to setup NGINX)

  • Add Java 17 to KPI Dockerfile

  • Add KoboCAT PIP dependencies

  • Add a new script that is called from the entrypoint to handle migration history from existing and new installs.[GitHub] [GitHub 2]

  • Merge CELERY_BEAT_SCHEDULE

  • Update [NGINX config]

  • Change

    ~~- SERVICE_ACCOUNT_WHITELISTED_HOSTS=kf.kobo.internal:8000,kf.kobo.internal~~
- SERVICE_ACCOUNT_WHITELISTED_HOSTS=kf.kobo.internal:8000,kf.kobo.internal,kc.kobo.internal:8001,kc.kobo.internal

  • Change basename of KPI UserViewSet to user-kpi to avoid conflicts with KoboCAT

  • Use CELERY_TASK_ROUTES to route KoboCAT tasks to their own queue

  • Create new celery worker for KoboCAT

  • Clean-up KoboCAT templates

  • Remove ConditionalRedirectMiddleware (move specific condition inside download_xlsform)

  • Prefix service_health and service_health_minimal endpoints with legacy/ in KoboCAT to avoids conflicts with KPI

  • Use KPI classes for Digest and Token authentication (no use of KoboCAT tables anymore)

To-do

  • Merge any helper, utility duplicate (e.g. MongoHelper)
  • Remove Django-Digest tables in KoboCAT database for existing environment
  • Remove DRF Token tables in KoboCAT database for existing environment
  • Rename auth_user table to koboauth_user to match Django app kobo_auth

Known bugs

  • Endpoints can respond under each domain
  • 500 errors are handled by KoboCAT templates even if it is a KPI error