diff --git a/404.html b/404.html index b93d54b44..b228bb87a 100644 --- a/404.html +++ b/404.html @@ -27,7 +27,7 @@ -

404

That's a Four-Oh-Four.
Take me home
+

404

There's nothing here.
Take me home
diff --git a/docs/acknowledgments/index.html b/docs/acknowledgments/index.html index a61587a10..94f4a0829 100644 --- a/docs/acknowledgments/index.html +++ b/docs/acknowledgments/index.html @@ -24,7 +24,7 @@ Acknowledgments | NotifyBC - +

Acknowledgments

NotifyBC is built on a myriad of open source software. At runtime it also depends on a few services. Credit goes to their contributors. Notably

diff --git a/docs/api-subscription/index.html b/docs/api-subscription/index.html index 5a74c59e4..5d324a5c4 100644 --- a/docs/api-subscription/index.html +++ b/docs/api-subscription/index.html @@ -24,7 +24,7 @@ Subscription | NotifyBC - +

Subscription

The subscription API encapsulates the backend workflow of user subscription and un-subscription of push notification service. Depending on whether a API call comes from user browser as a user request or from an authorized server as an admin request, NotifyBC applies different validation rules. For user requests, the notification channel entered by user is unconfirmed. A confirmation code will be associated with this request. The confirmation code can be created in one of two ways:

  • by NotifyBC based on channel dependent subscription.confirmationRequest.<channel>.confirmationCodeRegex config.
  • by a trusted third party. This trusted third party encrypts the confirmation code using the public RSA key of the NotifyBC instance (see more about RSA Key Config) and pass the encrypted confirmation code to NotifyBC via user browser in the same subscription request. NotifyBC then decrypts to obtain the confirmation code. This method allows user subscribe to multiple notification services provided by NotifyBC instances in different trust domains (i.e. service providers) and only have to confirm the subscription channel once during one browser session. In such case only one NotifyBC instance should be chosen to deliver confirmation request to user.

Equipped with the confirmation code and a message template, NotifyBC can now send out confirmation request to unconfirmed subscription channel. At a minimum this confirmation request should contain the confirmation code. When user receives the message, he/she echos the confirmation code back to a NotifyBC provided API to verify against saved record. If match, the state of the subscription request is changed to confirmed.

For admin requests, NotifyBC can still perform the above confirmation process. But admin request has full CRUD privilege, including set the subscription state to confirmed, bypassing the confirmation process.

The workflow of user subscribing to notification services offered by a single service provider is illustrated by sequence diagram below. In this case, the confirmation code is generated by NotifyBC. single service provider subscription

In the case user subscribing to notifications offered by different service providers in separate trust domains, the confirmation code is generated by a third-party server app trusted by all NotifyBC instances. Following sequence diagram shows the workflow. The diagram indicates NotifyBC API Server 2 is chosen to send confirmation request.

multi service provider subscription

Model Schema

The API operates on following subscription data model fields:

NameAttributes

serviceName

name of the service. Avoid prefixing the name with underscore (_), or it may conflict with internal implementation.

typestring
requiredtrue

channel

name of the delivery channel. Valid values: email and sms. Notice inApp is invalid as in-app notification doesn't need subscription.

typestring
requiredtrue
defaultemail

userChannelId

user's delivery channel id, for example, email address

typestring
requiredtrue

id

subscription id

typestring, format depends on db
requiredfalse
auto-generatedtrue

state

state of subscription. Valid values: unconfirmed, confirmed, deleted

typestring
requiredfalse
defaultunconfirmed

userId

user id. Auto-populated for authenticated user requests.

typestring
requiredfalse

created

date and time of creation

typedate
requiredfalse
auto-generatedtrue

updated

date and time of last update

typedate
requiredfalse
auto-generatedtrue

confirmationRequest

an object containing these child fields
  • confirmationCodeRegex
    • type: string
    • regular expression used to generate confirmation code
  • confirmationCodeEncrypted
    • type: string
    • encrypted confirmation code
  • sendRequest
    • type: boolean
    • whether to send confirmation request
  • from, subject, textBody, htmlBody
    • type: string
    • these are email template fields used for sending email confirmation request. If confirmationRequest.sendRequest is true and channel is email, then these fields should be supplied in order to send confirmation email.
typeobject
requiredtrue for user request with encrypted confirmation code; false otherwise

broadcastPushNotificationFilter

a string conforming to jmespath filter expressions syntax after the question mark (?). The filter is matched against the data field of the subscription. Examples of filter
  • simple
    province == 'BC'
  • calling jmespath's built-in functions
    contains(province,'B')
  • calling custom filter functions
    contains_ci(province,'b')
  • compound
    (contains(province,'BC') || contains_ci(province,'b')) && city == 'Victoria'
All of above filters will match data object {"province": "BC", "city": "Victoria"}
typestring
requiredfalse

data

An object used by

data object can only be populated by non-anonymous requests.

typeobject
requiredfalse

unsubscriptionCode

generated randomly according to RegEx config anonymousUnsubscription.code.regex during anonymous subscription if config anonymousUnsubscription.code.required is set to true

typestring
requiredfalse
auto-generatedtrue

unsubscribedAdditionalServices

generated if parameter additionalServices is supplied in unsubscription request. Contains 2 sub-fields: ids and names, each being a list identifying the additional unsubscribed subscriptions.

typeobject
requiredfalse
auto-generatedtrue

Get Subscriptions

GET /subscriptions
diff --git a/docs/config-cronJobs/index.html b/docs/config-cronJobs/index.html
index f3e807307..c890c87f3 100644
--- a/docs/config-cronJobs/index.html
+++ b/docs/config-cronJobs/index.html
@@ -24,7 +24,7 @@
     
     Cron Jobs | NotifyBC
     
-    
+    
   
   
     
 Cron Jobs
NotifyBC runs several cron jobs described below. These jobs are controlled by sub-properties defined in config object cron. To change config, create the object and properties in file /src/config.local.js.
By default cron jobs are enabled. In a multi-node deployment, cron jobs should only run on the master node to ensure single execution.
All cron jobs have a property named timeSpec with the value of a space separated fields conforming to unix crontab formatopen in new window with an optional left-most seconds field. See allowed rangesopen in new window of each field.
 Purge Data
This cron job purges old notifications, subscriptions and notification bounces. The default frequency of cron job and retention policy are defined by cron.purgeData config object in file /src/config.ts
module.exports = {
diff --git a/docs/config-database/index.html b/docs/config-database/index.html
index dd062703d..93c03a400 100644
--- a/docs/config-database/index.html
+++ b/docs/config-database/index.html
@@ -24,7 +24,7 @@
     
     Database | NotifyBC
     
-    
+    
   
   
     
 Database
By default NotifyBC uses in-memory database backed up by folder /server/database/ for local and docker deployment and MongoDB for Kubernetes deployment. To use MongoDB for non-Kubernetes deployment, add file /src/datasources/db.datasource.(local|<env>).(json|js|ts) with MongoDB connection information such as following:
module.exports = {
diff --git a/docs/config-httpHost/index.html b/docs/config-httpHost/index.html
index 24aa7d910..c2e23bbbc 100644
--- a/docs/config-httpHost/index.html
+++ b/docs/config-httpHost/index.html
@@ -24,7 +24,7 @@
     
     HTTP Host | NotifyBC
     
-    
+    
   
   
     
 HTTP Host
httpHost config sets the fallback http host used by
  • mail merge token substitution
  • internal HTTP requests spawned by NotifyBC
httpHost can be overridden by other configs or data. For example
  • internalHttpHost config
  • httpHost field in a notification
There are contexts where there is no alternatives to httpHost. Therefore this config should be defined.
Define the config, which has no default value, in /src/config.local.js
module.exports = {
diff --git a/docs/developer-notes/index.html b/docs/developer-notes/index.html
index 686192369..cfe602a32 100644
--- a/docs/developer-notes/index.html
+++ b/docs/developer-notes/index.html
@@ -24,7 +24,7 @@
     
     Developer Notes | NotifyBC
     
-    
+    
   
   
     
 Developer Notes
 Setup development environment
Install Visual Studio Code and following extensions:
  • Prettier
  • ESLint
  • Vetur
  • Code Spell Checker
  • Debugger for Chrome
Multiple run configs have been created to facilitate debugging server, client, test and docs.
Client certificate authentication doesn't work in client debugger
Because Vue cli webpack dev server cannot proxy passthrough HTTPS connections, client certificate authentication doesn't work in client debugger. If testing client certificate authentication in web console is needed, run npm run build to generate prod client distribution and launch server debugger on https://localhost:3000
 Automated Testing
NotifyBC uses Jestopen in new window test framework bundled in NestJS. To launch test, run npm run test:e2e. A Test launch config is provided to debug in VS Code.
Github Actions runs tests as part of the build. All test scripts should be able to run unattended, headless, quickly and depend only on local resources.
 Writing Test Specs
Thanks to supertestopen in new window and MongoDB In-Memory Serveropen in new window, test specs can be written to cover nearly end-to-end request processing workflow (only sendMail and sendSMS need to be mocked). This allows test specs to anchor onto business requirements rather than program units such as functions or files, resulting in regression tests that are more resilient to code refactoring. Whenever possible, a test spec should be written to
  • start at a processing phase as early as possible. For example, to test a REST end point, start with the HTTP user request.
  • assert outcome of a processing phase as late and down below as possible - the HTTP response body/code, the database record created, for example.
  • avoid asserting middleware function input/output to facilitate code refactoring.
  • mock email/sms sending function (implemented by default). Inspect the input of the function, or at least assert the function has been called.
 Install Docs Website
If you want to contribute to NotifyBC docs beyond simple fix ups, run
cd docs && npm install && npm run dev
diff --git a/docs/health-check/index.html b/docs/health-check/index.html
index 91cfc7fdb..5a4a8f4af 100644
--- a/docs/health-check/index.html
+++ b/docs/health-check/index.html
@@ -24,7 +24,7 @@
     
     Health Check | NotifyBC
     
-    
+    
   
   
     
 Health Check
Health status of NotifyBC can be obtained by querying /health API end point. For example
$ curl -s http://localhost:3000/api/health | jq
diff --git a/docs/overview/index.html b/docs/overview/index.html
index bd9f63c67..e810ee3ac 100644
--- a/docs/overview/index.html
+++ b/docs/overview/index.html
@@ -24,7 +24,7 @@
     
     Overview | NotifyBC
     
-    
+    
   
   
     
 Overview
NotifyBC is a general purpose API Server to manage subscriptions and dispatch notifications. It aims to implement some common backend processes of a notification service without imposing any constraints on the UI frontend, nor impeding other server components' functionality. This is achieved by interacting with user browser and other server components through RESTful API and other standard protocols in a loosely coupled way.
 Features
NotifyBC facilitates both anonymous and authentication-enabled secure webapps implementing notification feature. A NotifyBC server instance supports multiple notification services. A service is a topic of interest that user wants to receive updates. It is used as the partition of notification messages and user subscriptions. A user may subscribe to a service in multiple push delivery channels allowed. A user may subscribe to multiple services. In-app pull notification doesn't require subscription as it's not intrusive to user.
 notification
  • both in-app pull notifications (a.k.a. messages or alerts) and push notifications
  • multiple push notifications delivery channels 
    • email
    • sms
  • unicast and broadcast message types
  • future-dated notifications
  • for in-app pull notifications 
    • support read and deleted message states
    • message expiration
    • deleted messages are not deleted immediately for auditing and recovery purposes
  • for broadcast push notifications 
    • allow both sync and async POST API calls. For async API call, an optional callback url is supported
    • can be auto-generated from RSS feeds
    • allow user to specify filter rules evaluated against data contained in the notification
    • allow sender to specify filter rules evaluated against data contained in the subscription
    • allow application developer to create custom filter functions used by the filter rules mentioned above
 subscription and un-subscription
  • verify the ownership of push notification subscription channel: 
    • generates confirmation code based on a regex input
    • send confirmation request to unconfirmed subscription channel
    • verify confirmation code
  • generate random un-subscription code
  • send acknowledgement message after un-subscription for anonymous subscribers
  • bulk unsubscription
  • list-unsubscribe by email
  • track bounces and unsubscribe the recipient from all subscriptions when hard bounce count exceeds threshold
  • sms user can unsubscribe by replying a shortcode keyword with Swift sms provider
 mail merge
Strings in notification or subscription message that are enclosed between curly braces { } are called tokens, also known as placeholders. Tokens are replaced based on the context of notification or subscription when dispatching the message. To avoid treating a string between curly braces as a token, escape the curly braces with backslash \. For example \{i_am_not_a_token\} is not a token. It will be rendered as {i_am_not_a_token}.
Tokens whose names are predetermined by NotifyBC are called static tokens; otherwise they are called dynamic tokens.
 static tokens
NotifyBC recognizes following case-insensitive static tokens. Most of the names are self-explanatory.
  • {subscription_confirmation_url}
  • {subscription_confirmation_code}
  • {service_name}
  • {http_host} - http host in the form http(s): //<host_name>:<port>. The value is obtained from the http request that triggers the message
  • {rest_api_root} - REST API URL path prefix
  • {subscription_id}
  • anonymous unsubscription related tokens 
    • {unsubscription_url}
    • {unsubscription_all_url} - url to unsubscribe all services the user has subscribed on this NotifyBC instance
    • {unsubscription_code}
    • {unsubscription_reversion_url}
    • {unsubscription_service_names} - includes {service_name} and additional services user has unsubscribed, prefixed with conditionally pluralized word service.
 dynamic tokens
Dynamic tokens are replaced with correspondingly named sub-field of data field in the notification or subscription if exist. Qualify token name with notification:: or subscription:: to indicate the source of substitution. If token name is not qualified, then both notification and subscription are checked, with notification taking precedence. Nested and indexed sub-fields are supported.
Examples
  • {notification::description} is replaced with field data.description of the notification if exist
  • {subscription::gender} is replaced with field data.gender of the subscription if exist
  • {addresses[0].city} is replaced with field data.addresses[0].city of the notification if exist; otherwise is replaced with field data.addresses[0].city of the subscription if exist
  • {nonexistingDataField} is unreplaced if neither notification nor subscription contains data.nonexistingDataField
As exception, in order to prevent spamming by unconfirmed subscribers, dynamic tokens in subscription confirmation request message and duplicated subscription message are not replaced with subscription data, for example {subscription::...} tokens are left unchanged.
Notification by RSS feeds relies on dynamic token
A notification created by RSS feeds relies on dynamic token to supply the context to message template. In this case the data field contains the RSS item.
 Architecture
 Request Types
NotifyBC, designed to be a microservice, doesn't use full-blown ACL to secure API calls. Instead, it classifies incoming requests into admin and user types. The key difference is while both admin and user can subscribe to notifications, only admin can post notifications.
Each type has two subtypes based on following criteria
  • super-admin, if the request meets both of the following two requirements
    1. The request carries one of the following two attributes
      • the source ip is in the admin ip list
      • has a client certificate that is signed using NotifyBC server certificate. See Client certificate authentication on how to sign.
    2. The request doesn't contain any of following case insensitive HTTP headers, with the first three being SiteMinder headers
      • sm_universalid
      • sm_user
      • smgov_userdisplayname
      • is_anonymous
  • admin, if the request is not super-admin and meets one of the following criteria
    • has a valid access token associated with an builtin admin user created and logged in using the administrator api, and the request doesn't contain any HTTP headers listed above
    • has a valid OIDC access token containing customizable admin profile attributes
    access token disambiguation
    Here the term access token has been used to refer two different things
    1. the token associated with a builtin admin user
    2. the token generated by OIDC provider.
    To reduce confusion, throughout the documentation the former is called access token and the latter is called OIDC access token.
  • authenticated user, if the request is neither super-admin nor admin, and meets one fo the following criteria
    • contains any of the 3 SiteMinder headers listed above, and comes from either trusted SiteMinder proxy or admin ip list
    • contains a valid OIDC access token
  • anonymous user, if the request doesn't meet any of the above criteria
The only extra privileges that a super-admin has over admin are that super-admin can perform CRUD operations on configuration, bounce and administrator entities through REST API. In the remaining docs, when no further distinction is necessary, an admin request refers to both super-admin and admin request; a user request refers to both authenticated and anonymous request.
An admin request carries full authorization whereas user request has limited access. For example, a user request is not allowed to
  • send notification
  • bypass the delivery channel confirmation process when subscribing to a service
  • retrieve push notifications through API (can only receive notification from push notification channel such as email)
  • retrieve in-app notifications that is not targeted to the current user
The result of an API call to the same end point may differ depending on the request type. For example, the call GET /notifications without a filter will return all notifications to all users for an admin request, but only non-deleted, non-expired in-app notifications for authenticated user request, and forbidden for anonymous user request. Sometimes it is desirable for a request from admin ip list, which would normally be admin request, to be voluntarily downgraded to user request in order to take advantage of predefined filters such as the ones described above. This can be achieved by adding one of the HTTP headers listed above to the request. This is also why admin request is not determined by ip or token alone.
The way NotifyBC interacts with other components is diagrammed below. architecture diagram
 Authentication Strategies
API requests to NotifyBC can be either anonymous or authenticated. As alluded in Request Types above, NotifyBC supports following authentication strategies
  1. ip whitelisting
  2. client certificate
  3. access token associated with an builtin admin user
  4. OpenID Connect (OIDC)
  5. CA SiteMinder
Authentication is performed in above order. Once a request passed an authentication strategy, the rest strategies are skipped. A request that failed all authentication strategies is anonymous.
The mapping between authentication strategy and request type is
AdminUser
Super-adminadminauthenticatedanonymous
ip whitelisting✔
client certifcate✔
access token✔
OIDC✔✔
SiteMinder✔
Which authentication strategy to use?
Because ip whitelist doesn't expire and client certificate usually has a relatively long expiration period (say one year), they are suitable for long-running unattended server processes such as server-side code of web apps, cron jobs, IOT sensors etc. The server processes have to be trusted because once authenticated, they have full privilege to NotifyBC. Usually the server processes and NotifyBC instance are in the same administrative domain, i.e. managed by the same admin group of an organization.
By contrast, OIDC and SiteMinder use short-lived tokens or session cookies. Therefore they are only suitable for interactive user sessions.
Access token associated with an builtin admin user should be avoided whenever possible.
Here are some common scenarios and recommendations
  • For server-side code of web apps
    • use OIDC if the web app is OIDC enabled and user requests can be proxied to NotifyBC by web app; otherwise
    • use ip whitelisting if obtaining ip is feasible; otherwise
    • use client certificate (requires a little more config than ip whitelisting)
  • For front-end browser-based web apps such as SPAs
    • use OIDC
  • For server apps that send requests spontaneously such as IOT sensors, cron jobs
    • use ip whitelisting if obtaining ip is feasible; otherwise
    • client certificate
  • If NotifyBC is ued by a SiteMinder protected web apps and NotifyBC is also protected by SiteMinder
    • use SiteMinder
 Application Framework
NotifyBC is created on NestJSopen in new window. Contributors to source code of NotifyBC should be familiar with NestJS. NestJS Docsopen in new window serves a good complement to this documentation.