Tools and Practices

This document details the custom tools and practices we use

Tools

Deep Field Populate Plugin

Coded here, this plugin helps us to recursively populate the data of foreign key fields from the foreign key reference tables in the queried table, best to understand with an example.

Example:

In our current db schema

• Material table has a foreign key link to Recognition table (with recognition_id as foreign key)
• Recognition table has a foreign key link to Users table (with recognition_for and recognition_by as foreign key)

Lets say we queried the material by the route /v1/materials/{material_id}, (please see swagger docs for api in Published Services), it will return us a response like this

{
    "material_id": "87105128-1359-4af8-8293-3b763812a7ce",
    "material_title": "First Video Material",
    "material_description": null,
    "material_link": "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4",
    "material_type": "video",
    "recognition_id": "e861c50e-6a95-45ee-a697-f74ad75837e6",
    "author_id": "095e63ab-f396-45c9-a4f9-c9ef13d385eb",
    "is_claimable": true,
    "created_at": "2022-12-30T22:51:42.065Z"
}

Here we have a foreign key (recognition_id) to recognitions table and we may want to get the details of the recognition_id associated with this material. For that the obvious course of action would be to make a separate request to /v1/recognitions/{recognition_id}.

But that would be an extra api and db call (which is expensive in large scale). Imagine we have a lot of materials to query and and for each material we want to get the recognition details, doing all this with separate api calls would be a lot of api/db calls, time consuming, and a mess.

With the populate plugin implementation, we can just pass a query param when we query for the materials, basically telling to fill details for recognition in the material response, v1/materials/87105128-1359-4af8-8293-3b763812a7ce?populate[]=recognition_id, giving us the response

{
    "material_id": "87105128-1359-4af8-8293-3b763812a7ce",
    "material_title": "First Video Material",
    "material_description": null,
    "material_link": "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4",
    "material_type": "video",
    "recognition_id": "e861c50e-6a95-45ee-a697-f74ad75837e6",
    "author_id": "095e63ab-f396-45c9-a4f9-c9ef13d385eb",
    "is_claimable": true,
    "created_at": "2022-12-30T22:51:42.065Z",
    "recognition": {
        "recognition_id": "e861c50e-6a95-45ee-a697-f74ad75837e6",
        "recognition_by": "32a575ac-7166-412b-8ee5-b30306a322b3",
        "recognition_for": "095e63ab-f396-45c9-a4f9-c9ef13d385eb",
        "recognition_description": null,
        "recognition_issued": "2022-12-30T22:51:43.189067",
        "status": "pending",
        "status_updated": "2022-12-30T22:51:43.187",
        "created_at": "2022-12-30T22:51:43.189067"
    }
}

We have now cut 1x extra api and db call. This saves us huge when we are querying a big collection with field population.

There is more to this field population plugin. The one we discussed above is 1st level deep field population. We can do nth level deep field population with this plugin (assuming we provide the correct mappings to db).

Now let's say we want to get details of recognition_for and recognition_by user in the populated recognition response. This is 2nd level deep field population. To populate nested or deep fields, we can use periods (.) between the populated param array passed in the url.

With these params v1/materials/87105128-1359-4af8-8293-3b763812a7ce?populate[]=recognition_id&populate[]=recognition_id.recognition_by&populate[]=recognition_id.recognition_for, we get this result

{
    "material_id": "87105128-1359-4af8-8293-3b763812a7ce",
    "material_title": "First Video Material",
    "material_description": null,
    "material_link": "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4",
    "material_type": "video",
    "recognition_id": "e861c50e-6a95-45ee-a697-f74ad75837e6",
    "author_id": "095e63ab-f396-45c9-a4f9-c9ef13d385eb",
    "is_claimable": true,
    "created_at": "2022-12-30T22:51:42.065Z",
    "recognition": {
        "recognition_id": "e861c50e-6a95-45ee-a697-f74ad75837e6",
        "recognition_by": {
            "user_id": "32a575ac-7166-412b-8ee5-b30306a322b3",
            "user_name": "huzaifa",
            "user_bio": "I am a human",
            "image_url": null,
            "reputation_stars": 2,
            "date_joined": "2022-11-25",
            "is_invited": null
        },
        "recognition_for": {
            "user_id": "095e63ab-f396-45c9-a4f9-c9ef13d385eb",
            "user_name": "john",
            "user_bio": "ready to explore",
            "image_url": "https://cryptologos.cc/logos/cardano-ada-logo.png",
            "reputation_stars": 0,
            "date_joined": "2022-12-09",
            "is_invited": null
        },
        "recognition_description": null,
        "recognition_issued": "2022-12-30T22:51:43.189067",
        "status": "pending",
        "status_updated": "2022-12-30T22:51:43.187",
        "created_at": "2022-12-30T22:51:43.189067"
    }
}

This is an extendable solution to multiple deep level field population, whenever we add/update our foreign keys, we have to update the db mappings in order to ensure the plugin works correctly.

In the base validation layer, we make sure that only correct populate field values pass through to the populate plugin.

Practices

Handling Business logic in API

We have 4 main layers in api

|- route # acts as the interaction point for the end user
|- validation # checks for validity of data
|- control # handles business logic and checks for correctness of data
|- service # handles modification to db (a worker to db)

There is an important distinction b/w the service and control layer i.e no business logic should be in the service layer. There is a neat trick to avoid mixing business logic in service layer, whenever you feel the need to import a service (lets say creation service) in another service (lets say user service), this is a place where we are mixing business logic. In this case the services should be imported in the relevant control layer and all business logic will move there.

The only purpose of service layer is to make modifications to db. Anything else it does is most probably a business logic

Error Handling

Frontend

In frontend, we have made custom supersets of html5 form components that act as a wrapper to a core component, available here. The important part of these components are that they include form validation if used correctly with the custom form component.

Form validations are done with react-hook-form composition api and in those we try to verify the validity of the submitted data, the correctness of this data is verified when the data is sent to api.

Example: The custom Input has form validation if it is passed a correct name prop and hooked to the form component.

Form Validation looks like this

After the submission of the form, the data gets sent to the api where we validate for both correctness and validity of data. If invalid, a response will be returned with a detailed error message which we display in the frontend as a separate block element like this (red block)

This response message is displayed as green (success) when the api call is successful and has no validation/correctness error.