[feature] Added configuration variables to DeviceGroup #738

Closes #738

README.rst

@@ -531,7 +531,24 @@ with templates, this feature is also known as *configuration context*, think of
 it like a dictionary which is passed to the function which renders the
 configuration, so that it can fill variables according to the passed context.
 
-The different ways in which variables are defined are described below.
+The different ways in which variables are defined are described below in
+the order (high to low) of their precedence:
+
+1. `User defined device variables <#user-defined-device-variables>`_
+2. `Predefined device variables <#predefined-device-variables>`_
+3. `Group variables <#group-variables>`_
+4. `Global variables <#global-variables>`_
+5. `Template default values <#template-default-values>`_
+
+User defined device variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the device configuration section you can find a section named
+"Configuration variables" where it is possible to define the configuration
+variables and their values, as shown in the example below:
+
+.. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/device-context.png
+   :alt: context
 
 Predefined device variables
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -543,15 +560,19 @@ Each device gets the following attributes passed as configuration variables:
 * ``name``
 * ``mac_address``
 
-User defined device variables
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Group variables
+~~~~~~~~~~~~~~~
 
-In the device configuration section you can find a section named
-"Configuration variables" where it is possible to define the configuration
-variables and their values, as shown in the example below:
+Variables can also be defined in `Device groups <#device-groups>`__.
 
-.. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/device-context.png
-   :alt: context
+Refer the `Group configuration variables <group-configuration-variables>`_
+section for detailed information.
+
+Global variables
+~~~~~~~~~~~~~~~~
+
+Variables can also be defined globally using the
+`OPENWISP_CONTROLLER_CONTEXT <#openwisp-controller-context>`_ setting.
 
 Template default values
 ~~~~~~~~~~~~~~~~~~~~~~~
@@ -574,12 +595,6 @@ The default values of variables can be manipulated from the section
 .. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/template-default-values.png
   :alt: default values
 
-Global variables
-~~~~~~~~~~~~~~~~
-
-Variables can also be defined globally using the
-`OPENWISP_CONTROLLER_CONTEXT <#openwisp-controller-context>`_ setting.
-
 System defined variables
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -591,6 +606,9 @@ in read-only mode.
 .. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/system-defined-variables.png
    :alt: system defined variables
 
+**Note:** `Group configuration variables <#group-configuration-variables>`__
+are also added to the **System Defined Variables** of the device.
+
 Example usage of variables
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -935,12 +953,9 @@ Device Groups provide features aimed at adding specific management rules
 for the devices of an organization:
 
 - Group similar devices by having dedicated groups for access points, routers, etc.
-- Store additional information regarding a group in the structured metadata field
-  (which can be accessed via the REST API).
-- Customize structure and validation of metadata field of DeviceGroup to standardize
-  information across all groups using `"OPENWISP_CONTROLLER_DEVICE_GROUP_SCHEMA" <#openwisp-controller-device-group-schema>`_
-  setting.
+- Define `group metadata <#group-metadata>`_.
 - Define `group configuration templates <#group-templates>`_.
+- Define `group configuration variables <#group-configuration-variables>`__.
 
 .. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.1/device-groups.png
   :alt: Device Group example
@@ -983,6 +998,33 @@ to new devices.
 This feature works also when editing group templates or the group assigned
 to a device via the `REST API <#change-device-group-detail>`__.
 
+Group Configuration Variables
+#############################
+
+Groups allow to define configuration variables which are automatically
+added to the device's context in the **System Defined Variables**.
+Check the `"How to use configuration variables" section <#how-to-use-configuration-variables>`_
+to learn about precedence of different configuration variables.
+
+This feature works also when editing group templates or the group assigned
+to a device via the `REST API <#change-device-group-detail>`__.
+
+Group Metadata
+##############
+
+Groups allow to store additional information regarding a group in the
+structured metadata field (which can be accessed via the REST API).
+
+The metadata field allows custom structure and validation to standardize
+information across all groups using the
+`"OPENWISP_CONTROLLER_DEVICE_GROUP_SCHEMA" <#openwisp-controller-device-group-schema>`_
+setting.
+
+**Note:** *Group configuration variables* and *Group metadata* serves different purposes.
+The group configuration variables should be used when the device configuration is required
+to be changed for particular group of devices. Group metadata should be used to store
+additional data for the devices. Group metadata is not used for configuration generation.
+
 Export/Import Device data
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -3063,7 +3105,7 @@ while all the other organizations will have all command types enabled.
 | **default**: | ``{'type': 'object', 'properties': {}}`` |
 +--------------+------------------------------------------+
 
-Allows specifying JSONSchema used for validating meta-data of `Device Group <#device-groups>`_.
+Allows specifying JSONSchema used for validating meta-data of `Device Group <#device-groups>`__.
 
 ``OPENWISP_CONTROLLER_SHARED_MANAGEMENT_IP_ADDRESS_SPACE``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

openwisp_controller/config/admin.py

@@ -26,10 +26,7 @@
 from openwisp_ipam.filters import SubnetFilter
 from swapper import load_model
 
-from openwisp_controller.config.views import (
-    get_relevant_templates,
-    get_template_default_values,
-)
+from openwisp_controller.config.views import get_default_values, get_relevant_templates
 from openwisp_users.admin import OrganizationAdmin
 from openwisp_users.multitenancy import MultitenantOrgFilter
 from openwisp_utils.admin import (
@@ -665,6 +662,7 @@ def _get_preview_instance(self, request):
         c.device.name = request.POST.get('name')
         c.device.mac_address = request.POST.get('mac_address')
         c.device.key = request.POST.get('key')
+        c.device.group_id = request.POST.get('group') or None
         if 'hardware_id' in request.POST:
             c.device.hardware_id = request.POST.get('hardware_id')
         return c
@@ -677,9 +675,9 @@ def get_urls(self):
                 name='get_relevant_templates',
             ),
             path(
-                'get-template-default-values/',
-                self.admin_site.admin_view(get_template_default_values),
-                name='get_template_default_values',
+                'get-default-values/',
+                self.admin_site.admin_view(get_default_values),
+                name='get_default_values',
             ),
         ] + super().get_urls()
         for inline in self.inlines + self.conditional_inlines:
@@ -1002,14 +1000,7 @@ def save(self, *args, **kwargs):
 
     class Meta(BaseForm.Meta):
         model = DeviceGroup
-        widgets = {'meta_data': DeviceGroupJsonSchemaWidget}
-        labels = {'meta_data': _('Metadata')}
-        help_texts = {
-            'meta_data': _(
-                'Group meta data, use this field to store data which is related'
-                'to this group and can be retrieved via the REST API.'
-            )
-        }
+        widgets = {'meta_data': DeviceGroupJsonSchemaWidget, 'context': FlatJsonWidget}
 
 
 class DeviceGroupAdmin(MultitenantAdminMixin, BaseAdmin):
@@ -1026,6 +1017,7 @@ class DeviceGroupAdmin(MultitenantAdminMixin, BaseAdmin):
         'organization',
         'description',
         'templates',
+        'context',
         'meta_data',
         'created',
         'modified',

openwisp_controller/config/api/serializers.py

@@ -304,6 +304,7 @@ def get_queryset(self):
 
 
 class DeviceGroupSerializer(BaseSerializer):
+    context = serializers.JSONField(required=False, initial={})
     meta_data = serializers.JSONField(required=False, initial={})
     templates = FilterGroupTemplates(many=True)
     _templates = None
@@ -316,6 +317,7 @@ class Meta(BaseMeta):
             'organization',
             'description',
             'templates',
+            'context',
             'meta_data',
             'created',
             'modified',

openwisp_controller/config/base/config.py

@@ -558,7 +558,7 @@ def _has_device(self):
         return hasattr(self, 'device')
 
     def get_vpn_context(self):
-        context = super().get_context()
+        context = {}
         for vpnclient in self.vpnclient_set.all().select_related('vpn', 'cert'):
             vpn = vpnclient.vpn
             vpn_id = vpn.pk.hex
@@ -599,8 +599,11 @@ def get_context(self, system=False):
         additional context passed to netjsonconfig
         """
         c = collections.OrderedDict()
-        extra = {}
+        # Add global variables
+        context = super().get_context()
         if self._has_device():
+            # These pre-defined variables are needed at the start of OrderedDict.
+            # Hence, they are added separately.
             c.update(
                 [
                     ('name', self.name),
@@ -609,14 +612,20 @@ def get_context(self, system=False):
                     ('key', self.key),
                 ]
             )
-            if self.context and not system:
-                extra.update(self.context)
-        extra.update(self.get_vpn_context())
-        for func in self._config_context_functions:
-            extra.update(func(config=self))
-        if app_settings.HARDWARE_ID_ENABLED and self._has_device():
-            extra.update({'hardware_id': str(self.device.hardware_id)})
-        c.update(sorted(extra.items()))
+            if self.device._get_group():
+                # Add device group variables
+                context.update(self.device._get_group().get_context())
+            # Add predefined variables
+            context.update(self.get_vpn_context())
+            for func in self._config_context_functions:
+                context.update(func(config=self))
+            if app_settings.HARDWARE_ID_ENABLED:
+                context.update({'hardware_id': str(self.device.hardware_id)})
+
+        if self.context and not system:
+            context.update(self.context)
+
+        c.update(sorted(context.items()))
         return c
 
     def get_system_context(self):

openwisp_controller/config/base/device.py

@@ -127,6 +127,9 @@ def __str__(self):
     def _has_config(self):
         return hasattr(self, 'config')
 
+    def _has_group(self):
+        return hasattr(self, 'group')
+
     def _get_config_attr(self, attr):
         """
         gets property or calls method of related config object
@@ -143,6 +146,11 @@ def _get_config(self):
         else:
             return self.get_config_model()(device=self)
 
+    def _get_group(self):
+        if self._has_group():
+            return self.group
+        return self.get_group_model()(device=self)
+
     def get_context(self):
         config = self._get_config()
         return config.get_context()

openwisp_controller/config/base/device_group.py

@@ -1,4 +1,5 @@
 import collections
+from copy import deepcopy
 
 import jsonschema
 from django.core.exceptions import ValidationError
@@ -39,6 +40,22 @@ class AbstractDeviceGroup(OrgMixin, TimeStampedEditableModel):
         default=dict,
         load_kwargs={'object_pairs_hook': collections.OrderedDict},
         dump_kwargs={'indent': 4},
+        help_text=_(
+            'Group meta data, use this field to store data which is related'
+            ' to this group and can be retrieved via the REST API.'
+        ),
+        verbose_name=_('Metadata'),
+    )
+    context = JSONField(
+        blank=True,
+        default=dict,
+        load_kwargs={'object_pairs_hook': collections.OrderedDict},
+        dump_kwargs={'indent': 4},
+        help_text=_(
+            'This field can be used to add meta data for the group'
+            ' or to add "Configuration Variables" to the devices.'
+        ),
+        verbose_name=_('Configuration Variables'),
     )
 
     def __str__(self):
@@ -58,6 +75,9 @@ def clean(self):
         except SchemaError as e:
             raise ValidationError({'input': e.message})
 
+    def get_context(self):
+        return deepcopy(self.context)
+
     @classmethod
     def templates_changed(cls, instance, old_templates, templates, *args, **kwargs):
         group_templates_changed.send(

openwisp_controller/config/migrations/0036_device_group.py

@@ -62,6 +62,12 @@ class Migration(migrations.Migration):
                         default=dict,
                         dump_kwargs={'ensure_ascii': False, 'indent': 4},
                         load_kwargs={'object_pairs_hook': collections.OrderedDict},
+                        help_text=(
+                            'Group meta data, use this field to store data which is'
+                            ' related to this group and can be retrieved via the'
+                            ' REST API.'
+                        ),
+                        verbose_name='Metadata',
                     ),
                 ),
                 (

openwisp_controller/config/migrations/0049_devicegroup_context.py

@@ -0,0 +1,31 @@
+# Generated by Django 3.2.19 on 2023-06-27 14:45
+
+import collections
+
+import jsonfield.fields
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('config', '0048_wifi_radio_band_migration'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='devicegroup',
+            name='context',
+            field=jsonfield.fields.JSONField(
+                blank=True,
+                default=dict,
+                dump_kwargs={'ensure_ascii': False, 'indent': 4},
+                help_text=(
+                    'This field can be used to add meta data for the group'
+                    ' or to add "Configuration Variables" to the devices.'
+                ),
+                load_kwargs={'object_pairs_hook': collections.OrderedDict},
+                verbose_name='Configuration Variables',
+            ),
+        ),
+    ]