#2711 DoF Kernel Code Generation

doc/developer_guide/APIs.rst

@@ -48,29 +48,29 @@ TBD
 
 .. Generating API-specific code
 .. ============================
-.. 
+..
 .. This section explains how to create a new API in PSyclone. PSyclone
 .. currently supports the following APIs: lfric and gocean.
-.. 
+..
 .. config.py
 .. ---------
-.. 
+..
 .. The names of the supported APIs and the default API are specified in
 .. ``configuration.py``. When adding a new API you must add the name you would like
 .. to use to the ``_supported_api_list``.
-.. 
+..
 .. parse.py
 .. --------
-.. 
+..
 .. The parser reads the algorithm code and associated kernel metadata.
-.. 
+..
 .. The parser currently assumes that all APIs will use the ``invoke()``
 .. API for the algorithm-to-psy layer but that the content and structure
 .. of the metadata in the kernel code may differ. If the algorithm API
 .. differs, then the parser will need to be refactored. This is beyond
 .. the scope of this document and is currently not considered in the
 .. PSyclone software architecture.
-.. 
+..
 .. The kernel metadata however, will be different from one API to
 .. another. To parse this kernel-API-specific metadata a
 .. ``KernelTypeFactory`` is provided which should return the appropriate
@@ -79,7 +79,7 @@ TBD
 .. in the ``KernelTypeFactory`` class. If the kernel metadata happens to be
 .. the same as another existing API then the existing ``KernelType``
 .. subclass can be used for the new API.
-.. 
+..
 .. The ``KernelType`` subclass needs to specialise the class constructor
 .. and initialise the ``KernelType`` base class with the
 .. supplied arguments. The role of the ``KernelType`` subclass is to create
@@ -88,95 +88,95 @@ TBD
 .. this is appends the kernel-metadata-specific subclass instance is
 .. appended to the ``_arg_descriptors`` list provided by the ``KernelType``
 .. base class.
-.. 
+..
 .. TBC
-.. 
+..
 .. This information
-.. 
+..
 .. KernelType base class assumes kernel metadata stored as a type. Searches for that type.
 .. Checks whether the metadata is public (it should be ?)
 .. Assumes iterates_over variable.
 .. Binding to a procedure - assumes one of two styles.
 .. Assumes a meta_args type
 .. *What about our func_args type???*
-.. 
+..
 .. type x
 .. meta_args=
 .. *meta_func=*
 .. iterates_over=
 .. code => or code =
 .. end type x
-.. 
+..
 .. The descriptor class ...
-.. 
+..
 .. psyGen.py
 .. ---------
-.. 
+..
 .. factory
 .. +++++++
-.. 
+..
 .. A new file needs to be created and the following classes found in
 .. psyGen.py need to be subclassed.
-.. 
+..
 .. PSy, Invokes, Invoke, InvokeSchedule, Loop, Kern, Arguments, Argument
 .. You may also choose to subclass the Inf class if required.
-.. 
+..
 .. The subclass of the PSy class then needs to be added as an option to
 .. the create method in the PSyFactory class.
-.. 
+..
 .. Initialisation
 .. ++++++++++++++
-.. 
+..
 .. The parser information passed to the PSy layer is used to create an
 .. invokes object which in turn creates a list of invoke objects. Each
 .. invoke object contains an InvokeSchedule which consists of loops and
 .. calls. Finally, a call contains an arguments object which itself
 .. contains a list of argument objects.
-.. 
+..
 .. To make sure the subclass versions of the above objects are created
 .. the __init__ methods of the subclasses must make sure they create
 .. the appropriate objects.
-.. 
+..
 .. Some of the baseclass constructors (__init__ methods) support the
 .. classname being provided. This allow them to instantiate the
 .. appropriate objects without knowing what they are.
-.. 
+..
 .. gen_code()
 .. ++++++++++
-.. 
+..
 .. All of the above classes (with the exception of PSy which supports a
 .. gen() method) have the gen_code() method. This method passes the
 .. parent of the generation tree and expect the object to add the code
 .. associated with the object as a child of the parent. The object is
 .. then expected to call any children. This approach is powerful as it
 .. lets each object concentrate on the code that it is responsible for.
-.. 
+..
 .. Adding code in gen_code()
 .. +++++++++++++++++++++++++
-.. 
+..
 .. The f2pygen classes have been developed to help create appropriate
 .. fortran code in the gen_code() method.
-.. 
+..
 .. When writing a gen_code() method for a particular object and API it is
 .. natural to add code as a child of the parent provided by the callee of
 .. the method. However, in some cases we do not want code to appear at
 .. the current position in the hierarchy.
-.. 
+..
 .. The add() method
 .. ++++++++++++++++
-.. 
+..
 .. PSyclone supports this via the add() method
-.. 
+..
 .. explicitly place at the appropriate place in the hierarchy. For example,
 .. parent.parent.add(...)
-.. 
+..
 .. optional argument. default is auto. This attempts to place code in the
 .. expected place. For example, specify a declaration. auto finds a
 .. correct place to put this code.
-.. 
+..
 .. Specify position explicitly
 .. "before", "after", "first", "last"
-.. 
+..
 .. Sometimes don't know exactly where to place. On example that is
 .. supported is when you want to add something before or after a loop
 .. nest. start_parent_loop(). This method recurses up until the parent is
@@ -333,7 +333,7 @@ grey) then we get:
 
 .. image:: dofs_cont_halos.png
 	   :width: 230
-		   
+
 An example for a depth-1 halo implementation with the earlier mesh
 split into 2 partitions is given below, with the halo cells drawn in
 grey and halo dofs coloured red. An example local indexing scheme is
@@ -403,9 +403,9 @@ Loop iterators
 --------------
 
 In the current implementation of the LFRic API it is possible to
-iterate (loop) either over cells or dofs. At the moment all coded
-kernels are written to iterate over cells and all Built-in kernels are
-written to iterate over dofs, but that does not have to be the case.
+iterate (loop) either over cells or dofs. At the moment coded
+kernels can be written to iterate over cells or dofs and all Built-in kernels
+are written to iterate over dofs, but that does not have to be the case.
 
 The loop iteration information is specified in the kernel metadata. In
 the case of Built-ins there is kernel metadata but it is part of
@@ -994,7 +994,7 @@ If an application is being built in parallel then it is possible that
 different invocations of PSyclone will happen simultaneously and
 therefore we must take care to avoid race conditions when querying the
 filesystem. For this reason we use ``os.open``::
-  
+
     fd = os.open(<filename>, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
 
 The ``os.O_CREATE`` and ``os.O_EXCL`` flags in combination mean that
@@ -1020,7 +1020,7 @@ of a given colour may be safely updated in parallel
 	   Example of the colouring of the horizontal cells used to
 	   ensure the thread-safe update of shared dofs (black
 	   circles).  (Courtesy of S. Mullerworth, Met Office.)
-	   
+
 The loop over colours must then be performed sequentially but the loop
 over cells of a given colour may be done in parallel. A loop that
 requires colouring may be transformed using the ``Dynamo0p3ColourTrans``
@@ -1206,13 +1206,13 @@ TBD
 
 .. OpenMP Support
 .. --------------
-.. 
+..
 .. Loop directives are treated as first class entities in the psyGen
 .. package. Therefore they can be added to psyGen's high level
 .. representation of the fortran code structure in the same way as calls
 .. and loops. Obviously it is only valid to add a loop directive outside
 .. of a loop.
-.. 
+..
 .. When adding a call inside a loop the placement of any additional calls
 .. or declarations must be specified correctly to ensure that they are
 .. placed at the correct location in the hierarchy. To avoid accidentally
@@ -1222,7 +1222,7 @@ TBD
 .. f2pygen*.  This method returns the location at the top of any loop
 .. hierarchy and before any comments immediately before the top level
 .. loop.
-.. 
+..
 .. The OpenMPLoopDirective object needs to know which variables are
 .. shared and which are private. In the current implementation default
 .. shared is used and private variables are listed. To determine the
@@ -1233,13 +1233,13 @@ TBD
 .. the directive and adds each calls list of private variables, returned
 .. with the local_vars() method. Therefore the OpenMPLoopDirective object
 .. relies on calls specifying which variables they require being local.
-.. 
+..
 .. Next ...
-.. 
+..
 .. Update transformation for colours
-.. 
-.. OpenMPLoop transformation in transformations.py. 
-.. 
+..
+.. OpenMPLoop transformation in transformations.py.
+..
 .. Create third transformation which goes over all loops in a schedule and
 .. applies the OpenMP loop transformation.
 

doc/user_guide/dynamo0p3.rst

@@ -2496,9 +2496,7 @@ with PSyclone's naming conventions, are:
 
    3) Include the unique number of degrees of freedom for the function space.
       This is an ``integer`` of kind ``i_def`` with intent ``in``. The name of
-      this argument is simply ``undf`` without a function space suffix (as for
-      general purpose kernels) since all fields will be on the same function
-      space.
+      this argument is ``"undf_"<field_function_space>``.
 
 .. _lfric-kernel-arg-intents:
 

src/psyclone/domain/lfric/function_space.py

@@ -238,13 +238,6 @@ def undf_name(self):
         '''
         return "undf_" + self.mangled_name
 
-    @property
-    def bare_undf_name(self):
-        ''':returns: undf with no FunctionSpace object name.
-        :rtype: str
-        '''
-        return "undf"
-
     def get_basis_name(self, qr_var=None, on_space=None):
         '''
         Returns a name for the basis function on this FunctionSpace. If

src/psyclone/domain/lfric/kern_call_acc_arg_list.py

@@ -43,7 +43,7 @@
 '''
 
 from psyclone import psyGen
-from psyclone.domain.lfric import KernCallArgList
+from psyclone.domain.lfric import KernCallArgList, LFRicConstants
 from psyclone.errors import InternalError
 
 
@@ -93,6 +93,27 @@ def cell_position(self, var_accesses=None):
         _, ref = self.cell_ref_name(var_accesses)
         self.append(ref.symbol.name)
 
+    def field(self, arg, var_accesses=None):
+        '''Add the field array associated with the argument 'arg' to the
+        argument list. If supplied it also stores this access in var_accesses.
+
+        :param arg: the field to be added.
+        :type arg: :py:class:`psyclone.dynamo0p3.DynKernelArgument`
+        :param var_accesses: optional VariablesAccessInfo instance to store
+            the information about variable accesses.
+        :type var_accesses: :py:class:`psyclone.core.VariablesAccessInfo`
+
+        '''
+        const = LFRicConstants()
+        suffix = const.ARG_TYPE_SUFFIX_MAPPING[arg.argument_type]
+        # Look-up the name of the variable that stores the reference to
+        # the data in this field.
+        sym = self._symtab.lookup_with_tag(f"{arg.name}:{suffix}")
+
+        # Add the field data array as being read.
+        self.append(sym.name, var_accesses, var_access_name=sym.name,
+                    mode=arg.access, metadata_posn=arg.metadata_index)
+
     def stencil(self, arg, var_accesses=None):
         '''Add general stencil information associated with the argument 'arg'
         to the argument list. OpenACC requires the full dofmap to be

src/psyclone/domain/lfric/kern_call_arg_list.py

@@ -48,7 +48,6 @@
 from psyclone.core import AccessType, Signature
 from psyclone.domain.lfric import ArgOrdering, LFRicConstants
 # Avoid circular import:
-from psyclone.domain.lfric.lfric_builtins import LFRicBuiltIn
 from psyclone.domain.lfric.lfric_types import LFRicTypes
 from psyclone.errors import GenerationError, InternalError
 from psyclone.psyir.nodes import ArrayReference, Reference, StructureReference
@@ -342,10 +341,9 @@ def field_vector(self, argvect, var_accesses=None):
 
         :param argvect: the field vector to add.
         :type argvect: :py:class:`psyclone.dynamo0p3.DynKernelArgument`
-        :param var_accesses: optional VariablesAccessInfo instance to store \
+        :param var_accesses: optional VariablesAccessInfo instance to store
             the information about variable accesses.
-        :type var_accesses: \
-            :py:class:`psyclone.core.VariablesAccessInfo`
+        :type var_accesses: :py:class:`psyclone.core.VariablesAccessInfo`
 
         '''
         suffix = LFRicConstants().ARG_TYPE_SUFFIX_MAPPING[
@@ -382,16 +380,15 @@ def field(self, arg, var_accesses=None):
         # the data in this field.
         sym = self._symtab.lookup_with_tag(f"{arg.name}:{suffix}")
 
-        if self._kern.iterates_over == "dof" and not isinstance(
-          self._kern, LFRicBuiltIn):
+        if self._kern.iterates_over == "dof":
             # If dof kernel, add access to the field by dof ref
             dof_sym = self._symtab.find_or_create_integer_symbol(
                 "df", tag="dof_loop_idx")
             self.append_array_reference(sym.name, [Reference(dof_sym)],
                                         ScalarType.Intrinsic.INTEGER,
                                         symbol=sym)
             # Then append our symbol
-            name = f"{sym.name}({Reference(dof_sym)})"
+            name = f"{sym.name}({dof_sym.name})"
             self.append(name, var_accesses, var_access_name=sym.name)
         else:
             # Add the field data array as being read.
@@ -624,10 +621,7 @@ def fs_compulsory_field(self, function_space, var_accesses=None):
             :py:class:`psyclone.core.VariablesAccessInfo`
 
         '''
-        if self._kern.iterates_over == "dof":
-            sym = self.append_integer_reference(function_space.bare_undf_name)
-        else:
-            sym = self.append_integer_reference(function_space.undf_name)
+        sym = self.append_integer_reference(function_space.undf_name)
         self.append(sym.name, var_accesses)
 
         map_name = function_space.map_name

src/psyclone/domain/lfric/kern_stub_arg_list.py

@@ -328,7 +328,6 @@ def fs_compulsory_field(self, function_space, var_accesses=None):
 
         '''
         self.append(function_space.undf_name, var_accesses)
-
         self.append(function_space.map_name, var_accesses)
 
     def basis(self, function_space, var_accesses=None):

src/psyclone/domain/lfric/lfric_kern.py

@@ -557,6 +557,19 @@ def base_name(self):
         '''
         return self._base_name
 
+    @property
+    def undf_name(self):
+        '''
+        Dynamically looks up the name of the 'undf' variable for the
+        space that this kernel updates.
+
+        :returns: the name of the undf variable.
+        :rtype: str
+
+        '''
+        field = self._arguments.iteration_space_arg()
+        return field.function_space.undf_name
+
     @property
     def argument_kinds(self):
         '''