Merge remote-tracking branch 'origin/master' into 1990_handle_importe…

…d_variable_in_kernel_extraction

changelog

@@ -579,6 +579,17 @@
 	same as Python keywords when using sympy within PSyclone, e.g. in
 	comparisons between expressions.
 
+	195) PR #2295 for #2294. Extends the PSyData extraction library
+	to support character variables
+
+	196) PR #2157 for #1396. Add tree-update mechanism and new ACCClause
+	nodes for data movement.
+
+	197) PR #2281 for #2258. Adds a PSyData wrapper for the TAU profiler.
+
+	198) PR #2283 for #2282 and #2286. Adds a get_ubound_expression
+	method to ArrayMixin (analagous to get_lbound_expression).
+
 release 2.3.1 17th of June 2022
 
 	1) PR #1747 for #1720. Adds support for If blocks to PSyAD.

doc/developer_guide/psy_data.rst

@@ -72,7 +72,7 @@ the same time as checking that read-only values are indeed not modified,
 different module names and data types must be used.
 
 PSyData divides its application into different classes. For example,
-the class "profile" is used for all profiling tools (e.g. DrHook or the
+the class "profile" is used for all profiling tools (e.g. TAU, DrHook or the
 NVIDIA profiling tools). This class name is used as a prefix for
 the module name, the ``PSyDataType`` and functions. So if a profiling application
 is linked the above code will actually look like this::
@@ -100,8 +100,8 @@ The class prefixes supported at the moment are:
 ======================= =======================================================
 Class Prefix            Description
 ======================= =======================================================
-profile                 All libraries related to profiling tools like DrHook,
-                        NVIDIA's profiling tools etc. See
+profile                 All libraries related to profiling tools like TAU,
+                        DrHook, NVIDIA's profiling tools etc. See
                         :ref:`user_guide:profiling` for details.
 extract                 For libraries used for kernel data extraction. See
                         :ref:`user_guide:psyke` for details.
@@ -654,6 +654,8 @@ takes the following parameters:
         64-bit integer value
     ``logical``:
         32-bit logical value
+    ``char``:
+        A default string value
 
     Default value is ``real,double,int``.
 
@@ -967,7 +969,7 @@ an index used by the profiling tool in ``profile_PSyDataType``, or
 by storing pointers to the profiling data to be able to
 print all results in a ProfileFinalise() subroutine.
 Some of the wrapper libraries use the PSyData base class (e.g. dl_timer,
-simple_timing, template), others do not (e.g. NVIDIA profiling,
+simple_timing, template), others do not (e.g. NVIDIA profiling, TAU,
 DrHook wrapper).
 
 

doc/developer_guide/psyir.rst

@@ -192,7 +192,8 @@ For a full list of methods available in any PSyIR node see the
     To achieve this, we sub-classed the Python list and redefined all
     methods that modify the list by calling first the PSyIR provided
     validation method and subsequently, if valid, calling the associated
-    list method.
+    list method and triggering an 'update' signal (see
+    :ref:`update_signals_label`).
 
 .. _nodesinfo-label:
 
@@ -256,6 +257,26 @@ relationship.
 Methods like ``node.detach()``, ``node.copy()`` and ``node.pop_all_children()``
 can be used to move or replicate existing children into different nodes. 
 
+.. _update_signals_label:
+
+Dynamic Tree Updates
+====================
+
+Certain modifications to a PSyIR tree will require that parent nodes
+also be updated. For instance, if nodes are added to or removed from
+an OpenACC data region, then the clauses describing the
+necessary data movement (to/from the accelerator device) may have to
+change. To support such use cases, the PSyIR Node has the
+``update_signal`` method which is used to signal that the tree has
+been modified. This signal is propagated up the tree (i.e. from parent
+to parent). The default handler for this signal, ``Node._update_node``, does
+nothing. If a sub-class must take action when the tree below it is
+modified then it must override the ``_update_node`` method as appropriate.
+
+Note that the signalling mechanism is fully contained within the ``Node``
+class and takes care of avoiding recursive updates to the same Node instance.
+It should therefore only be necessary for a class to implement the
+``_update_node`` handler.
 
 Selected Node Descriptions
 ==========================

doc/user_guide/examples.rst

@@ -265,7 +265,7 @@ Example 5.2: Profiling
 This example shows how to use the profiling support in PSyclone.
 It instruments two invoke statements and can link in with any
 of the following profiling wrapper libraries: template,
-simple_timer, dl_timer, and DrHook (see
+simple_timer, dl_timer, TAU, and DrHook (see
 :ref:`profiling_third_party_tools`). The ``README.md``
 file contains detailed instructions on how to build the
 different executables. By default (i.e. just using ``make``

doc/user_guide/libraries.rst

@@ -70,7 +70,7 @@ Profiling
 ^^^^^^^^^
 
 PSyclone provides wrapper libraries for some common performance profiling
-tools, such as dl_timer and Dr Hook. More information can be found in
+tools, such as dl_timer, TAU, and Dr Hook. More information can be found in
 the :ref:`Profiling <profiling>` section.
 
 Profiling libraries are located in the ``lib/profiling`` `directory

doc/user_guide/profiling.rst

@@ -51,7 +51,7 @@ transformation within a transformation script.
 
 
 PSyclone can be used with a variety of existing profiling tools.
-It currently supports dl_timer, Dr Hook, the NVIDIA GPU profiling
+It currently supports dl_timer, TAU, Dr Hook, the NVIDIA GPU profiling
 tools and it comes with a simple stand-alone timer library. The
 :ref:`PSyData API <psy_data>` (see also the
 :ref:`Developer Guide <dev_guide:psy_data>`)
@@ -77,7 +77,7 @@ Interface to Third Party Profiling Tools
 ----------------------------------------
 
 PSyclone comes with :ref:`wrapper libraries <libraries>` to support
-usage of Dr Hook, dl_timer, NVTX (NVIDIA Tools Extension library),
+usage of TAU, Dr Hook, dl_timer, NVTX (NVIDIA Tools Extension library),
 and a simple non-thread-safe timing library. Support for further
 profiling libraries will be added in the future. To compile the
 wrapper libraries, change into the directory ``lib/profiling``
@@ -112,6 +112,10 @@ libraries that come with PSyclone:
     support. Additional link options might therefore be required
     (e.g. enabling OpenMP, or linking with MPI).
 
+``lib/profiling/tau``
+    This wrapper uses TAU profiling and tracing toolkit. It can be
+    downloaded from ``https://www.cs.uoregon.edu/research/tau``.
+
 ``lib/profiling/drhook``
     This wrapper uses the Dr Hook library. You need to contact
     ECMWF to obtain a copy of Dr Hook.
@@ -145,9 +149,10 @@ module. The functions that need to be implemented are described in
 the developer's guide (:ref:`dev_guide:psy_data`).
 
 Most libraries in ``lib/profiling`` need to be linked in
-with the corresponding 3rd party profiling tool. The
-exceptions are the template and simple_timing libraries,
-which are stand alone. The profiling example in
+with the corresponding 3rd party profiling tool, or use a compiler
+wrapper provided by the tool which will provide the required additional
+compiler parameters. The exceptions are the template and simple_timing
+libraries, which are stand alone. The profiling example in
 ``examples/gocean/eg5/profile`` can be used with any of the
 wrapper libraries (except ``nvidia``) to see how they work.
 

examples/gocean/eg5/profile/Makefile

@@ -65,7 +65,7 @@ run: compile
 
 compile: transform template
 transform: psy.f90
-all: template simple_timing drhook dl_timer lfric
+all: template simple_timing drhook dl_timer lfric tau
 
 template:
 	$(MAKE) WRAPPER_DIR="$(PROFILE_DIR)/template" \
@@ -87,6 +87,16 @@ drhook:
 			PROFILE_LINK="-L $(DRHOOK_DIR)/build/lib -ldrhook" \
 	        drhook_lib $(NAME).drhook
 
+TAU_DIR?=$(ROOT_DIR)/../tau
+# By adding the dependency to INF_LIB here, we will be invoking the original
+# compiler, not tau_f90.sh (since we don't want to instrument the whole
+# infrastructure library with tau)
+tau: $(INF_LIB)
+	$(MAKE) WRAPPER_DIR="$(PROFILE_DIR)/tau"                \
+	        WRAPPER_NAME="tau_psy"                          \
+			PROFILE_LINK="-L $(TAU_DIR)/build/lib"  \
+	        tau_lib $(NAME).tau
+
 # The dl-timer library is available from:
 # https://bitbucket.org/apeg/dl_timer/src/master/
 # As default assume dl_timer is installed 'next' to PSyclone.
@@ -112,6 +122,11 @@ lfric:
 # in order to preserve temporary files:
 #.SECONDARY:
 
+$(NAME).tau: $(INF_LIB) $(WRAPPER_DIR)/lib$(WRAPPER_NAME).a $(KERNELS) psy.o alg.o
+	tau_f90.sh $(F90FLAGS) $(KERNELS) psy.o alg.o -o $@                             \
+		   -L $(WRAPPER_DIR) -l $(WRAPPER_NAME)                                 \
+		   $(PROFILE_LINK) $(INF_LIB)
+
 $(NAME).%: $(INF_LIB) $(WRAPPER_DIR)/lib$(WRAPPER_NAME).a $(KERNELS) psy.o alg.o
 	$(F90) $(F90FLAGS) $(KERNELS) psy.o alg.o -o $@                             \
 		   -L $(WRAPPER_DIR) -l $(WRAPPER_NAME)                                 \
@@ -131,6 +146,9 @@ $(INF_LIB):
 drhook_lib :
 	$(MAKE) DRHOOK_ROOT=$(DRHOOK_DIR) -C $(WRAPPER_DIR)
 
+tau_lib :
+	$(MAKE) F90=tau_f90.sh TAU_ROOT=$(TAU_DIR) -C $(WRAPPER_DIR)
+
 dl_timer_lib:
 	$(MAKE) DL_TIMER_ROOT=$(DL_TIMER_ROOT) -C $(WRAPPER_DIR)
 
@@ -146,10 +164,10 @@ $(DEP_LIB):
 # Special rule to create psy_LIBNAME.o for each wrapper library (required
 # since PSyDataType will differ from one wrapper library to the next)
 psy.o: psy.f90 $(KERNELS)
-	$(F90) -o $@ $(F90FLAGS) -I $(INF_INC) -I $(WRAPPER_DIR) -c $<
+	$(F90) -o $@ $(F90FLAGS) -I$(INF_INC) -I$(WRAPPER_DIR) -c $<
 
 %.o: %.f90
-	$(F90) $(F90FLAGS) -I $(INF_INC) -I $(WRAPPER_DIR) -c $<
+	$(F90) $(F90FLAGS) -I$(INF_INC) -I$(WRAPPER_DIR) -c $<
 
 allclean: clean
 	${MAKE} -C $(PROFILE_DIR) allclean

examples/gocean/eg5/profile/README.md

@@ -16,6 +16,7 @@ profile wrapper libraries:
 - dl_timer
 - drhook
 - lfric
+- tau
 
 By default (``make`` without an argument) the ``template`` library will 
 be used, which just prints the name of the regions called.
@@ -37,6 +38,11 @@ uses the "template" profiling library in ``lib/profiling/template``.
 More detailed instructions for compiling these libraries are are given in
 the corresponding subdirectories.
 
+If you are using the TAU profiling library, you need to install
+it yourself, and make sure that the ``tau_f90.sh`` compiler wrapper
+is in your path. The Makefile will automatically call ``tau_f90.sh``, there
+is no need to set ``$F90`` in this case.
+
 If you are using ``dl_timer`` or ``drhook``, you need to compile these
 libraries yourself first, and modify the ``Makefile`` in this directory
 to specify the required linking parameters. The ``Makefile``
@@ -97,7 +103,7 @@ library, you should see:
 
 BSD 3-Clause License
 
-Copyright (c) 2020-2021, Science and Technology Facilities Council.
+Copyright (c) 2020-2023, Science and Technology Facilities Council.
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

examples/gocean/eg5/profile/test.x90

@@ -80,4 +80,4 @@ Program test
 
     print *,a_fld%data(1:5, 1:5)
     call profile_PSyDataShutdown()
-end program test
+end program test

lib/extract/netcdf/read_kernel_data_mod.f90

@@ -256,10 +256,7 @@ subroutine ReadArray1dChar(this, name, value)
         endif
 
         retval = CheckError(nf90_inq_varid(this%ncid, name, varid))
-        ! Initialise it with 0, so that an array comparison will work
-        ! even though e.g. boundary areas or so might not be set at all.
-        ! The compiler will convert the double precision value to the right
-        ! type (e.g. int or single precision).
+        ! Initialise the whole array with "".
         value = ""
         retval = CheckError(nf90_get_var(this%ncid, varid, value))
 
@@ -312,10 +309,7 @@ subroutine ReadArray2dChar(this, name, value)
         endif
 
         retval = CheckError(nf90_inq_varid(this%ncid, name, varid))
-        ! Initialise it with 0, so that an array comparison will work
-        ! even though e.g. boundary areas or so might not be set at all.
-        ! The compiler will convert the double precision value to the right
-        ! type (e.g. int or single precision).
+        ! Initialise the whole array with "".
         value = ""
         retval = CheckError(nf90_get_var(this%ncid, varid, value))
 
@@ -372,10 +366,7 @@ subroutine ReadArray3dChar(this, name, value)
         endif
 
         retval = CheckError(nf90_inq_varid(this%ncid, name, varid))
-        ! Initialise it with 0, so that an array comparison will work
-        ! even though e.g. boundary areas or so might not be set at all.
-        ! The compiler will convert the double precision value to the right
-        ! type (e.g. int or single precision).
+        ! Initialise the whole array with "".
         value = ""
         retval = CheckError(nf90_get_var(this%ncid, varid, value))
 
@@ -436,10 +427,7 @@ subroutine ReadArray4dChar(this, name, value)
         endif
 
         retval = CheckError(nf90_inq_varid(this%ncid, name, varid))
-        ! Initialise it with 0, so that an array comparison will work
-        ! even though e.g. boundary areas or so might not be set at all.
-        ! The compiler will convert the double precision value to the right
-        ! type (e.g. int or single precision).
+        ! Initialise the whole array with "".
         value = ""
         retval = CheckError(nf90_get_var(this%ncid, varid, value))
 

lib/extract/netcdf/read_kernel_data_mod.jinja

@@ -289,6 +289,10 @@ contains
         value = tmp == 1
         deallocate(tmp)
         {% else %}
+        {% if name == "Char" %}
+        ! Initialise the whole array with "".
+        value = ""
+        {% else %}
         ! Initialise it with 0, so that an array comparison will work
         ! even though e.g. boundary areas or so might not be set at all.
         ! The compiler will convert the double precision value to the right