Merge branch 'EasyInstall'

.github/workflows/cmake-build-test.yml

@@ -25,7 +25,7 @@ jobs:
           - os: ubuntu-latest
             mpi: 'openmpi'
             check_shared_lib: ldd lib/libyt.so
-          - os: macos-latest
+          - os: macos-12
             mpi: 'mpich'
             check_shared_lib: otool -l lib/libyt.dylib
         python-version: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
@@ -38,7 +38,7 @@ jobs:
         run: cmake --version
 
       - name: Setup Python ${{ matrix.python-version }} environment
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
           architecture: 'x64'

.github/workflows/example-test-run.yml

@@ -24,7 +24,7 @@ jobs:
         platform:
           - os: ubuntu-latest
             mpi: 'openmpi'
-          - os: macos-latest
+          - os: macos-12
             mpi: 'mpich'
         python-version: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
 
@@ -36,7 +36,7 @@ jobs:
         run: cmake --version
 
       - name: Setup Python ${{ matrix.python-version }} environment
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
           architecture: 'x64'
@@ -52,12 +52,7 @@ jobs:
 
       - name: Install yt, mpi4py, and yt_libyt
         run: |
-          pip install mpi4py yt
-          cd ${{ github.workspace }}
-          cd ..
-          git clone --depth 1 -b libyt-dev --single-branch "https://github.com/cindytsai/yt_libyt.git" yt_libyt
-          cd yt_libyt
-          pip install .
+          pip install mpi4py yt yt_libyt
 
       - name: Update GitHub Environment Variables
         run: |

.pre-commit-config.yaml

@@ -12,7 +12,7 @@ ci:
 
 repos:
   - repo: https://github.com/pre-commit/mirrors-clang-format
-    rev: v18.1.1
+    rev: v18.1.2
     hooks:
       - id: clang-format
         types_or: ['c++', 'c']

CMakeLists.txt

@@ -35,7 +35,7 @@ option(BUILD_SHARED_LIBS "Building using shared libraries"
 
 ## find dependencies ##
 set(Python_ROOT_DIR ${PYTHON_PATH})
-find_package(Python COMPONENTS Development NumPy REQUIRED)
+find_package(Python 3.7 COMPONENTS Development NumPy REQUIRED)
 
 if (NOT SERIAL_MODE)
     set(MPI_HOME ${MPI_PATH})

README.md

@@ -1,10 +1,8 @@
 # libyt
-[![CMake Build Test](https://github.com/cindytsai/libyt/actions/workflows/cmake-build-test.yml/badge.svg?branch=main)](https://github.com/cindytsai/libyt/actions/workflows/cmake-build-test.yml)
-[![Example Test Run](https://github.com/cindytsai/libyt/actions/workflows/example-test-run.yml/badge.svg?branch=main)](https://github.com/cindytsai/libyt/actions/workflows/example-test-run.yml)
+[![CMake Build Test](https://github.com/yt-project/libyt/actions/workflows/cmake-build-test.yml/badge.svg?branch=main)](https://github.com/yt-project/libyt/actions/workflows/cmake-build-test.yml)
+[![Example Test Run](https://github.com/yt-project/libyt/actions/workflows/example-test-run.yml/badge.svg?branch=main)](https://github.com/yt-project/libyt/actions/workflows/example-test-run.yml)
 [![Documentation Status](https://readthedocs.org/projects/libyt/badge/?version=latest)](https://libyt.readthedocs.io/en/latest/?badge=latest)
 
-**_This repo is only for development purpose._**
-
 `libyt` is an open source C library for simulation, that allows researchers to analyze and visualize data using [`yt`](https://yt-project.org/) or other Python packages in parallel during simulation runtime. In this way, we can skip the step of writing data to local disk before doing analysis using Python. This greatly reduce the disk usage, and increase the temporal resolution.
 
 - **Documents**: https://libyt.readthedocs.io/

doc/FAQs.md

@@ -0,0 +1,82 @@
+# FAQs
+
+## When Installing
+
+### Get errors when using CMake
+
+Make sure the folder where CMake generates build files is empty or not exist yet by removing the folder:
+```bash
+cd libyt
+rm -rf <build-folder>
+cmake -S . -B <build-folder>
+```
+
+### Get errors when linking shared library libyt.so
+
+If we compile Python from source, and we get errors like this when compiling `libyt`:
+```text
+ld: <some-path-to-python-library>: relocation R_X86_64_32S against symbol `_Py_FalseStruct' can not be used when making a shared object; recompile with -fPIC
+ld: failed to set dynamic section sizes: bad value
+```
+We need to re-compile and install Python and add `--enable-shared` when configuring.
+
+The simplest way to avoid this is to install Python from package manager.
+
+### Unable to link to dependencies fetched by libyt after installation
+
+Keep `libyt` project repo after installation. `libyt` fetches and stores dependencies under `libyt/vendor` folder, so that the content can be reused in different builds.
+
+---
+
+## When Running Applications
+
+### How does libyt run Python script?
+`libyt` runs Python script synchronously, which means every MPI process runs the same piece of Python code.
+They do the job together under the process space of MPI tasks using [`mpi4py`](https://mpi4py.readthedocs.io/en/stable/index.html).
+
+### Have problems when running libyt in parallel mode
+
+If we happen to get errors related to one-sided MPI, for example:
+```text
+*** An error occurred in MPI_Win_attach
+*** reported by process [3353411585,1]
+*** on win rdma window 3
+*** MPI_ERR_RMA_ATTACH: Could not attach RMA segment
+*** MPI_ERRORS_ARE_FATAL (processes in this win will now abort,
+***    and potentially your MPI job)
+3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal
+Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
+```
+Remember to set `OMPI_MCA_osc=sm,pt2pt` before running the application. It is for one-sided MPI communication. For example:
+
+```bash
+OMPI_MCA_osc=sm,pt2pt mpirun -np 3 ./example
+```
+
+### Why does my program hang and how do I solve it?
+Though `libyt` can execute any Python module, when it comes to reading simulation data, it requires every MPI process to participate.
+The program hanging problem is due to only some MPI processes are accessing the data, but not all of them.
+
+**Please do**:
+1. Check if there is an `if` statements that makes MPI processes non-symmetric. For example, only root process runs the statement:
+    ```python
+    def func():
+        if yt.is_root():
+            ...  # <-- This statement only executes in MPI root rank
+    ```
+2. Move the statement out of `if yt.is_root()` (for the case here).
+
+> {octicon}`calendar;1em;sd-text-secondary;` When accessing simulation data, `libyt` requires every process to participate.
+> We are working on this in both `yt` and `libyt`.
+
+### Have Problems when Using Interactive Prompt
+
+#### Why can't I find the prompt `>>>`?
+`>>> `  is probably immersed inside the output.
+We can hit enter again, which is to provide an empty statement, and it will come out.
+
+We can make prompt more smoothly by setting [`YT_VERBOSE`](./libyt-api/yt_initialize.md#yt_param_libyt) to `YT_VERBOSE_INFO`.
+
+#### Where can I use interactive mode?
+`libyt` interactive Python prompt only works on local machine or submit the job to HPC platforms using interactive jobs like `qsub -I` in PBS scheduler.
+The reason is that the user interface is exposed in the terminal.

doc/debug-and-profiling/time-profiling.md

@@ -2,7 +2,7 @@
 
 ## How to Configure
 
-Compile `libyt` with [`-DSUPPORT_TIMER=ON`](../how-to-install.md#-dsupport_timer-off).
+Compile `libyt` with [`-DSUPPORT_TIMER=ON`](../how-to-install/details.md#-dsupport_timer-off).
 
 ## Chrome Tracing -- Visualizing the Profile
 1. Since each process dumps its profile `libytTimeProfile_MPI*.json` separately, we run the following to concatenate all of them:

doc/example.md

@@ -21,7 +21,7 @@ The example initializes `libyt`, loads data to `libyt` in every simulation time
 
 ### Using CMake
 
-1. Follow [Install](./how-to-install.md#install).
+1. Follow [Install](./how-to-install/how-to-install.md#libyt).
 2. Enter example folder. Assume we build the project under `libyt/build`:
    ```bash
    cd libyt                        # go to libyt project root folder
@@ -30,7 +30,7 @@ The example initializes `libyt`, loads data to `libyt` in every simulation time
 
 ### Using Make
 
-1. Follow [Install](./how-to-install.md#install).
+1. Follow [Install](./how-to-install/how-to-install.md#libyt).
 2. Go to `libyt/example/amr-example` folder.
    ```bash
    cd libyt/example/amr-example
@@ -134,6 +134,8 @@ The example initializes `libyt`, loads data to `libyt` in every simulation time
    After loading simulation data using libyt API, we call Python functions defined in `inline_script.py` using [`yt_run_Function`](./libyt-api/run-python-function.md#yt_run_function)/[`yt_run_FunctionArguments`](./libyt-api/run-python-function.md#yt_run_functionarguments) and use `yt` to analyze the data and plot figures. For how to use `yt` see:
      - [Using `yt`](./in-situ-python-analysis/using-yt.md)
 
+---
+
 ## What's Next
 
 ###### Change Python Script Name