Using SLURM to Start a Wepy Simulation
======================================

To start a Wepy simulation using SLURM, you can use the following SLURM
script template. Customize the script according to your specific
resource requirements and environment setup.

Example SLURM Script
--------------------

.. code:: bash

   #!/bin/bash --login
   ########## SBATCH Lines for Resource Request ##########

   #SBATCH --time=TIME_LIMIT            # limit of wall clock time - how long the job will run (e.g., 168:00:00 for 168 hours)
   #SBATCH -A YOUR_ALLOCATION_ACCOUNT   # replace with your allocation account
   #SBATCH --nodes=NUMBER_OF_NODES      # the number of nodes requested (e.g., 1)
   #SBATCH --ntasks=NUMBER_OF_TASKS     # the number of tasks to run (e.g., 1)
   #SBATCH --cpus-per-task=CPUS_PER_TASK # the number of CPUs (or cores) per task (e.g., 8)
   #SBATCH --gpus-per-task=GPUS_PER_TASK # request GPUs per task (e.g., v100:4)
   #SBATCH --mem=MEMORY_PER_NODE        # total memory per node (e.g., 64G)
   #SBATCH --job-name=JOB_NAME          # give your job a name for easier identification (e.g., wepy_run)
   #SBATCH --output=%x.out              # standard output file (e.g., wepy_run.out)
   #SBATCH --error=%x.err               # standard error file (e.g., wepy_run.err)

   ########## Command Lines to Run ##########

   # Load necessary modules (customize as needed)
   module load Conda/3                  # load the Conda module (modify if needed)

   # Initialize Conda (replace with the path to your Conda installation)
   eval "$(/path/to/conda/bin/conda shell.bash hook)"  # e.g., /mnt/home/username/anaconda3/bin/conda

   # Activate your Conda environment (replace with your environment name)
   conda activate your_environment_name # e.g., wepy_env

   # Set your home directory for the project (replace with your home directory)
   HOME_DIR="/path/to/your/home/directory"  # e.g., /mnt/home/username/project_dir
   JOBNAME=${SLURM_JOB_ID}

   # Set your log directory (replace with your log directory)
   LOG_DIR="$HOME_DIR/logs"  # e.g., $HOME_DIR/logs

   # Log the beginning of the run
   echo 'Beginning of the run' 1>> "$LOG_DIR/$JOBNAME.log" 2>> "$LOG_DIR/$JOBNAME.log"

   # Load CUDA module if necessary (customize as needed)
   module load centos7/lib/cuda/12  # modify CUDA version if needed

   # Change to the home directory
   cd "$HOME_DIR"

   # Log the SLURM_JOB_ID number
   echo "SLURM_JOB_ID: $SLURM_JOB_ID" 1>> "$LOG_DIR/$JOBNAME.log" 2>> "$LOG_DIR/$JOBNAME.log"

   # Running the Wepy simulation script (replace with your script name)
   echo "Running Wepy simulation script" 1>> "$LOG_DIR/$JOBNAME.log" 2>> "$LOG_DIR/$JOBNAME.log"
   python wepy_run.py  # replace with your Wepy simulation script name

Instructions for Customization
------------------------------

#. **Resource Requests:**

   -  Replace TIME\ :sub:`LIMIT` with the desired wall clock time limit
      (e.g., ``168:00:00`` for 168 hours).
   -  Replace YOUR\ :sub:`ALLOCATIONACCOUNT` with your specific
      allocation account.
   -  Replace NUMBER\ :sub:`OFNODES` with the number of nodes you need
      (e.g., ``1``).
   -  Replace NUMBER\ :sub:`OFTASKS` with the number of tasks to run
      (e.g., ``1``).
   -  Replace CPUS\ :sub:`PERTASK` with the number of CPUs per task
      (e.g., ``8``).
   -  Replace GPUS\ :sub:`PERTASK` with the type and number of GPUs per
      task (e.g., ``v100:4``).
   -  Replace MEMORY\ :sub:`PERNODE` with the total memory per node
      (e.g., ``64G``).
   -  Replace JOB\ :sub:`NAME` with a name for your job (e.g.,
      ``wepy_run``).

#. **Conda Setup:**

   -  Replace ``/path/to/conda/bin/conda`` with the actual path to your
      Conda installation.
   -  Replace ``your_environment_name`` with the name of your Conda
      environment.

#. **Home and Log Directory:**

   -  Set HOME\ :sub:`DIR` to the directory where your project files are
      located.
   -  Ensure LOG\ :sub:`DIR` points to where you want the log files to
      be saved.

#. **CUDA Module:**

   -  Adjust the ``module load centos7/lib/cuda/12`` line depending on
      your cuda path.

#. **Wepy Simulation Script:**

   -  Replace ``wepy_run.py`` with the name of your Wepy simulation
      script.

This template provides flexibility for users to customize the SLURM
script according to their specific needs while maintaining a general
structure for running a Wepy simulation.