CANDY

A library and benchmark suite for Approximate Nearest Neighbor Search (ANNS). This project is compatible with LibTorch.

Table of Contents

• Quick Start Guide
  • Docker Support
  • Build Without Docker
    • Build with CUDA Support
    • Build without CUDA (CPU-Only Version)
  • Installing PyCANDY
  • CLion Configuration
• Evaluation Scripts
• Additional Information

---

Quick Start Guide

Docker Support

We provide Docker support to simplify the setup process.

1. Navigate to the ./docker directory:

   cd ./docker

2. Build and start the Docker container:

   ./start.sh

   This script will build the Docker container and start it.

3. Inside the Docker container, run the build script to install dependencies and build the project:

• With CUDA support:

  ./buildWithCuda.sh

• Without CUDA (CPU-only version):

  ./buildCPUOnly.sh

Build Without Docker

If you prefer to build without Docker, follow these steps.

Build with CUDA Support

To build CANDY and PyCANDY with CUDA support:

./buildWithCuda.sh

Build without CUDA (CPU-Only Version)

For a CPU-only version:

./buildCPUOnly.sh

These scripts will install dependencies and build the project.

Installing PyCANDY

After building, you can install PyCANDY to your default Python environment:

python3 setup.py install --user

CLion Configuration

When developing in CLion, you must manually configure:

1. CMake Prefix Path:

• Run the following command in your terminal to get the CMake prefix path:

  python3 -c 'import torch; print(torch.utils.cmake_prefix_path)'

• Copy the output path and set it in CLion's CMake settings as:

  -DCMAKE_PREFIX_PATH=<output_path>
  

2. Environment Variable CUDACXX:

• Manually set the environment variable CUDACXX to:

  /usr/local/cuda/bin/nvcc
  

Evaluation Scripts

Evaluation scripts are located under benchmark/scripts.

To run an evaluation (e.g., scanning the number of elements in matrix A's row):

cd build/benchmark/scripts/scanARow
sudo ls  # Required for perf events
python3 drawTogether.py
cd ../figures

Figures will be generated in the figures directory.

---

Additional Information

Click to Expand

Table of Contents

• Extra CMake Options
• Manual Build Instructions
  • Requirements
  • Build Steps
  • CLion Build Tips
• CUDA Installation (Optional)
  • Install CUDA (if using CUDA-based Torch)
  • CUDA on Jetson Devices
• Torch Installation
  • Install Python and Pip
  • Install PyTorch
• PAPI Support (Optional)
  • Build PAPI
  • Verify PAPI Installation
  • Enable PAPI in CANDY
• Distributed CANDY with Ray (Optional)
  • Build with Ray Support
  • Running with Ray
  • Ray Dashboard (Optional)
• Local Documentation Generation (Optional)
  • Install Required Packages
  • Generate Documentation
    • Accessing Documentation
• Known Issues

---

Extra CMake Options

You can set additional CMake options using cmake -D<option>=ON/OFF:

• ENABLE_PAPI (OFF by default)
  • Enables PAPI-based performance tools.
  • Setup:
    • Navigate to the thirdparty directory.
    • Run installPAPI.sh to enable PAPI support.
    • Alternatively, set REBUILD_PAPI to ON.
• ENABLE_HDF5 (OFF by default)
  • Enables loading data from HDF5 files.
  • The HDF5 source code is included; no extra dependency is required.
• ENABLE_PYBIND (OFF by default)
  • Enables building Python bindings (PyCANDY).
  • Ensure the pybind11 source code in the thirdparty folder is complete.

Manual Build Instructions

Requirements

• Compiler: G++11 or newer.
  • The default gcc/g++ version on Ubuntu 22.04 (Jammy) is sufficient.
• BLAS and LAPACK:

  sudo apt install liblapack-dev libblas-dev

• Graphviz (Optional):

  sudo apt-get install graphviz
  pip install torchviz

Build Steps

1. Set the CUDA Compiler Path (if using CUDA):

   export CUDACXX=/usr/local/cuda/bin/nvcc

2. Create Build Directory:

   mkdir build && cd build

3. Configure CMake:

   cmake -DCMAKE_PREFIX_PATH=`python3 -c 'import torch; print(torch.utils.cmake_prefix_path)'` ..

4. Build the Project:

   make

For Debug Build:

cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=`python3 -c 'import torch; print(torch.utils.cmake_prefix_path)'` ..
make

CLion Build Tips

• Manually retrieve the CMake prefix path:

  python3 -c 'import torch; print(torch.utils.cmake_prefix_path)'

• Set the -DCMAKE_PREFIX_PATH in CLion's CMake settings.

• Set the environment variable CUDACXX to /usr/local/cuda/bin/nvcc in CLion.

CUDA Installation (Optional)

Install CUDA (if using CUDA-based Torch)

Refer to the NVIDIA CUDA Installation Guide for more details.

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt-get update
sudo apt-get install cuda
sudo apt-get install nvidia-gds
sudo apt-get install libcudnn8 libcudnn8-dev libcublas-11-7

Note: Ensure CUDA is installed before installing CUDA-based Torch. Reboot your system after installation.

CUDA on Jetson Devices

• No need to install CUDA if using a pre-built JetPack on Jetson.

• Ensure libcudnn8 and libcublas are installed:

  sudo apt-get install libcudnn8 libcudnn8-dev libcublas-*

Torch Installation

Refer to the PyTorch Get Started Guide for more details.

Install Python and Pip

sudo apt-get install python3 python3-pip

Install PyTorch

• With CUDA:

  pip3 install torch==1.13.0 torchvision torchaudio

• Without CUDA:

  pip3 install --ignore-installed torch==1.13.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

Note: Conflict between torch1.13.0+cpu and torchaudio+cpu may occur with Python versions > 3.10.

PAPI Support (Optional)

PAPI provides a consistent interface for collecting performance counter information.

Build PAPI

• Navigate to the thirdparty directory.
• Run installPAPI.sh.
• PAPI will be compiled and installed in thirdparty/papi_build.

Verify PAPI Installation

• Navigate to thirdparty/papi_build/bin.
• Run sudo ./papi_avail to check available events.
• Run ./papi_native_avail to view native events.

Enable PAPI in CANDY

• Set -DENABLE_PAPI=ON when configuring CMake.

• Add the following to your top-level config file:

  usePAPI,1,U64
  perfUseExternalList,1,U64
  

• To specify custom event lists, set:

  perfListSrc,<path_to_your_list>,String
  

• Edit perfLists/perfList.csv in your build directory to include desired events.

Distributed CANDY with Ray (Optional)

Build with Ray Support

1. Install Ray:

   pip install ray==2.8.1 ray-cpp==2.8.1

2. Get Ray Library Path:

   ray cpp --show-library-path

3. Set RAYPATH Environment Variable:

   export RAYPATH=<ray_library_path>

4. Configure CMake:

   cmake -DENABLE_RAY=ON ..

Running with Ray

• Start the Head Node:

  ray start --head

• Start Worker Nodes:

  ray start --address <head_node_ip>:6379 --node-ip-address <worker_node_ip>

• Run the Program:

  export RAY_ADDRESS=<head_node_ip>:6379
  ./<your_program_with_ray_support>

Notes:

• Ensure the file paths and dependencies are identical across all nodes.
• For different architectures, recompile the source code on each node.
• torch::Tensor may not be serializable; consider using std::vector<float> instead.

Ray Dashboard (Optional)

Refer to the Ray Observability Guide to set up a dashboard.

Local Documentation Generation (Optional)

Install Required Packages

sudo apt-get install doxygen graphviz
sudo apt-get install texlive-latex-base texlive-fonts-recommended texlive-fonts-extra texlive-latex-extra

Generate Documentation

./genDoc.SH

Accessing Documentation

• HTML Pages: Located in doc/html/index.html.
• PDF Manual: Found at refman.pdf in the root directory.

Known Issues

• Conflicts may occur with certain versions of PyTorch and Python.