Metadata-Version: 2.4
Name: amd-dask-hip
Version: 1.0.0
Summary: Utilities for Dask and HIP interactions
Author: Advanced Micro Devices, Inc., NVIDIA Corporation
License: Apache-2.0 AND MIT
Project-URL: Homepage, https://github.com/ROCm-DS/dask-hip
Project-URL: Documentation, https://rocm.docs.amd.com/projects/dask-hip/en/latest/
Project-URL: Source, https://github.com/ROCm-DS/dask-hip
Classifier: Intended Audience :: Developers
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE.txt
Requires-Dist: dask==2025.9.1
Requires-Dist: distributed==2025.9.1
Requires-Dist: hip-python==7.2.*
Requires-Dist: hip-python-as-cuda==7.2.*
Requires-Dist: numba-hip==0.1.6
Requires-Dist: amdsmi==26.2.*
Requires-Dist: click>=8.1
Requires-Dist: numpy<3.0a0,>=1.23
Requires-Dist: pandas>=1.3
Requires-Dist: zict>=2.0.0
Requires-Dist: rich
Provides-Extra: docs
Requires-Dist: rocm-docs-core==1.27.0; extra == "docs"
Requires-Dist: myst-parser; extra == "docs"
Requires-Dist: sphinx-autoapi; extra == "docs"
Requires-Dist: sphinx-copybutton; extra == "docs"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Requires-Dist: pytest-rerunfailures!=16.0.0; extra == "test"
Requires-Dist: pytest-timeout; extra == "test"
Requires-Dist: psutil; extra == "test"
Requires-Dist: pyarrow; extra == "test"
Provides-Extra: test-ucxx
Requires-Dist: amd-hipucxx==0.1.*; extra == "test-ucxx"
Requires-Dist: amd-distributed-hipucxx==0.1.*; extra == "test-ucxx"
Provides-Extra: test-hipdf
Requires-Dist: amd-dask-hipdf==3.0.*; extra == "test-hipdf"
Dynamic: license-file

# dask-hip

dask-hip is an extension of [Dask.distributed](https://distributed.dask.org/)
that simplifies deploying Dask clusters on multi-GPU systems with AMD GPUs. It
is derived from [dask-cuda](https://github.com/rapidsai/dask-cuda) by NVIDIA
Corporation and is part of the [ROCm-DS](https://github.com/ROCm-DS) ecosystem.

Key capabilities include one-worker-per-GPU scheduling with automatic
`HIP_VISIBLE_DEVICES` management, CPU affinity, UCX-based high-performance
communication (ROCm-IPC, InfiniBand, TCP), GPU memory spilling, and hipMM (RMM)
pool integration.

For full documentation, see the
[dask-hip documentation](https://rocm.docs.amd.com/projects/dask-hip/en/latest/).

## Directory layout

```
dask-hip/
├── dask_cuda/            Python package (preserves upstream module name)
│   ├── cli.py              CLI entry point (dask cuda worker, dask cuda config)
│   ├── local_cuda_cluster.py   LocalCUDACluster
│   ├── cuda_worker.py      CUDAWorker
│   ├── initialize.py       Client-side UCX initialization
│   ├── explicit_comms/     Explicit communication API
│   ├── plugins.py          Worker plugins (CPU affinity, RMM, hipDF)
│   └── tests/              Test suite
├── pynvml2amdsmi/        pynvml compatibility shim for AMD SMI
├── examples/             Usage examples (UCX integration)
├── docs_amd/             Sphinx documentation source
├── scripts/              Utility scripts
├── pyproject.toml        Package metadata and dependencies
└── LICENSE / NOTICE.txt  License files
```

## Environment setup

Create and activate a Python environment before installing or building
dask-hip. Either Conda or a Python virtual environment can be used.

**Conda:**

```bash
conda create --name dask-hip python=3.12
conda activate dask-hip
```

**Python virtual environment:**

```bash
python3 -m venv dask-hip-env
source dask-hip-env/bin/activate
```

## Prerequisites

### AMD SMI

dask-hip requires the `amdsmi` Python package, which is distributed with ROCm.
Install it from the ROCm installation:

```bash
cd /opt/rocm/share/amd_smi
pip install .
```

### UCX (optional, for high-performance communication)

UCX-based communication via [hip-ucxx](https://github.com/ROCm-DS/hip-ucxx)
enables ROCm-IPC and InfiniBand transports. See the
[hip-ucxx build guide](https://rocm.docs.amd.com/projects/hip-ucxx/en/latest/install/build.html)
for UCX installation instructions.

## Installing

Install from AMD PyPI:

```bash
pip install amd-dask-hip --extra-index-url=https://pypi.amd.com/simple
```

For UCX communication support:

```bash
pip install amd-distributed-hipucxx --extra-index-url=https://pypi.amd.com/simple
```

For building from source, see the
[build guide](https://rocm.docs.amd.com/projects/dask-hip/en/latest/install/build.html).

## License

dask-hip is licensed under a combination of the Apache License 2.0 (for code
derived from NVIDIA dask-cuda) and the MIT License (for AMD additions). See
[LICENSE](LICENSE) and [NOTICE.txt](NOTICE.txt) for the full license texts.
