opendalfs

opendalfs makes storage services supported by Apache OpenDAL available through the fsspec API. You can use the same filesystem interface with pandas, Dask, Xarray, Zarr, PyArrow, and other libraries that accept fsspec URLs or filesystem objects.

Install and try it

Install opendalfs from PyPI:

pip install opendalfs

Then create an in-memory filesystem and use the familiar fsspec interface. This example does not need credentials or leave files on disk:

from opendalfs import OpendalFileSystem

fs = OpendalFileSystem("memory")
fs.pipe_file("hello.txt", b"hello from opendalfs\n")

assert fs.cat_file("hello.txt") == b"hello from opendalfs\n"

Continue with the Quickstart to use file-like objects and fsspec URLs.

Get started

Follow the first user guide from installation through fsspec URL access.

Quickstart

Connect storage

Choose an OpenDAL service and pass its configuration through fsspec.

Connecting to storage

Use an integration

Find examples backed by the repository’s integration tests.

Integrations

Read the API reference

Inspect the public classes and registration helpers provided by opendalfs.

API reference

Where opendalfs fits

OpenDAL connects to storage services. fsspec defines the Python filesystem interface used by the wider data ecosystem. opendalfs adapts the OpenDAL Python binding to that interface.

Configuration travels down through the stack, while file operations travel up:

pandas, Dask, Xarray, Zarr, PyArrow, ...
                    |
               fsspec API
                    |
                opendalfs
                    |
         OpenDAL Python binding
                    |
          S3, GCS, Azure, ...

OpenDAL remains the source of truth for service configuration. This site documents the fsspec adapter, its URL rules, and the integrations tested in this repository.