Dependency profiles¶

Requirements below come from pyproject.toml for version 0.2.0. They are allowed package ranges, not a runtime qualification matrix.

Python requirement: >=3.11,<4.0.

Core installation¶

python -m pip install datacoolie installs only the core requirements:

python-dateutil>=2.9,<3.0
tzdata>=2024.1; sys_platform == 'win32'

Choose an engine and the formats/transports you use. Extras may be combined:

python -m pip install "datacoolie[cli,polars-delta,source-api]"

For the matching-release wheel route, interpreter setup and first local run, follow Installation.

Optional profiles¶

Extra	Python requirements	Use / additional prerequisites
`spark`	`pyspark>=3.5`	Spark engine; provide a compatible Java runtime separately.
`polars`	`polars>=1.0`	Polars engine without lakehouse-format dependencies.
`polars-sql`	`polars>=1.0` `sqlglot>=30,<31`	Polars SQL with qualified relation rewriting.
`polars-hash`	`polars-hash>=0.6`	Optional Polars hashing dependency.
`spark-delta`	`pyspark>=3.5` `delta-spark>=3.0`	Spark engine and Delta format support; Java/JVM artifacts are separate.
`polars-delta`	`polars>=1.0` `deltalake>=0.15`	Polars engine and path-based Delta reads/writes.
`polars-iceberg`	`polars>=1.0` `pyiceberg>=0.12.0,<0.13.0`	Polars engine and catalog-backed Iceberg support; configure a catalog.
`aws`	`boto3>=1.43.2`	AWS platform SDK; configure credentials and permissions separately.
`fabric-external`	`azure-identity>=1.17,<2.0` `azure-storage-file-datalake>=12.14,<13.0` `azure-keyvault-secrets>=4.8,<5.0`	Fabric platform outside its native runtime using Azure SDK backends.
`databricks-external`	`databricks-sdk>=0.121,<0.122`	Databricks platform outside its native runtime using the SDK.
`source-api`	`httpx>=0.24`	REST API source transport; combine with an engine profile.
`source-excel-polars`	`polars>=1.0` `fastexcel>=0.9` `openpyxl>=3.1`	Excel data reads with Polars; separate from Excel metadata.
`source-db-polars`	`polars>=1.0` `connectorx>=0.4,<0.5`	Polars ConnectorX database reads; PostgreSQL uses this by default.
`source-db-native-polars`	`polars>=1.0` `pymysql>=1.1,<3.0` `pymssql>=2.3,<3.0`	Polars native MySQL/MSSQL password reads. Native PostgreSQL requires a separately installed psycopg2 driver.
`source-db-oracle-polars`	`polars>=1.0` `oracledb>=2.0,<5.0`	Polars native Oracle reads with oracledb.
`source-db-mssql-odbc-polars`	`polars>=1.0` `sqlalchemy>=2.0,<3.0` `pyodbc>=5.0,<6.0`	Polars MSSQL token/identity reads. Install system ODBC Driver 18 separately.
`metadata-yaml`	`pyyaml>=6.0,<7.0`	YAML file metadata parsing; JSON metadata needs no extra.
`metadata-excel`	`openpyxl>=3.1`	Excel metadata parsing; does not add Excel data-source support.
`metadata-db`	`sqlalchemy>=2.0,<3.0`	Database metadata provider via SQLAlchemy; install the selected DB driver separately.
`cli`	`pyyaml>=6.0,<7.0` `build>=1.2,<2.0` `jsonschema>=4.0,<5.0` `packaging>=24.0,<27.0` `typing-extensions>=4.4`	Project validation, inspection and build CLI; choose an engine separately to execute dataflows.
`all`	`pyspark>=3.5` `delta-spark>=3.0` `polars>=1.0` `sqlglot>=30,<31` `polars-hash>=0.6` `deltalake>=0.15` `pyiceberg>=0.12.0,<0.13.0` `boto3>=1.43.2` `databricks-sdk>=0.121,<0.122` `azure-identity>=1.17,<2.0` `azure-storage-file-datalake>=12.14,<13.0` `azure-keyvault-secrets>=4.8,<5.0` `httpx>=0.24` `fastexcel>=0.9` `openpyxl>=3.1` `connectorx>=0.4,<0.5` `pymysql>=1.1,<3.0` `pymssql>=2.3,<3.0` `oracledb>=2.0,<5.0` `sqlalchemy>=2.0,<3.0` `pyodbc>=5.0,<6.0` `pyyaml>=6.0,<7.0` `build>=1.2,<2.0` `jsonschema>=4.0,<5.0` `packaging>=24.0,<27.0` `typing-extensions>=4.4`	Union of declared Python profiles; system drivers, credentials and backend configuration are still required.

Select the transport¶

Source dependencies and metadata-provider dependencies serve different operations. For engine/database URL syntax and native, ConnectorX, JDBC or ODBC selection, see Database and SQL sources. A registry entry alone does not verify that its optional dependencies are installed.