.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
.. NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE
OVERWRITTEN WHEN PREPARING PACKAGES.
.. IF YOU WANT TO MODIFY TEMPLATE FOR THIS FILE, YOU SHOULD MODIFY THE TEMPLATE
PROVIDER_README_TEMPLATE.rst.jinja2
IN the dev/breeze/src/airflow_breeze/templates
DIRECTORY
Package apache-airflow-providers-apache-spark
Release: 4.11.3
Apache Spark <https://spark.apache.org/>
__
Provider package
This is a provider package for apache.spark
provider. All classes for this provider package
are in airflow.providers.apache.spark
python package.
You can find package information and changelog for the provider
in the documentation <https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/4.11.3/>
_.
Installation
You can install this package on top of an existing Airflow 2 installation (see Requirements
below
for the minimum Airflow version supported) via
pip install apache-airflow-providers-apache-spark
The package supports the following python versions: 3.9,3.10,3.11,3.12
Requirements
================== ==================
PIP package Version required
================== ==================
apache-airflow
>=2.8.0
pyspark
>=3.1.3
grpcio-status
>=1.59.0
================== ==================
Cross provider package dependencies
Those are dependencies that might be needed in order to use all the features of the package.
You need to install the specified provider packages in order to use them.
You can install such cross-provider dependencies when installing from PyPI. For example:
.. code-block:: bash
pip install apache-airflow-providers-apache-spark[cncf.kubernetes]
====================================================================================================================== ===================
Dependent package Extra
====================================================================================================================== ===================
apache-airflow-providers-cncf-kubernetes <https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes>
_ cncf.kubernetes
apache-airflow-providers-common-compat <https://airflow.apache.org/docs/apache-airflow-providers-common-compat>
_ common.compat
====================================================================================================================== ===================
The changelog for the provider package can be found in the
changelog <https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/4.11.3/changelog.html>
_.