Skip to main content

Dagster & AWS Athena

This integration allows you to connect to AWS Athena, a serverless interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Using this integration, you can issue queries to Athena, fetch results, and handle query execution states within your Dagster pipelines.

Installation​

pip install dagster-aws

Examples​

from dagster_aws.athena import AthenaClientResource

import dagster as dg


@dg.asset
def example_athena_asset(athena: AthenaClientResource):
    return athena.get_client().execute_query("SELECT 1", fetch_results=True)


defs = dg.Definitions(
    assets=[example_athena_asset], resources={"athena": AthenaClientResource()}
)

About AWS Athena​

AWS Athena is a serverless, interactive query service that allows you to analyze data directly in Amazon S3 using standard SQL. Athena is easy to use; point to your data in Amazon S3, define the schema, and start querying using standard SQL. Most results are delivered within seconds. With Athena, there are no infrastructure setups, and you pay only for the queries you run. It scales automatically—executing queries in parallel—so results are fast, even with large datasets and complex queries.