AWS S3, Redshift for data storage
Hive for metadata store
DBT, Spark (Python, SQL) for ETL development
Airflow for orchestration
Kafka for data streaming
AWS S3, Redshift for data storage
Hive for metadata store
DBT, Spark (Python, SQL) for ETL development
Airflow for orchestration
Kafka for data streaming