Data Engineer – AWS Kinesis & Databricks - AU Citizen
Accenture — Canberra Brindabella Business Park
Posted 2026-10-19
We’re looking for a hands-on Data Engineer with strong experience building reliable, real-time data pipelines using Amazon Kinesis and Databricks . You’ll work with modern AWS data technologies to develop and optimise streaming pipelines, moving high-volume data from Kinesis into the Databricks Lakehouse using Spark Structured Streaming and Delta tables. What you’ll do : Build and support real-time data pipelines using Amazon Kinesis Data Streams and Kinesis Data Firehose . Develop streaming solutions in Databricks using Spark Structured Streaming . Ingest and manage streaming data within Delta tables . Optimise pipelines for performance, latency, reliability and cost. Build monitoring and alerting using AWS CloudWatch . Work with batch data engineers to ensure streaming and batch pipelines integrate effectively. Develop clean, testable Python and SQL code. Troubleshoot pipeline and data-processing issues and contribute to continuous improvement. What you’ll bring: Hands-on production experience with Amazon Kinesis Data Streams and/or Kinesis Data Firehose . Kinesis experience is essential. Strong Python and SQL skills. Experience with Apache Spark Structured Streaming . Hands-on e