Databricks Engineer Resume Example

Senior Databricks Engineer · QuantEdge Analytics

Chicago, IL · 2023 - Present

• Architected and implemented a real-time risk analytics platform on Databricks, processing over 100TB of streaming data daily with a 99.9% uptime and reducing latency by 40% for critical financial models.
• Led the migration of on-premise data warehouses to Delta Lake on Azure Databricks, achieving a 25% reduction in compute costs and improving data query performance by an average of 50%.
• Developed and enforced data governance policies using Unity Catalog, ensuring 100% compliance with financial regulations (e.g., SOX, GDPR) across 50+ critical data assets.
• Mentored a team of 4 junior data engineers, fostering best practices in Spark optimization, CI/CD pipelines (Azure DevOps), and Databricks workspace management, enhancing team productivity by 15%.

Databricks Engineer · Liberty Financial Group

Chicago, IL · 2020 - 2022

• Designed and implemented ETL pipelines using PySpark and Databricks notebooks to ingest, transform, and load data from diverse sources into a centralized data lake, improving data availability for reporting by 30%.
• Optimized existing Spark jobs by refactoring code and tuning configurations, reducing daily processing time by an average of 20 hours and saving $5,000 monthly in cloud infrastructure costs.
• Collaborated with data scientists to deploy machine learning models (MLflow) for credit risk assessment, resulting in a 10% increase in model accuracy and a 5% decrease in default rates.
• Developed robust data quality checks and monitoring frameworks within Databricks, reducing data discrepancies by 90% and improving overall data reliability.

Master of Science in Computer Science · University of Illinois Urbana-Champaign

Urbana, IL · 2018 - 2019

• Graduated with High Honors, GPA: 3.9/4.0.
• Relevant Coursework: Advanced Distributed Systems, Machine Learning, Data Mining, Cloud Computing Architectures, Big Data Analytics.
• Master's Thesis: "Optimizing Low-Latency Data Streaming Architectures on Apache Spark" focusing on financial transaction processing.

Databricks Platform Engineer · Equifax

Atlanta, GA · 2024 - Present

• Architected and deployed a multi-tenant Databricks Lakehouse platform on AWS, facilitating secure data access for over 50 data science teams and reducing data provisioning time by 40%.
• Developed and optimized Spark-based ETL pipelines using Delta Lake, processing over 50TB of raw data daily and improving data freshness by 35% for critical business reports.
• Implemented CI/CD pipelines for Databricks notebooks and jobs using Azure DevOps, reducing deployment cycles from days to hours and increasing team productivity by 25%.
• Led the migration of legacy data warehouses to Databricks, resulting in a 20% reduction in operational costs and a 50% improvement in query performance for analytical workloads.

Data Engineer · Expedia Group

Austin, TX · 2022 - 2024

• Designed and implemented scalable data ingestion pipelines using Apache Kafka and Spark Streaming, processing over 10 billion events per day with an average latency of under 5 seconds.
• Developed and maintained data warehouses in AWS Redshift and Snowflake, optimizing query performance by 25% through schema redesign and indexing strategies for critical business intelligence dashboards.
• Collaborated with data scientists to deploy machine learning models into production environments, improving model inference time by 15% and supporting real-time recommendations for over 100 million users.

Embedded Systems Engineer · Lockheed Martin

Marietta, GA · 2020 - 2021

• Developed and tested firmware for mission-critical aerospace systems, reducing software defects by 18% through rigorous unit testing and integration strategies.
• Optimized real-time operating system performance, achieving a 15% reduction in latency for sensor data processing and improving system responsiveness.
• Contributed to the design and implementation of communication protocols for distributed embedded systems, ensuring reliable data exchange across heterogeneous hardware platforms.

Bachelor of Science in Electrical Engineering · Georgia Institute of Technology

Atlanta, GA · 2016 - 2020

• Graduated with High Honors, GPA: 3.8/4.0
• Relevant Coursework: Data Structures & Algorithms, Digital Signal Processing, Machine Learning Fundamentals, Embedded Systems Design, Computer Architecture.
• Senior Capstone Project: Developed an energy-efficient sensor network for environmental monitoring, achieving 30% power consumption reduction compared to baseline designs.