Spark Developer Resume Example

Lead Spark Engineer · Apex Financial Solutions

Austin, TX · 2021 - Present

• Designed and implemented high-frequency Spark Streaming applications to process over 1TB of real-time transactional data daily, reducing analytics latency from hours to minutes.
• Optimized complex ETL pipelines for financial reporting, cutting processing times by 35% and improving data availability for critical business intelligence dashboards.
• Developed robust data quality frameworks using Spark and Scala, ensuring 99.99% accuracy for regulatory compliance reports and minimizing data discrepancies.
• Led a team in migrating legacy data warehouse processes to a cloud-native Spark architecture on AWS, resulting in a 20% reduction in operational costs.

Spark Data Engineer · Innovate Data Corp.

Austin, TX · 2020 - 2021

• Built scalable data ingestion pipelines using Apache Spark and Kafka, processing over 500 million events per day from diverse client data sources.
• Improved the efficiency of batch processing jobs by 2x through performance tuning of Spark configurations and data partitioning strategies.
• Collaborated with data scientists to deploy machine learning models on Spark clusters, facilitating faster model training and inference for customer segmentation.
• Automated data validation and monitoring scripts for critical data flows, reducing manual intervention by 60% and enhancing data reliability for client projects.

Master of Science in Computer Science · University of Texas at Austin

Austin, TX · 2018 - 2020

• Specialization in Data Science and Distributed Systems
• Relevant Coursework: Advanced Algorithms, Distributed Databases, Machine Learning
• GPA: 3.9/4.0

Bachelor of Science in Computer Engineering · University of Houston

Houston, TX · 2014 - 2018

• Graduated Magna Cum Laude
• Relevant Coursework: Data Structures, Operating Systems, Computer Networks
• Capstone Project: Real-time traffic analysis using stream processing

Spark Developer · Nordstrom Technology

Seattle, WA · 2022 - Present

• Designed and implemented high-performance Apache Spark ETL pipelines using PySpark, processing over 10TB of daily transactional data, reducing processing time by 30%.
• Developed and maintained data quality checks and monitoring tools, improving data accuracy for critical business reports by 25%.
• Optimized Spark applications on AWS EMR by tuning configurations and refactoring code, leading to a 20% cost reduction in cloud compute resources.
• Collaborated with data scientists to deploy machine learning models on Spark, enabling real-time personalization features for over 10 million customers.
• Mentored junior developers in best practices for Spark development and distributed computing.

Data Engineer · Expedia Group

Bellevue, WA · 2020 - 2022

• Migrated legacy Hadoop MapReduce jobs to Apache Spark, achieving a 50% reduction in execution time for key data aggregation processes.
• Built and maintained scalable data lakes on AWS S3 using Spark and Glue, integrating data from 15+ disparate sources for analytical consumption.
• Developed custom Spark UDFs to handle complex data transformations, increasing data preparation efficiency by 35% for the analytics team.
• Implemented robust error handling and logging frameworks for data pipelines, improving system reliability and reducing data reprocessing efforts by 40%.
• Participated in on-call rotation, resolving critical data pipeline incidents within strict SLAs and minimizing impact on business operations.

Data Analyst · Tableau Software

Seattle, WA · 2018 - 2020

• Analyzed large datasets to identify key trends and insights, contributing to a 15% increase in customer engagement for a flagship product.
• Developed and automated weekly performance dashboards using SQL and Tableau, reducing manual reporting efforts by 10 hours per week.
• Collaborated with product managers to define KPIs and tracked product usage, informing strategic decisions that led to a 5% improvement in user retention.
• Performed A/B testing analysis for new feature rollouts, providing data-driven recommendations that optimized user experience.
• Extracted and transformed data from various databases (SQL Server, PostgreSQL) to support ad-hoc reporting and analytical requests.

Master of Science in Computer Science · University of Washington

Seattle, WA · 2016 - 2018

• Specialized in Distributed Systems and Big Data Analytics.
• Completed thesis on 'Optimizing Large-Scale Graph Processing with Apache Spark'.
• GPA: 3.8/4.0

Bachelor of Science in Data Science · University of California, Berkeley

Berkeley, CA · 2012 - 2016

• Graduated Summa Cum Laude with a focus on statistical modeling and machine learning.
• Recipient of the Dean's Award for Academic Excellence, 2015 and 2016.
• GPA: 3.9/4.0