Job Type: Full Time
Job Title: Big Data Engineer – AWS | Scala | Python | CI/CD | Microservices
Location: Chennai, Bengaluru, Hyderabad, Pune, Mumbai, Noida, NCR
Experience: 4–8 years
Job Type: Full-Time
Department: Data Engineering / Technology
Job Summary:
We are looking for a Big Data Engineer with strong experience in building robust, scalable data pipelines and platforms using AWS, Scala, Python, and microservices architecture. The ideal candidate will be responsible for end-to-end data engineering processes including ingestion, transformation, and delivery, while ensuring reliability and performance through automated CI/CD pipelines.
Key Responsibilities:
- Design and implement large-scale data processing systems using Big Data technologies (Spark, Hadoop, Kafka).
- Develop scalable data pipelines in Scala and Python for batch and real-time processing.
- Deploy and maintain data engineering solutions on AWS (EMR, S3, Glue, Lambda, Redshift, etc.).
- Build and expose data services through microservices and APIs for downstream applications.
- Automate testing and deployment processes using CI/CD tools like Jenkins, GitHub Actions, CodePipeline, etc.
- Collaborate with Data Scientists, Analysts, and Software Engineers to ensure seamless data flow and model integration.
- Monitor, troubleshoot, and optimize data infrastructure for performance and cost.
- Ensure data governance, quality, and compliance throughout the data lifecycle.
Required Skills & Qualifications:
- Bachelor’s or Master’s in Computer Science, Engineering, or a related field.
- 4+ years of experience as a Big Data Engineer or Data Engineer.
- Proficiency in Scala and Python for data processing and scripting.
- Hands-on experience with Apache Spark, Kafka, and Hadoop ecosystems.
- Solid experience working with AWS data services such as EMR, S3, Glue, Redshift, Athena.
- Experience developing RESTful APIs and working with microservices architectures.
- Proficiency in CI/CD pipeline implementation and automation (Jenkins, GitHub Actions, CodeBuild, etc.).
- Strong understanding of distributed computing, performance optimization, and data modeling.
- Familiarity with version control systems like Git and infrastructure-as-code tools like Terraform or CloudFormation.
Preferred Qualifications:
- Experience with Kubernetes and container orchestration on cloud (EKS, ECS).
- Exposure to real-time data streaming solutions.
- Knowledge of data security and compliance in cloud environments.
- Certifications such as AWS Certified Big Data – Specialty, Data Analytics, or Developer Associate.
What We Offer:
- Opportunity to work on high-impact, enterprise-grade big data projects.
- Flexible work environment and modern tech stack.
- A collaborative team with continuous learning culture.
- Competitive salary and performance-based incentives.