Specialist - Data Engineering

Job Summary Seeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design and implement scalable big data solutions within the Hadoop ecosystem Job Description  Extensive hands-on experience with Apache Spark for largescale data processing and analytics Proficient in big data technologies within the Hadoop ecosystem Strong programming skills in Scala for data engineering tasks Deep understanding of distributed computing frameworks and performance optimization techniques Experience with batch and real-time data processing pipelines Knowledge of data ingestion transformation and storage best practices Familiarity with data quality assurance validation and troubleshooting in big data environments  Ability to work with complex data architectures and cloudbased big data platforms  Skilled in debugging and optimizing Spark jobs to ensure efficient resource utilization  Understanding of data security compliance and governance within big data solutions Roles and Responsibilities  Design develop and maintain scalable and robust data pipelines using Apache Spark Spark SQL Scala and related big data technologies  Collaborate with cross functional teams to gather business requirements and translate them into technical solutions  Optimize Spark applications for performance scalability and cost efficiency  Lead the implementation of data processing workflows ensuring data quality and consistency  Provide technical mentorship and guidance to junior team members and peers  Participate in architectural reviews and recommend best practices for big data solution design  Troubleshoot and resolve complex issues in data processing pipelines within defined SLAs  Stay updated with emerging big data technologies and incorporate relevant advancements into existing systems  Ensure adherence to organizational standards for data security compliance and governance

Mandatory Skills : Java Microservices, Java Spring Boot