Big Data Developer

Big Data Developer Experience 48 Years Employment Type Full-Time Job Summary We are seeking a highly skilled Big Data Developer to design, develop, and maintain large-scale data processing systems and data pipelines. The ideal candidate will have experience working with distributed computing frameworks, cloud platforms, and modern big data technologies to support data analytics, reporting, and machine learning initiatives. The candidate will collaborate with data engineers, data scientists, business analysts, and technology teams to build scalable, reliable, and high-performance data solutions. Key Responsibilities Design, develop, and maintain scalable big data solutions and data pipelines. Build and optimize ETL/ELT processes for processing structured and unstructured data. Develop data ingestion frameworks from multiple data sources including databases, APIs, streaming systems, and file-based systems. Work with distributed data processing technologies such as Spark, Hadoop, and Kafka. Develop batch and real-time data processing applications. Ensure data quality, integrity, security, and governance across data platforms. Monitor and troubleshoot production data workflows and resolve performance issues. Collaborate with Data Architects and Business Stakeholders to understand data requirements. Implement best practices for data modeling, code quality, testing, and deployment. Participate in code reviews and contribute to continuous improvement initiatives. Support cloud-based data solutions and migration projects. Required Technical Skills Big Data Technologies Apache Spark (PySpark/Scala Spark) Hadoop Ecosystem (HDFS, Hive, Yarn) Apache Kafka Apache Airflow Databricks Programming Languages Python Scala Java SQL Database Technologies SQL Server Oracle PostgreSQL MySQL NoSQL Databases (MongoDB, Cassandra, HBase) Cloud Platforms Microsoft Azure (ADLS, Azure Data Factory, Synapse Analytics) AWS (S3, EMR, Glue, Redshift) Google Cloud Platform (BigQuery, DataProc) – Preferred DevOps & Version Control Git Azure DevOps Jenkins CI/CD Pipelines Required Qualifications Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. 3+ years of experience in Big Data development and data engineering projects. Strong experience with Spark, Hadoop, Kafka, and distributed computing environments. Proficiency in Python, Scala, or Java. Strong SQL and data modeling skills. Experience working with cloud-based data platforms (Azure/AWS/GCP). Understanding of data warehousing concepts and ETL design patterns. Experience in Agile/Scrum development methodologies.