Source description
About the role
Job Title: Junior Data Engineer Location:Mumbai Job Type: Full-Time Key Responsibilities: - Develop, maintain, and optimize data pipelines to ensure the smooth extraction, transformation, and loading (ETL) of data. - Build and maintain web crawlers and scraping solutions to collect structured and unstructured data from various online sources. - Collaborate with the Data Science and Analytics teams to ensure that the data collected meets the required standards for analysis. - Work with relational and non-relational databases to store and retrieve large datasets. - Implement and monitor data quality processes, ensuring data integrity and consistency across the pipeline. - Assist in performance tuning and troubleshooting of ETL processes and scripts. - Write clean, efficient, and scalable Python code for data processing tasks. - Continuously stay updated with the latest data engineering and web scraping tools, techniques, and technologies. Required Skills and Qualifications: - 1+ year of experience in data engineering, preferably working with Python and web scraping tools. - Proficiency in Python: - Experience with web scraping/crawling techniques, handling CAPTCHAs, and avoiding IP blocks. - Basic understanding of SQL and experience with relational databases like MySQL, PostgreSQL, or similar. - Familiarity with non-relational databases like MongoDB or Cassandra is a plus. - Experience working with cloud platforms (AWS, GCP, Azure) is desirable but not mandatory. - Knowledge of version control systems (e.g., Git) and agile development methodologies. - Strong problem-solving skills and attention to detail. - Ability to work both independently and collaboratively in a team environment. Preferred Qualifications: - Hands-on experience with big data processing tools such as Apache Spark or Hadoop. - Bachelors Degree. Compensation: 400,000.00 - 600,000.00 per year Perks: - Leave encashment - Paid sick time - Provident Fund - Work from home Schedule: - Day shift Work Lo .