Snowflake, python, scala Professional

JD for your reference : JD: Snowflake + Python /ETL : Grade G6 CTC :24 LPA MAx Strong ETL and DWH experience and knowledge Worked in any one of the following tool for end to end data process implementation Redshift Snowflake Databricks Ability to solve data engineering issues with SQL Open to learn additional tools and technologies for Dev ops and AI in data space. JD : Spark (Python, Scala) + SQL Grade G6 CTC : 24 LPA MAX Responsibilities : 5-8 years of development experience in object oriented applications with 2-4 years of experience in Scala. 1.In depth understanding/knowledge of Hadoop & spark Architecture and its components such as HDFS, Job Tracker, Task Tracker, executor cores and memory params. 2.Experience in Hadoop development, working experience on SPARK, SCALA is mandatory and Database exposure is must 3.Hands on experience in Spark and Spark Streaming creating RDDs, applying operations -Transformation and Actions. 4.Experience in code optimize to fine tune the applications. 5.Expertise in writing Hadoop/spark Jobs for analyzing data using Spark, Scala, Hive, Kafka and Python 6.Experienced with streaming work flow operations. 7.Experience with developing large-scale distributed applications and developing solutions to analyze large data sets efficiently. 8.Integration with Hadoop/HDFS, Real-Time Systems, Data Warehouses, and Analytics solutions. Experience in Data Warehousing and ETL processes. 9.Strong database, SQL, ETL and data analysis skills JD : Python Data Engineer (Numpy/Panda)+ SQL Garde G5 CTC : 18 LPA Strong knowledge on developing python applications processing large volume of data with overall 6 to 8 years of development experience. Ability to write complex SQL and profile data Deep knowledge on performance tuning SQL and python application which are dealing with large volume of data Exposure to Airflow, K8 and Performance tuning 3+ working experience on leading data framework like panda, pedantic Experience with working on data sets from snowflake Developer services and fine tunes application (python) deployed in containers. Expertise on building unit testing using pytest Prior experience in working with data project using data mapping, reference data etc Experience in working on Gxp compliance project or life science project is preferred JD: ADB + Datstage Grade G5 Max: 18 LPa Primary skil: Datastage +Databricks As a Data Engineer, you will be responsible for: Design & Hands-on engineering of Big Data and cloud environments that support our clients advanced analytics and data science initiatives. Provide support in defining the scope and sizing of work Working closely with various enterprise architects Information security teams, Data management team, to ensure the architected solution meets all the needs of a customer, from a functionality perspective and IT solution engineering perspective. Translate business requirements into technology solutions Mentor and guide Jr. team members to deliver the solutions on-time Technical Skills: Overall 6-9 years of experience of experience with Data Management, Big Data, Data Warehousing and Analytics. At least 1 years of experience in designing and migrating Oracle, DataStage based Enterprise Wherehouse to Azure using Databricks. 1-2 years of experience in designing and developing ETL solutions using Data Stage. Understanding and reverse engineering of DataStage parallel ,server and sequence ETL jobs to Databricks SparkSQL/PySpark/Spark Declarative pipelines. Analysis and converting custom stages, local containers, shared containers, routines and functions to Databricks SparkSQL/PySpark code. Analysis of the DataStage Configuration & Batch-job Shell scripts. Deep expertise in reverse engineering of Data Stage code and producing complexity reports. 3+ years of hands-on experience in creating an end-to-end lineage of all inbound and outbound file/data delivery including regulatory or operational reporting 3+ years of hands-on experience with ADF, Pyspark 1-2 years of experience with Tidal. One to two years of experience in Collibra DQ and DG is plus. Hands on working experience with scripting languages such as Python, Pyspark, Spark SQL or Shell Scripting, etc Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.