Data Engineer

You will be responsible for the following tasks as a Mid-Level Data Engineer at SMC Global Securities: - Design, build, and maintain highly efficient and scalable real-time & batch data pipelines. - Implement and enforce data modeling standards. - Build data pipelines using a Medallion Architecture, progressing data through Bronze, Silver, and Gold. - Develop and optimize composable data architectures and data transformation processes. - Manage and maintain data storage solutions across our Data Lake (S3), Data Warehouse. - Write and optimize complex SQL queries and Python scripts. - Implement and orchestrate data workflows. Qualifications required for this role include: - Strong proficiency in SQL and Python. - Good understanding of data modeling concepts. - Strong understanding of Medallion and composable data architectures. - Solid understanding of data architecture concepts including Data Lake, Data Warehouse, and Data Lakehouse. - Hands-on experience with AWS cloud services, including but not limited to S3, Redshift, Athena, Glue, EMR, and Lambda. - Experience with open-source data tools like Airflow, DBT, and Airbyt. - Familiarity with Hive & Iceberg tables is essential. Additionally, it is good to have exposure to DBT (Data Build Tool) and experience with reporting tools, preferably Metabase. You will be responsible for the following tasks as a Mid-Level Data Engineer at SMC Global Securities: - Design, build, and maintain highly efficient and scalable real-time & batch data pipelines. - Implement and enforce data modeling standards. - Build data pipelines using a Medallion Architecture, progressing data through Bronze, Silver, and Gold. - Develop and optimize composable data architectures and data transformation processes. - Manage and maintain data storage solutions across our Data Lake (S3), Data Warehouse. - Write and optimize complex SQL queries and Python scripts. - Implement and orchestrate data workflows. Qualifications required for this role include: - Strong proficiency in SQL and Python. - Good understanding of data modeling concepts. - Strong understanding of Medallion and composable data architectures. - Solid understanding of data architecture concepts including Data Lake, Data Warehouse, and Data Lakehouse. - Hands-on experience with AWS cloud services, including but not limited to S3, Redshift, Athena, Glue, EMR, and Lambda. - Experience with open-source data tools like Airflow, DBT, and Airbyt. - Familiarity with Hive & Iceberg tables is essential. Additionally, it is good to have exposure to DBT (Data Build Tool) and experience with reporting tools, preferably Metabase.