Big Data Engineer

Big Data Engineer

Position : Big Data Engineer

Job Duties :

Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.Strong analytic skills related to working with unstructured datasets.Build processes supporting data transformation, data structures, metadata, dependency and workload management. A successful history of manipulating, processing and extracting value from large disconnected datasets. Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores. Strong project management and organizational skills. Experience supporting and working with cross-functional teams in a dynamic environment. Implement the use case with Bigdata open source tools and technologies like SQOOP, HADOOP Ecosystems, HIVE, Unix Scripting, SPARK, SCALA, PYTHON, PYSPARK and SCALA to achieve the requirements. Ability to utilize the bigdata tools and techniques to integrate traditional databases like Oracle, SQL Server and Mysql into Hadoop Datalake. Create the Hive databases and store the processed data into Hive tables and then perform the aggregate operations to generate the new datasets and analyze the data. Develop an Enterprise Data Lake Application using Bigdata tools like Hive, Spark Streaming, Kafka, Sqoop, Oozie, Spark, Shell Scripting, Python, SQL, HDFS, Streamsets and Tidal.Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. Provide automation solutions for the jobs which are run manually to generate frequent reports which are utilized by the business team for data analysis.

Work Locations :

Various unanticipated work locations throughout the United States; relocation may be required. Must be willing to relocate.

Minimum Qualifications Education

• Bachelor – Computer Science

• Experience : None

• Job Order No : 9534132

• This position is eligible for a referral bonus through our employee referral program

• Position : Big Data Engineer

• Job Duties :

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

• Strong analytic skills related to working with unstructured datasets.

Build processes supporting data transformation, data structures, metadata, dependency and workload management. A successful history of manipulating, processing and extracting value from large disconnected datasets. Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores. Strong project management and organizational skills. Experience supporting and working with cross-functional teams in a dynamic environment. Implement the use case with Bigdata open source tools and technologies like SQOOP, HADOOP Ecosystems, HIVE, Unix Scripting, SPARK, SCALA, PYTHON, PYSPARK and SCALA to achieve the requirements. Ability to utilize the bigdata tools and techniques to integrate traditional databases like Oracle, SQL Server and Mysql into Hadoop Datalake. Create the Hive databases and store the processed data into Hive tables and then perform the aggregate operations to generate the new datasets and analyze the data. Develop an Enterprise Data Lake Application using Bigdata tools like Hive, Spark Streaming, Kafka, Sqoop, Oozie, Spark, Shell Scripting, Python, SQL, HDFS, Streamsets and Tidal.Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. Provide automation solutions for the jobs which are run manually to generate frequent reports which are utilized by the business team for data analysis.

Work Locations :

Various unanticipated work locations throughout the United States; relocation may be required. Must be willing to relocate.

Minimum Qualifications Education

• Bachelor – Electronics Engineering or Telecommunications

• Experience : None

• Job Order No : 8753175

• This position is eligible for a referral bonus through our employee referral program

• Position :

• Big Data Engineer

• Job Duties :

Responsible for designing and building big data platforms, tools, and solutions that help manage, secure, and generate value from data. Mostly involved in creating scalable and reusable solutions for gathering, collecting, storing, processing, and serving data at scales. On a day-to-day basis his responsibilities include:Gathering requirements for the business use case and design an implementation plan for the application.Provide different prototypes for the Business use case using Bigdata tools and let client’s Analysts team to confirm on best model for Data Analysis.Implement the use case with Bigdata open source tools and technologies like SQOOP, HADOOP, HIVE, Unix Scripting, SPARK, SCALA, PYTHON, PYSPARK, SCALA, AWS and EMR to achieve the requirements.Create the Hive databases and store the processed data into Hive tables and then perform the aggregate operations to generate the new datasets and analyze the data.Provide Big Data Solutions in Cloudera Environment and act as a technical expert addressing problems related to system and application design, Data Lake, Data warehouse, performance optimization, code integration, Automation workflows, etc.Develop an Enterprise Data Lake Application using Bigdata tools like Hive, Spark Streaming, Kafka, Flume, Sqoop, Oozie, Spark, Shell Scripting, Python, SQL, HDFS, Streamsets.Develop the automation scripts for data visualization reports out of processed data to present the research for the end clients.Integrate and deploy the code into lower environments to check for the merge conflicts of the code.Involve in research on the latest open source big data tools to implement the client’s requirements at low cost and better performance.Implementing the code and integrating the code into the production servers.Contribute for knowledge share and transfer with the team members for better outputs and best implementation solutions.Provide automation solutions for the jobs which are run manually to generate frequent reports which are utilized by the business team for data analysis.

Work Locations :

Various unanticipated work locations throughout the United States; relocation may be required. Must be willing to relocate.

Minimum Qualifications Education

• Bachelor – Electronics Engineering or Telecommunications

• Experience : None

• Job Order No : 8746283

• This position is eligible for a referral bonus through our employee referral program