Data Engineer

Data EngineerOn behalf of our client, Procom Consultants Group is currently looking for an Intermediate Data Engineer. Requested resources to provide developer expertise on coding of data transformation pipelines and systems integration tasks relating to enterprise data management applications that span across different business areas and platforms.Data Engineer Job Details• Creating mapping documentation of data elements from source to target and liaising with other Data Engineers, Data Architects and SMEs.• Developing & testing Data Transformation pipelines by leveraging the latest Big Data tools and technologies• Collaborate with upstream data teams, develop data validation and test plans, performing user acceptance testing, and provide feedback to development and sustainment teams• Conducting analysis for moderate to complex data requests, defining data fields and determining data availability, developing information layout, format and interactivity. Presenting findings and providing clarification.• Act as a lead developer to implement code changes in various environments and platforms leveraging our DevOps & DataOps• Work closely with the testing team to assist with issues raised in user acceptance testing (UAT) and Business Value Testing (BVT)Deliverables: Fully implement the new data format across all IM supported platforms includingBig Data (Hadoop), Enterprise Data Warehouse (EDW) and Operational Data Store(ODS)Data Engineer Mandatory SkillsCommunication & Problem Solving Skills: • Communicating & presenting solutions to internal developers• Ability to apply critical thinking skills to develop efficient code, data transformation logic and solution design• Experience and/or understanding of Agile MethodologiesTechnical Skills: • Advanced Database Skills & SQL• Experience with or exposure to Big Data platforms, ideally exposure to Hadoop ecosystem (HDFS, Apache Hive, Apache Spark, SparkSQL)• Extensive programming experience in either Java, C++ or preferably Scala, IntelliJ, Jenkins• Working experience with Apache Spark Data Processing Framework, Parquet data format• Good understanding of Relational (Oracle/DB2) and Hadoop Big Data platforms• Experience working with JiraData Engineer Start DateOct 24, 2022 Data Engineer Assignment DurationMarch 31 2023Data Engineer Working LocationFully remote but PST hours