Data Engineer - PySpark (MOH-ITDG)

Position Overview

The Data Engineer designs, implements and oversees maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time processing, and analysis of information from structured and unstructured sources in a scalable, repeatable and secure manner

Role & Responsibilities

• Responsible for creating end to end data pipelines for data ingestion into the MOH data lake.
• Identify the desired state of coordinated data and information flow through the organisation
• Collaborate with stakeholders to understand needs for data structure, availability, scalability and accessibility
• Provide thought leadership to stakeholders in determining which data management techniques and solutions will enable the enterprise to achieve defined business goals
• Design the data flow channels and processing systems to extract, transform, load and integrate data from various sources
• Design data architecture to fulfill analytics use case needs
• Development of prototypes and proof of concepts for the selected solutions
• Research hardware and software needs to support selected solutions
• Maintain knowledge of existing and emerging data management principles/theories/techniques
• Provide advice on high-level Data Management or Big Data Analytics solution designs.
• Evaluate technologies/services, providing regular reporting on emerging trends, value add information and potential impact to the healthcare landscape

Requirements

• Minimum 8 years of experience in delivering data warehouse or advanced analytics solutions, especially in designing large Big Data technologies or analytics solutions.
• Minimum 4 years of relevant hands-on experience in delivering end-to-end industry analytics solutions, from conceptualisation to deployment, within the industry.
• Demonstrate good, in-depth knowledge in Big Data hardware/software products, frameworks and methodologies.
• Degree/Master in Computer Science, Information Technology, Computer Engineering or equivalent.
• Comfortable working with and staffing senior management, proficient in usage of Office - Productivity Suite Software (e.g. Microsoft Excel, Powerpoint, etc).
• Possess good verbal and written communication, analytical and conflict resolution skills with proven ability to translate complex, technical subjects into clear and concise communications to a variety of key stakeholders of different levels
• Strong ability to handle ambiguity and high work pressure
• Versatile in working independently as well as an effective team player
• Good knowledge of various analytics systems/applications and outputs within healthcare industry domains will be advantageous

Experience with 

• big data techniques and solutions (e.g. Hadoop, Hive, Spark)
• databases (e.g. Oracle, DB2, MS SQL, MySQL, Teradata, Greenplum)
• data repository design (e.g. operational data stores, dimensional data stores, data marts)
• data interrogation techniques (e.g. SQL, NoSQL)
• real-time structured and unstructured data storage and ingestion
• data quality tools and processes
• data transformation and terminology equivalence mapping
• data modelling for analytics (e.g. star schemas, snowflake schemas)
• Cloud data management solutions

Apply Now

NOTE: It only takes a few minutes to apply for a meaningful career in HealthTech - GO FOR IT!!