BI Big Data Engineer At Discovery
Business Unit: Discovery Health
Function: Business Intelligence
Date: 10-Jan-2023
About Discovery
Discovery’s core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and whose own values align with those of Discovery. Our fast-paced and dynamic environment enables smart, self-driven people to be their best. As global thought leaders, Discovery is passionate about innovating in order to not only achieve financial success, but to ignite positive and meaningful change within our society.
About (DSU – Business Intelligence)
The Data Science Unit (DSU) – Business Intelligence is part of the Discovery Health Company. We are responsible for building all information assets and delivering information to the business through various reporting channels from an executive level down to the call center agent. The information assets that we are building is not only limited to report delivery, but also integrated information assets to support the data scientists, risk analysts and the data analytics teams.
Key Purpose
Interpret requirements provided by business and produce effective Big Data solutions. Programming exposure for data transformations and integrating the big data solution with existing systems. Develop information solutions from a variety of sources for both structured and unstructured data. Productionisation of machine learning models. Technical ownership of Big Data solutions for structured and unstructured data.
Areas of responsibility may include but not limited to
- Develop and implement big data models and solutions
- Research, prototyping and design
- Feature set engineering and automation
- Design and implement ETL/ELT methodologies and technologies and the integration with big data
- Work in an Agile environment and attend to daily standups, retros, and sprint planning
- Conduct root cause analysis on production issues
- Technical leadership of entire information management process of both structured and unstructured data
- Provide ongoing support and enhancement to ETL/ELT system
- Optimization of the solutions
- Implementing machine learning algorithms in production through integration
- Configuration of the Hadoop infrastructure and environment for optimal performance
- Integrate with statistical and actuarial analysts to build models
- Producing relevant technical documentation and specifications
- Estimate time and resource requirements for business requirement
- Integration of big data solutions with existing reporting and analytical solutions
- Optimization of machine learning models produced by Data Scientists.
- Develop data processing functions (DPF’s) using Java and Python
Education and Experience
- Matric (Essential)
- National Diploma in IT (BTech) or appropriate certification (Essential)
- Bachelor of Science (Information Systems, Computer Science, Mathematics) Advantageous
- SQL (Advanced)
- Data Warehouse principles and practices (Advanced)
- Docker and container setup (Intermediate)
- Linux Shell Scripting (Intermediate)
- Python/R/Scala Programming (Intermediate)
- Git versioning
- Flask/Django framework
- CI/CD
- Systems Development Life Cycle (SDLC) (Intermediate)
- Spark framework Configuration (Intermediate)
- Data Security and Protection Policies (Intermediate)
- MS Excel (Intermediate)
- Kimball Methodology (Intermediate)
- ETL development using SSIS, Python & Java (Intermediate)
- Java/.net Programming (Advantageous)
- Big Data using Hadoop (Advantageous)
- Big Data Ingestion using Sqoop/Kafka (Advantageous)
- Linux administration (Advantageous)
- Distributed programming skills on a cluster environment (Advantageous)
EMPLOYMENT EQUITY
The Company’s approved Employment Equity Plan and Targets will be considered as part of the recruitment process. As an Equal Opportunities employer, we actively encourage and welcome people with various disabilities to apply.
