Sr. Software Engineer - Data Warehouse - ELT

Transguard Group was established in 2001 and has diversified significantly, we lead in the fields of Cash Services, Security Services, Manpower Services, Facilities Management & Aviation. Transguard Group is the UAE’s most trusted business support and outsourcing provider and has a large, dynamic and culturally diverse workforce with 50,000 employees!  We are currently recruiting for a Sr. Software Engineer - Data Warehouse - ELT for our trusted client in Aviation industry to be based out of their headquarters in Dubai.

Job Purpose

• Data Ingestion Engineer - Responsible for developing ETL ingestion pipelines and data streams from structured and non-structured (IoT) data sources
• Data Warehouse Architect - Ability to design Dimensional Data Warehouse model in star and snowflake schema
• Responsible for data transformation from operational 3NF store to analytical data sets, including data validation, query optimization ensuring data consistency and system performance
• Design and develop streaming data ingestion pipelines using Change Data Capture for data replication in real-time
• Design and develop ETL pipelines for handling large amount of data from various data sources
• Worked on the data integration and data migration projects
• Automate, optimize and fine tune ingestion pipelines to gain optimal performance
• Develop and design APIs
• Skilled Big Data Engineer- experienced in big data platform
• Ensure data quality, data integrity and data availability
• Accountable for managing delivery of assigned project(s), including scope, schedule and quality management, and status reporting.  Manages requirements gathering, communication and risk assessment. Complies with company's Project Management & Project governance practices.

• Bachelor’s Degree holder in computer science or related field
• Minimum 8 years IT experience covering development, solution design, configuration, end user testing across business applications for large complex organizations
• Minimum 4 years in software development and systems testing using structured methodologies

Core Activities Experience

• StreeamSets Data Collector, Kafka ,DBT ,Airflow, PostgreSQL + Citus
• Data Warehousing and Dimensional Modeling

Tools and Systems Experience:

• Design, build and deploy effective data ingestion pipelines/streams in Streeamsets Data Collector & Kafka
• Implement stored procedures and effectively query a different database
• Manage distributed database systems like Teradata, Oracle Exadata , PostgreSQL + Citus
• Excellent knowledge in Dimensional Data Modelling, Star and Snowflake schema
• Develop and organize data transformations in DBT and Apache Airflow
• Translate requirements from the business and translate into technical code
• Identify and test for bugs and bottlenecks in the ETL solution
• Ensure the best possible performance and quality in the packages
• Provide support and fix issues in the packages
• Excellent written and verbal communication in English
• Experience forming and maintaining relationships with Senior Leaders and external stakeholders
• Knowledge of the aviation industry and aviation solutions is preferable