Data Engineer Full-time Job

Role & Responsibilities:

Design and Develop PySpark-Based Data Pipelines:

You will design, develop, and maintain PySpark-based data pipelines for processing large datasets. You will write efficient and optimized PySpark code to perform data transformations and manipulations. You will ensure that the data pipelines are scalable, reliable, and efficient.

Collaborate with Data Scientists and Analysts:

You will work closely with data scientists & analysts to understand their data needs and develop solutions that enable them to gain insights from data. You will work with senior and lead data engineers and help them to extract, transform, and load data from various sources and perform exploratory data analysis to identify patterns and insights.

Perform Exploratory Data Analysis:

You will participate and perform exploratory data analysis to identify patterns and insights that can help in making informed decisions. You will use various statistical and visualization techniques to identify trends, correlations, and anomalies in the data, as needed.

Develop and Maintain Data Quality Checks:

You will develop and maintain data quality checks to ensure the accuracy and completeness of data. You will monitor the health of the data pipelines and ensure that they are running smoothly.

Integrate Data Pipelines into Production Systems:

You will work with software engineers to integrate data pipelines into production systems. You will ensure that the data pipelines are integrated seamlessly and that they meet the performance and scalability requirements of the production systems.

Apply Knowledge of Distributed Computing:

You will apply basic knowledge of distributed computing to design and optimize data processing solutions that can handle large volumes of data. You will understand the concepts of distributed computing and their implications on data processing.

Understand and Follow Software Lifecycle:

You will understand and follow the software lifecycle to develop, test, and deploy data processing solutions. You will follow best practices in software engineering to ensure that the solutions are reliable, maintainable, and scalable.

Emirates NBD (DFM: Emirates NBD) is a leading banking group in the MENAT (Middle East, North Africa and Turkey) region with a presence in 13 countries, serving over 17 million customers. As at 31st December 2022, total assets were AED 742 billion, (equivalent to approx. USD 202 billion). The Group has operations in the UAE, Egypt, India, Turkey, the Kingdom of Saudi Arabia, Singapore, the United Kingdom, Austria, Germany, Russia and Bahrain and representative offices in China and Indonesia with a total of 879 branches and 4,130 ATMs / SDMs. Emirates NBD is the leading financial services brand in the UAE with a Brand value of USD 3.6 billion. Emirates NBD Group serves its customers (individuals, businesses, governments, and institutions) and helps them realise their financial objectives through a range of banking products and services including retail banking, corporate and institutional banking, Islamic banking, investment banking, private banking, asset management, global markets and treasury, and brokerage operations. The Group is a key participant in the global digital banking industry with 97% of all financial transactions and requests conducted outside of its branches. The Group also operates Liv., the lifestyle digital bank by Emirates NBD, with close to half a million users, it continues to be the fastest-growing bank in the region.Emirates NBD contributes to the construction of a sustainable future as an active participant and supporter of the UAEs main development and sustainability initiatives, including financial wellness and the inclusion of people of determination. Emirates NBD is also an early supporter to Dubai Can sustainability initiative, a city-wide initiative aimed to reduce use of single-use plastic bottled water.