Data Engineer

Data Engineer - HR Data Platform

Human Resources Department

Chennai, India.

We are looking for a skilled Data Engineer with 3–4 years of experience in designing, developing, and maintaining cloud-based data integration solutions on Microsoft Azure.

The ideal candidate should have hands-on experience building Azure Data Factory (ADF) pipelines, integrating data from REST APIs, developing SQL solutions, and creating Power BI reports. The role requires someone who can independently handle ad hoc data requests, troubleshoot production issues, and maintain code using Azure DevOps.

Key Responsibilities

Design, develop, and maintain data ingestion pipelines using Azure Data Factory (ADF).

Develop pipelines to extract data from REST APIs, including handling:

·       Authentication mechanisms (OAuth, API Keys, Bearer Tokens)

·       Pagination

·       Incremental and historical data loads (multiple months of data)

·       Error handling and retries

Load and transform data into Azure target systems such as:

·       Azure Blob Storage

·       Azure SQL Database

Develop and optimize SQL queries, stored procedures, views, and data transformations.

Handle ad hoc data extraction, transformation, and reporting requests from business stakeholders.

Monitor scheduled pipelines, investigate failures, and perform production support activities.

Maintain source code, pipelines, and deployment artifacts using Azure DevOps with proper version control and branching strategies.

Collaborate with business users to understand reporting requirements and build interactive Power BI dashboards and reports.

Ensure data quality, consistency, and completeness across ingestion and reporting processes.

Participate in code reviews and follow development best practices.

 

 

 

 

Required Skills

Azure

·       3–4 years of hands-on experience with Microsoft Azure.

·       Strong experience with Azure Data Factory (ADF).

·       Experience creating and scheduling pipelines, triggers, parameterized pipelines, and reusable datasets.

·       Good understanding of linked services and integration runtimes.

·       Experience integrating data from REST APIs.

·       Ability to work with JSON payloads.

·       Experience handling:

o   Pagination

o   Rate limiting

o   Authentication

o   Historical backfills

o   Incremental loads

·       Azure DevOps

o   Experience maintaining code repositories in Azure DevOps.

o   Knowledge of Git branching strategies.

o   Experience with pull requests and deployment pipelines.

o   Data Storage

o   Experience working with:

§  Azure Blob Storage

§  Azure SQL Database

 

SQL

·       Strong hands-on experience in SQL.

·       Experience writing:

·       Complex queries

·       Stored procedures

·       Views

·       Joins

·       Window functions

·       Performance tuning

 

Power BI

·       Experience developing Power BI dashboards and reports.

·       Ability to create data models, measures (DAX), and visualizations.

·       Experience connecting Power BI to Azure SQL and Azure Blob data sources.

 

Preferred Qualifications

 

·       Experience working with Python for data processing is a plus.

·       Familiarity with Azure Functions or Logic Apps is an advantage.

·       Understanding of ETL/ELT design principles.

·       Experience working in Agile/Scrum environments.

·       Knowledge of data warehousing concepts and dimensional modelling.

·       Strong analytical and problem-solving skills.

·       Ability to independently own development tasks.

·       Good communication and stakeholder management skills.

·       Ability to manage multiple priorities and ad hoc requests.

·       Proactive approach to troubleshooting and production support.

 

Experience

·       3–4 years of experience as a Data Engineer or ETL Developer.

·       Hands-on experience with Azure data integration technologies.

·       Experience delivering end-to-end data pipelines and reporting solutions.

Who we are

Glencore is one of the world’s largest global diversified natural resource companies and a major producer and marketer of more than 60 commodities. Through a network of assets, customers and suppliers that spans the globe, we produce, process, recycle, source, market and distribute the commodities that advance everyday life.

 

With over 140,000 employees and contractors and a strong footprint in over 30 countries in both established and emerging regions for natural resources, our marketing and industrial activities are supported by a global network of offices.

 

Glencore’s customers are principally industrial consumers, such as those in the automotive, steel, power generation, battery manufacturing and oil sectors. We also provide financing, logistics and other services to producers and consumers of commodities.

Important informationThis material does not purport to contain all of the information you may wish to consider. For further important information, including in connection with forward-looking statements and other cautionary information, refer to the Important notice section of Glencore’s 2025 Annual Report, which is available at glencore.com/publications. By their nature, forward-looking statements involve known and unknown risks, uncertainties and other factors which may cause actual results, performance or achievements to differ materially from any future events, results, performance, achievements or other outcomes expressed or implied by such forward-looking statements. This document does not constitute or form part of any offer or invitation to sell or issue, or any solicitation of any offer to purchase or subscribe for any securities. Other informationThe companies in which Glencore plc directly and indirectly has an interest are separate and distinct legal entities. In this document, “Glencore”, “Glencore group” and “Group” are used for convenience only where references are made to Glencore plc and its subsidiaries in general. These collective expressions are used for ease of reference only and do not imply any other relationship between the companies. Likewise, the words “we”, “us” and “our” are also used to refer collectively to members of the Group or to those who work for them. These expressions are also used where no useful purpose is served by identifying the particular company or companies.​Note*: All applications will be subject to potential background checks. You will be informed if this will be the case. Further information about the processing of your personal data will be made available during the application process. We'll seek your explicit consent prior to processing the data.