DSRCprovides competitive compensation that is revised purely on performance, flexible work hours & friendly work environment. At DSRC you will have opportunity not only to learn but also explore your area of interest with respect to technology and also effectively use the skills acquired over few years of IT experience.Experience: 5 to 7 yearsRequirement: Data EngineerResponsibilities:Develop an understanding of core business data from the perspective of value to the customer and meeting the needs of the customer Data Strategy.Design and deliver centralised data infrastructure including Data Lake, Data Warehouses and Marts, ETL pipelines, and so on using AWS cloud technologies.Maintain high levels of data quality by developing and applying data quality measures. Develop methods of publicising data quality measures and methods to improve quality.Ensure the highest levels of security are applied to all data management systems, recommend data security and privacy controls, identify and resolve weaknesses in existing data infrastructure, manage the preparation and implementation of improved security procedures.Acquire relevant data from third party sources and integrate into core data sets to enrich offerings, e.g. weather data, traffic data, etc.Work with our academic partners in the development of new analytic algorithms and provide thought leadership within the team on data management practices, technology, and analytics.Mentor junior team members and facilitate skills development and transfer.Stay up to date with new data tools and technologies and facilitate their introduction.Qualifications:Senior level experience delivering enterprise-grade Data Engineering and Data Infrastructure solutions for analytics.Strong data engineering skills, experienced in technologies like Data Lakes, Data Warehouses, AWS, Glue, Lambda, RDS, Athena & Quicksight ETL, Hadoop and Spark, Relational and Non-Relational databases.Experienced in SQL / NoSQL, Python, and messaging or data streaming technologies (Kafka, ActiveMQ or similar).Familiarity with concepts and techniques for wrangling multiple data sources into cohesive data sets.Familiarity with DAMA Data Management principles an advantage.Experience working in an Agile/Scrum development process.Familiarity with JIRA, Gitlab, Confluence, MS Teams/Slack.Strong interpersonal skills, especially written and verbal communications and a proactive ability to work with local colleagues and remote teams.Bachelor/Master/Engineering degree in IT / Computer Science/ software engineering or relevant field.Experience building and optimizing big data data pipelines, architectures and data sets, preferably on AWS data lake.Build processes supporting data transformation, data structures, metadata, dependency and workload management.AWS Certification will be an advantage.
