Returning Candidate?

Data Engineer

Data Engineer

Job ID 
Posted Date 
Amazon Data Services Ireland Limited
Recruiting Team 

Job Description

Does the challenge of designing scaleable big-data systems to detect and prevent advanced security threats excite you? Do you enjoy working in an agile entrepreneurial environment solving complex technical problems and delivering innovative solutions?

The Security Intelligence team, part of Information Security at Amazon, enables the monitoring of and response to malicious activity on the Amazon network. We are seeking a Data Engineer who is excited to work on a talented team to solve challenging problems related to protecting Amazon and our customer’s data. Our challenges are broad and deep -- we need to build a flexible, secure, scalable, high-performance and robust security platform. The platform provides a set of services that enable our security engineers and analysts to detect and respond to potential malicious activity on our network. The platform processes massive quantities of data to produce actionable security intelligence by enabling advanced threat matching, graph modeling, statistical and machine learning techniques. We protect customer data from adversaries that want it.

As a Data Engineer on’s Security Intelligence team, you will be building the next generation of our security data platform, and drive the adoption of new technologies and new practices in existing implementations. You will be responsible for designing and implementing complex ETL pipelines and other BI/Security solutions to support the rapidly growing and dynamic demand for data. You will deliver the data as service solutions which will have an immediate influence on day-to-day security decision making at

· Interfacing with customers, gathering requirements and developing new datasets in our data repositories
· Building and managing our complex ETL pipelines
· Optimizing the performance of business-critical queries and dealing with ETL job related issues
· Tuning application and query performance on petabyte scale data-stores
· Identifying any data quality issues to address them immediately and provide a great user experience
· Extracting and combining data from various heterogeneous data sources
· Designing, implementing, and supporting a platform that can provide ad-hoc access to very large datasets
· Modeling data and metadata to support ad-hoc and pre-built reporting

Basic Qualifications

· A desire to work in a collaborative, intellectually curious environment.
· Degree in Computer Science, Engineering, Mathematics, or a related field and 5 years industry experience
· Demonstrated ability in data modeling, ETL development, and data warehousing.
· Experience with Big Data Technologies (Hadoop, Hive, Hbase, Pig, Spark, etc.)

Preferred Qualifications

· Industry experience as a Data Engineer or related specialty (e.g., Software Engineer, Business Intelligence Engineer, Data Scientist) with a track record of manipulating, processing, and extracting value from large datasets.
· Coding proficiency in at least one modern programming language (Python, Ruby, Java, Scala etc)
· Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
· Experience building data products incrementally and integrating and managing datasets from multiple sources
· Query performance tuning skills
· Experience leading large-scale data warehousing and analytics projects, including using AWS technologies – Redshift, S3, EC2, Data-pipeline and other big data technologies
· Experience providing technical leadership and mentor other engineers for the best practices on the data engineering space
· Linux/UNIX including to process large data sets.
· Experience with AWS