Job Title: AWS Infrastructure/Data Engineer (Hybrid, UK)
Below covers everything you need to know about what this opportunity entails, as well as what is expected from applicants.
Hybrid: Remote + 1 day a week in West midland office location
Clearance: DV
JobID: 02
Company Overview:
Atreides helps organizations transform large and complex multi-modal datasets into information-rich, geo-spatial data subscriptions used across various use cases. We focus on providing defense intelligence professionals with high-fidelity data solutions to derive insights quickly. As a fast-growing, high-performance early-stage company, we value diversity and inclusion, trust, and autonomy. A mission-driven mindset and entrepreneurial spirit are essential as we work to unlock the power of massive-scale data for a safer, stronger, and more prosperous world.
Team Overview:
We are a passionate team of technologists, data scientists, and analysts with backgrounds in operational intelligence, law enforcement, large multinationals, and cybersecurity operations. We obsess about designing products that will change the way global companies, governments and nonprofits protect themselves from external threats and global adversaries.
Position Overview:
We are looking for a skilled AWS Infrastructure/Data Engineer with a focus on Infrastructure as Code (IaC) and data engineering. Your role will involve designing, implementing, and maintaining scalable, secure AWS cloud infrastructure, as well as creating data pipelines and storage solutions. You will ensure the reliability and availability of infrastructure for data processing on a petabyte scale and collaborate closely with our infrastructure engineers and data engineers to automate and streamline data processes.
Team Principles:
At Atreides, we believe that teams work best when they:
- Remain curious and passionate in all aspects of our work
- Promote clear, direct, and transparent communication
- Embrace the 'measure twice, cut once' philosophy
- Value and encourage diverse ideas and technologies
- Lead with empathy in all interactions
Responsibilities:
- Design, implement, and maintain scalable, secure AWS infrastructure with a focus on Infrastructure as Code (IaC).
- Design, develop, and maintain scalable data pipelines using Spark, Python, Java and. Nice to have: Scala, Golang
- Implement and manage Iceberg tables, ensuring efficient data storage and retrieval.
- Optimize data storage solutions, including hot, cold, and glacier storage tiers.
- Develop and enforce data retention policies and ensure compliance with data governance standards.
- Collaborate with software engineers to ensure the infrastructure effectively supports application requirements.
- Ensure data security and implement necessary measures to protect sensitive information.
- Monitor and troubleshoot data pipelines and infrastructure to ensure high availability and performance.
- Document infrastructure design, data engineering processes, and maintain comprehensive documentation.
Desired Qualifications:
- 3+ years of experience in AWS Infrastructure and/or SRE roles, with applicable Associate or Professional AWS certificates being a plus.
- Strong background in Infrastructure as Code (IaC) using tools such as Pulumi, Terraform, or CloudFormation.
- Experience with data orchestration tools such as Airflow, Prefect, Dagster, or Temporal.
- Proficiency in Spark, Java, and Python. Nice to have: Scala, Golang
- Experience with Postgres, GraphQL, and other data manipulation tools.
- Knowledge of big data tools and environments, including Apache Iceberg and other datalake concepts.
- Familiarity with geospatial data formats such as Parquet/GeoParquet, GeoJSON, and Shapefiles.
- Experience with DevOps tools such as Git, Docker, Jenkins, etc.
- Knowledge of networking concepts such as IP Protocol, DNS, and load balancing.
- Excellent problem-solving skills and the ability to think quickly in a high-performance environment.
- Effective communication skills to convey technical concepts to both technical and non-technical stakeholders.
Compensation and Benefits:
? Competitive salary
? Comprehensive health, dental, and vision insurance plans
? Flexible hybrid work environment
? Additional benefits like flexible hours, work travel opportunities, competitive vacation time and parental leave
While meeting all of these criteria would be ideal, we understand that some candidates may meet most, but not all. If you're passionate, curious and ready to "work smart and get things done," we'd love to hear from you.
Remote working/work at home options are available for this role.