Senior Data Engineer

PositionOpsGuru is a global engineering and consulting group. We are experts in the container ecosystem, data processing and analytics, cloud-native technologies, data science, and machine learning. Our team is formed by network, data, security, DevOps, AI specialists and application developers. OpsGuru empowers customers with technology to solve their business problems and provide the tools to assure success in their digital transformation.As a Data Engineer, you will be part of the OpsGuru professional services team architecting, designing, automating, and deploying production-grade data services and flows on behalf of our customers to a variety of public clouds such as AWS, Google Cloud and Azure.IncludesBuilding and optimizing ETL/ELT processes for efficiency and reliability, employing modern tools such as Dbt, Glue, and custom scripts in Python, Scala, or Java.Building robust data pipelines that integrate data science and machine learning models, leveraging Spark, Airflow, and MLflow.Implementing and maintaining scalable data storage and computation solutions, including Data Lakes, Data Warehouses, and NoSQL databases, to support advanced analytics and AI-driven applications.Developing and maintaining real-time data processing and analytics platforms using Kafka, Spark Streaming, Kinesis, Pub Sub, and other streaming technologies to enable instant insights and decision-making.Establishing credibility and build impactful relationships with our customersQualification & Experience5+ years of hands-on experience in data engineering, with a proven track record of designing and implementing large-scale data processing and analytics systems.2+ years of experience working with AI/ML models in production environments, including experience with machine learning frameworks (e.g., TensorFlow, PyTorch) and model deployment and monitoring tools.3+ years’ of experience in production database architectures (such as MySQL, PostgreSQL, MS SQL Server, Oracle) and data pipeline developmentDemonstrated knowledge of software development tools and methodologiesDeep understanding of data modeling, data architecture, and data storage strategies for both structured and unstructured data.Strong foundation in distributed systems and big data technologies, with extensive experience in at least a few of the following: Spark, Hadoop, Kafka, Airflow, MLflow, Cassandra, ElasticSearch.Expertise in cloud platforms and services for data and ML workloads (AWS, Google Cloud, Azure), including managed services (e.g., AWS Sagemaker, Google AI Platform, Azure ML) and serverless computing.Excellent problem-solving abilities and strong communication skills, capable of effectively articulating complex technical concepts to non-technical stakeholders.A Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, Statistics, or a related field, with additional certifications or coursework in data science and machine learning highly regarded.Demonstrated ability to adapt to new technologies and learn quickly