Senior Data Engineer

As part of our data architecture discussions, we have agreed to create a Proof of Concept (POC) to test one of the target scenarios. We are looking for a skilled Big Data Engineer with strong communication and collaboration skills to create this POC and help with scope gathering for the next phase.

Client Description

The client supports farmers by providing real-time actionable information to help them manage their herds. It also provides pet owners with smart devices and data that offer a better understanding of their pets’ activity and health needs, strengthening the bond between people and their animals. In addition, it helps conservationists safeguard natural environments and wildlife.

Leveraging decades of research and development experience across multiple markets, technologies, and species, along with mature development environments and quality assurance processes, the client continuously develops innovative solutions to support animal health, well-being, and conservation. Its expertise spans enhancing pet care, advancing animal healthcare, and supporting wildlife preservation through technology-driven solutions.

Details on Tech Stack

Required Skills

• Cloud: AWS
• Programming Language: Python
• Frameworks: Apache Spark, Apache Airflow

Nice-to-Have Skills

• Experience with data warehouses such as AWS Redshift
• Experience with time-series databases such as InfluxDB
• Experience running Apache Spark pipelines on Kubernetes

• • Experience building streaming data pipelines in Apache Spark consuming data from AWS Kinesis or Apache Kafk