Data Engineer (Python)
Dataforest is seeking an experienced Data Engineer (Python) to join our dynamic team. You will be responsible for developing and maintaining data-processing architecture, as well as optimizing and monitoring our internal systems.
Requirements:
- 1,5+ years of commercial experience with Python.
- Experience with ElasticSearch and PostgreSQL.
- Knowledge and experience with Redis, Kafka, and SQS.
- Experience setting up monitoring systems with CloudWatch, Prometheus, and Grafana.
- Deep understanding of algorithms and their complexities.
- Excellent programming skills in Python with a focus on optimization and code structuring.
- Knowledge of ETL principles and practices.
- Ability to work collaboratively and communicate effectively.
- Experience with Linux environments, cloud services (AWS), and Docker.
Will be plus:
- Knowledge in web scraping, data extraction, cleaning, and visualization.
- Understanding of multiprocessing and multithreading, including process and thread management.
- Experience with Flask / Flask-RESTful for API development.
Key Responsibilities:
- Develop and maintain data processing architecture using Python.
- Efficiently utilize ElasticSearch and PostgreSQL for data management.
- Implement and manage data pipelines using Redis, Kafka, and SQS.
- Set up and monitor logging systems using CloudWatch, Prometheus, and Grafana.
- Optimize code and improve its structure and performance.
- Understand and implement ETL processes.
- Analyze algorithms and code complexity to enhance efficiency.
- Work with the AWS stack to ensure flexibility and reliability in data processing.
We offer:
- Great networking opportunities with international clients, challenging tasks;
- Building interesting projects from scratch using new technologies;
- Personal and professional development opportunities;
- Competitive salary fixed in USD;
- Paid vacation and sick leaves;
- Flexible work schedule;
- Friendly working environment with minimal hierarchy;
- Team building activities and corporate events.