Regular/Senior Data Engineer, Zaandam (offline)

PROJECT DESCRIPTION
We are looking brand new Data Analytics team for new customer.
In essence, client is leveraging the Azure PaaS platform with all kind of (business) data to build an advanced analytics platform aiming at delivering better insights and applications to the business.
The platforms are continuously being enhanced to support (additional) CI/CD and validated learning environment for science, machine learning and AI capabilities for all areas customer-facing like digital omni-channel interaction and commerce, commerce relevance, personalisation, loyalty and marketing and non-customer-facing like assortment optimization, supply chain optimization, external parties and IoT.
We will be working on end to end functionality including architecture, data preparation, processing and consumption by systems.

RESPONSIBILITIES
• Designing and implementing batch and real-time data ingestion pipelines for Azure Data Lake
• Implementing data quality checks
• Implementing data lineage and reconciliation mechanisms (including monitoring dashboards and alerts)
• Implementing data cataloguing, archiving and disaster recovery
• Working with data source and data consumption teams to align on data structures and schemas

SKILLS

MUST
• General knowledge of PaaS offerings Azure platform
• Azure Data Storage tools (Azure SQL, CosmosDB, Storage Account)
• Azure Analytics Data Platform (Data Factory, Data Lake, Data Warehouse)
• Azure Event hubs, Azure Stream Analytics
• Azure Databricks (with pyspark) or HD Insights

NICE TO HAVE
• C#/.NET
• Retail industry experience

LANGUAGES
English: Advanced/Fluent