Java Developer ( Big Data project ) Offline

The primary purpose of this platform is to calculate various trading analytics metrics in both real-time and batch modes to feed this data to trading systems and reporting dashboards for quantitative traders.

Responsibilities:

The candidate will develop high-performance calculation engine platform which processes huge volumes of data in streaming and batch modes as a part of a co-located team in Warsaw with close interaction with other teams over the globe.

- Analysis of existing systems to be replaced with new solution

- Design and architecture of the new platform

- Development of components to support existing functionality

- Creating components to implement new business demands

- Scalability and performance optimization

- Integration with various external data sources (Storm, JDBC, KDB+, websockets, REST services, flat files)

- Creating high-performance in-memory intermediate layer (Spark streaming, Redis)

- Organizing data logistics and processing (Kafka, Spark)

- Organizing efficient persistent data storage for reporting (HDFS, AVRO, parquet)

- Creating high-performance data processing engines both near real-time and batch (Spark, Spark Streaming)

Mandatory Skills:

Skills required:

- Java 8 (+Spring) 3+ years

- Parquet, AVRO

- HDFS, Hive, HBase, Impala

Nice-to-Have Skills:

Nice to have:

- Scala, Python

- Spark, Spark Streaming 2+ years

- Kafka 1+ years

- Apache Storm

- JDBC, KDB+, websockets, REST

- Redis, in-memory storage