Lead - Senior Scala/Java/HDFS Developer Offline

Detailed:

Data engineer can be specialized in either NoSQL data stores (Cassandra, HBase etc) or Spark & Spark streaming - need both specializations

3+ years of professional experience with Big Data systems, pipelines and data processing

Hands on experience Big Data, data ingestion, data processing using Spark, Spark Streaming, Flink, HIVE, Kafka, Hadoop, HDFS, S3

Hands-on experience with design and development with NoSQL technologies Cassandra, HBase or similar scalable Key valueStores and time series data stores like Druid, influx or similar

Understanding on various distributed file formats such as Apache AVRO, Apache Parquet and common methods in data transformation

Confirmed understanding of design and development of large scale, high throughput and low latency applications is a plus

Understanding and experience with Micro Services is desired

Excellent problem solving and programming skills

Experience with containerization technologies like Kubernetes, Docker, Mesos, Marathon is desirable

Experience with CI/CD, debugging and monitoring applications and big data jobs is desirable