Minimum of 4 years experience implementing large-scale big data production systems in an agile environment. High-volume deployment infrastructure.
Python is required. Experience with Test driven development is must.
Knowledge of PySpark is must.
Hadoop, Spark, HBase, Kafka and other big data technologies knowledge.
Relational databases and SQL.
Knowledge of Travis, Jenkins, Airflow, IaaS (AWS/ Azure/ Google Compute) and CI/ CD is recommended.