Nicoll Curtin are currently looking for an experienced senior Big Data Engineer with a depth of experience in leading multi-discipline teams through the full delivery lifecycle of complex data products and pipelines.
The candidate will have a breadth of experience in both engineering and architecture across technology disciplines and the unique challenges of each, including software development, automated test and quality assurance, data and integration.
The candidate must have significant hands-on experience in the design and implementation of software with most of the following:
- Proven knowledge of the Data Science Lifecycle
- Demonstrable experience of these software engineering languages Java 8, Scala 10/11, Maven / SBT
- Python (preferably with pandas, numpy and/or sci-kit learn)
- Great understanding of the processes and principles of machine learning systems
- Good knowledge of HPC/Batch computing and GPU offload scenarios for MPI/parametric sweep
- Strong knowledge of relational databases and strong associated knowledge of TSQL, plus NoSQL document and time series databases including CosmosDB, MongoDB, Cassandra
- Knowledge of data orchestration technologies such as Oozie, AirFlow, Apache NiFi or Azure Data Factory
- Sound knowledge of the Apache Big Data stacks such as Hadoop and Spark including, HDFS MapReduce, YARN
- Good understanding of real-time streaming technologies such as Apache Kafka, Azure EventHub, Spark Streaming, Apache Storm, Apache Flink etc
- Strong knowledge of Microsoft Azure data capabilities (Azure Data Factory, Azure Stream Analytics, SQL DB / DW, CosmosDB, Azure Data Lake etc)
- Good data visualisation skills including proven usage of PowerBI or Tableau
If you would like to know more about this position then APPLY NOW
Java 8 Scala Maven Python C# Machine Learning TSQL NoSQL Apache Kafka Azure Spark Hadoop Tableau