- Our client is looking for a Databrick's Engineer to work on greenfield projects across Data Lake integrations and data transformations.
- Continuously improve the data load performance and presentation of data to external systems and users
- Assist in planning and implementation of software releases or changes to the environment
- You will be responsible for application and end-to-end technical support for the data lake.
- Previous experience in data validation, system monitoring and improving data load performance is highly beneficial.
- Upskilling peers and sharing knowledge on Databricks and Azure.
- Extensive Experience with Databricks or Apache Spark – Databricks preferable
- Knowledge of Spark/Databricks configuration and management
- Experience managing and setting up Spark streaming and batch processing
- Knowledge of Azure – Datalake, Data Factory
- Experience with Microsoft SQL Server
- Good understanding of Security principles, certificates, firewalls and PCI