Job Description

Location: Preferred location is Montreal. For very strong profiles we would consider Quebec, Ontario, Atlantic provinces. As a Data Engineer you will be responsible for designing and implementing code for a new big data platform. You will be working with a team of data scientists and data engineers to provide an advanced security monitoring solution for Bell. Your responsibilities will be to support the architecture team to realise this ambitious new programme of work. You will need to be comfortable building both new systems and implementing and integrating with existing open source and commercial products. Wherever possible you will aim to support continuous integration and deployment (CI/CD). Part of your daily work will be to challenge certain concepts and ideals. The goal is to help your peers' knowledge to evolve and to open their minds to new approaches. Your responsibilities: • Developing a log enrichment pipeline capable of functioning at massive scale. • Write code for data enrichment and normalization and deploying it to Openshift • Configure Kafka and other components for handling data pipeline • Integrating machine learning models onto a Hadoop based big data environment using streaming technology. • Building connectivity with Elastic Stack based analytics. • Working with data scientists to develop an entirely new security risk scoring function. • Liaise with the DevOps team maintaining the underlying hardware and supporting technologies. • Implement automation tools and mentoring with a DevSecOps mindset • Maintain up-to-date documentation on the systems and procedures Required skills: • University degree in Computer Science or five years' equivalent experience in Information Technology • Previous experience writing code for use at scale i.e. 'big data' - Information security related applications e.g. SIEM a benefit. • Proven track record of developing effective and efficient real-time big data processing pipelines • Experience with Hadoop, Cloudera or Hortonworks. • Proven experience with OpenShift or Kubernetes • Experience with Apache Spark and Kafka. • Desired experience with the Elastic Stack. • Strong ability to develop in Python. • Experience with CI/CD • Skilled thinker, proactive, team-player, highly resourceful and detail oriented. • Able to track the work by following agile methodologies