Kafka Engineer

Must Have Skills - This specific role will be building and deploying Kafka clusters on both onPrem and Public cloud Kubernetes environments. Relevant experience > 10 yrs.

• Provide expertise in Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control center.
• Provide expertise and hands on experience working on Kafka connect using schema registry in a very high volume.
• Provide administration and operations of the Kafka platform - provisioning, access lists Kerberos and SSL configurations. Provide expertise and hands on experience working on Kafka connectors such as MQ connectors,
• Elastic Search connectors, JDBC connectors, File stream connector, JMS source connectors, Tasks, Workers, converters, Transforms.

Detailed Job Description - Work alongside product owners, technical leads and customers to strategize 3?5-year plans for streaming software.

• Produce and maintain product and platform architecture in line with the central architecture governance functions.
• Implement patterns that provide customers with robust deployment patterns meeting HSBC service levels tiers.
• Ensure that platform and product architectures remain cost effective.
• Standing up and administer On-Prem clusters.
• Provide expertise in Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control center.
• Provide expertise and hands on experience working on Kafka connect using schema registry in a very high volume.
• Provide administration and operations of the Kafka platform - provisioning, access lists Kerberos and SSL configurations. Provide expertise and hands on experience working on Kafka connectors such as MQ connectors,
• Elastic Search connectors, JDBC connectors, File stream connector, JMS source connectors, Tasks, Workers, converters, Transforms.
• Provide expertise and hands on experience on custom connectors using the Kafka core concepts and API.
• Involve in design and capacity review meetings to provide suggestions in Kafka usage.
• Participate in work planning and estimation to ensure optimum performance, high availability, and stability of solutions.
• Create topics, setup redundancy cluster, deploy monitoring tools, alerts and has good knowledge of best practices.
• Create stubs for producers, consumers, and consumer groups for helping onboard applications from different languages/platforms.
• Use automation tools like provisioning using Docker, Jenkins, and GitLab.
• Setting up security on Kafka, Monitor, prevent and troubleshoot security-related issues.
• Ability to perform data related benchmarking, performance analysis and tuning.