QA & AI Engineer

About the position

A fast-growing and technology-driven company is seeking a detail-oriented and analytical QA & AI Engineer to join their remote team. This role reports directly to both the General Manager and Engineering Lead and offers an exciting opportunity for a technically minded professional who is passionate about AI systems, quality assurance, automation, and continuous improvement. The successful candidate will play a critical role in ensuring the quality, performance, and optimisation of AI agents and automated systems while supporting broader operational and technical initiatives. This position is fully remote, with a requirement to attend quarterly team meetings in Randburg.

Responsibilities:

AI Agent Monitoring & Optimisation:

• Regularly monitor AI agent interactions to ensure accuracy, correct tone, and compliance with company standards and guidelines.

• Continuously update AI agents with new customer, product, and platform information to improve relevance and response quality.

• Assist with the setup, configuration, and rigorous testing of new AI agents prior to deployment into live environments.

• Proactively identify and resolve logic gaps, workflow issues, and recurring AI-related errors.

• Act as an escalation point for customer queries where AI systems are unable to provide appropriate responses.

Quality Assurance & Automated Testing:

• Review and verify ticket descriptions written for automated testing to ensure technical accuracy, clarity, and completeness.

• Apply platform knowledge and understanding of customer setups to ensure automated testing scripts accurately reflect real-world usage scenarios.

• Support continuous improvement of automated testing processes and standards.

Systems Testing:

• Conduct full internal testing prior to the release of new enhancements, fixes, and platform updates.
• Assist clients with testing and validation processes where required.
• Create and maintain detailed documentation relating to system rules, workflows, and testing procedures.

Reporting & Continuous Improvement:

• Use reporting tools and analytics to monitor AI performance metrics, customer satisfaction trends, and operational effectiveness.

• Prepare comprehensive monthly reports detailing AI performance, QA findings, support metrics, and improvement opportunities.

• Analyse AI interactions and QA audit outcomes to identify trends and recommend process enhancements for both automated systems and manual support functions.

Operational & Business Support:

• Provide assistance with ad hoc operational, administrative, and business support tasks when required.
• Participate in company projects and cross-functional initiatives outside the core QA and AI scope when necessary.
• The role requires flexibility and willingness to support broader business operations when needed.
• Maintain flexibility and adaptability within a fast-paced and evolving technology environment.

Requirements:

• NQF Level 6 qualification with relevant practical experience.
• Minimum of 3 years experience within a Quality Assurance or similar technical role.
• Relevant experience within AI systems, automation, or technology-driven environments is essential.

Skills:

Technical Proficiency:

• Familiarity with AI agent platforms and a foundational understanding of Large Language Models (LLMs).
• Experience working with CRM systems, chatbot platforms, and email management systems.
• Familiarity with reporting tools such as Metabase will be highly advantageous.

Analytical & QA Mindset:

• Exceptional attention to detail and the ability to identify subtle inaccuracies in tone, workflows, or technical documentation.

• Strong understanding of system behaviour and the downstream impact of system changes.

• Ability to translate complex workflows into clear quality standards and testing processes.

Communication Skills:

• Strong verbal and written communication skills in English.
• Ability to provide constructive and professional technical feedback.
• Additional South African languages will be advantageous.

Adaptability & Work Ethic:

• Proactive and self-motivated learner who can adapt to rapidly evolving AI technologies and product updates.
• Strong organisational and problem-solving abilities.
• Ability to work independently while collaborating effectively within a team environment.

Desired Skills:

• CRM
• Chatbot platforms
• Metabase
• LLM's
• Quality Assurance
• AI
• AI interactions

Desired Work Experience:

• 2 to 5 years