LLM Evaluation Engineer

We are committed to improving patient outcomes and advancing industry standards through innovative AI-powered solutions. Join our team and help shape the future of biopharma technology.

About us:

• We are a team of experts creating software for managing different phases of clinical trials.
• We create end-to-end products that enable our clients to discover new treatments and therapies.
• Our biggest strength is our team of talented, passionate and devoted people.
• We are driven by enthusiasm for working on cutting edge solutions and problems.
• Reference: p_OOyVswSHiBbyelY0XS83fgBW-EXoVzMldknEALvG4.

Here's what you will be doing:

• Design, develop, and maintain automated test frameworks for LLM and AI-powered applications, primarily using Python.
• Collaborate with DevOps, AI engineers, and developers to understand model requirements, data pipelines, and application workflows.
• Develop and execute test plans and test cases for GenAI features, APIs, and cloud-based services.
• Implement automated testing for model evaluation, data validation, and API endpoints using modern test automation tools (e.g., Pytest, Robot Framework).
• Integrate automated tests into CI/CD pipelines in GitLab.
• Monitor, analyze, and report on test results, identifying root causes of failures and ensuring high quality releases.
• Ensure security, scalability, and performance testing of AI models and cloud applications.
• Contribute to best practices in test automation, code quality, and DevOps within the GenAI team.

About you: 

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in software development and test automation with Python.
• Strong understanding of cloud-based services, with experience in testing GenAI features and APIs.
• Experience with CI/CD pipelines (GitLab CI/CD experience is a plus).
• Strong problem-solving, communication, and collaboration skills.
• Experience in performance, scalability, and security testing of cloud applications.

Nice to have:

• Experience leading test strategy design and mentoring team members.
• Familiarity with DevOps practices and a commitment to continuous improvement in testing strategies.
• Ability to stay current with industry trends and advancements in AI and cloud technologies.
• Interest in staying current with AI/cloud advancements and bringing new ideas into practice.

Company benefits:

• Fully flexible working hours
• 100% remote working environment
• Private health insurance (for domestic citizens only)
• 20 PTO days + 2 Additional well-being days
• Home office budget + work equipment (for domestic citizens only)

If you're excited by the challenge of developing cutting-edge software and working with a team that explores new technological frontiers, join us in our journey.

By submitting your application, you allow 4Create to handle and store your data.