Query Vetting Specialist – AI/LLM Evaluation (Japanese/German/Portuguese)

Hello Everyone,

Greetings from the Datamundi team!!

We are looking for highly motivated and detail-oriented Query Vetting Specialists with expertise in Japanese , German or Portuguese language evaluation and a strong understanding of Generative AI and Large Language Models (LLMs).

Please find the details below.

Job Type: Contract (Short-Term / Long-Term based on performance and project requirements)
Work Mode: Remote (WFH)
Location Requirement: Candidates must be located in the United States
Domain: IT / Generative AI
Working Hours: US Shift – 35 Hours per Week

In this role, you will support AI research and evaluation initiatives by designing and executing data collection, annotation, fact-checking, and query evaluation workflows. You will work closely with AI researchers and engineers to improve the quality, safety, and performance of LLM systems.

This opportunity is ideal for candidates with strong linguistic skills, research capabilities, analytical thinking, and an interest in cutting-edge AI technologies.

Key Responsibilities

Vet, review, and evaluate user queries and AI-generated responses for quality, relevance, and factual accuracy

Collaborate with GenAI researchers and engineers to understand project requirements and evaluation goals

Expand high-level requirements into detailed evaluation workflows and documentation

Conduct deep research using LLM tools and reliable external sources

Perform fact-checking and accuracy verification to minimize misinformation risks

Assess curated and AI-generated content for linguistic quality, cultural appropriateness, and informativeness

Create and maintain annotation guidelines and quality standards

Identify prompt quality issues and recommend improvements for better LLM performance

Ensure high throughput while maintaining exceptional quality standards

Collaborate with cross-functional teams in a fast-paced remote environment

Required Qualifications

Language Proficiency

Candidates must demonstrate Native, Bilingual, or Full Professional Proficiency in at least one of the following:

Japanese (Native of Japan)

German (Native of Germany)

Portuguese – (Native of Brazil)

Educational Background

Graduate with strong academic performance and deep cultural understanding

Degree from a reputed US or international university/institute preferred

Experience

Minimum 2+ years of professional experience in Japanese or German language-related roles

Prior experience in AI/LLM evaluation, content moderation, annotation, localization, linguistic QA, or research preferred

Understanding of AI, NLP, and Large Language Models is mandatory

Preferred Skills

Excellent reading, writing, grammar, and communication skills

Strong research and analytical capabilities

Exceptional attention to detail and accuracy

Familiarity with AI tools and prompt evaluation

Understanding of ethical AI practices and bias mitigation

Data literacy and experience handling structured/unstructured datasets

Ability to work independently with minimal supervision

Interview Process

Resume Screening

Assessment Test

Take-Home Assignment / Interview

Managerial Round

Why Join Datamundi?

Work on cutting-edge Generative AI projects

Collaborate with global AI researchers and engineers

Flexible remote work environment

Opportunity for long-term engagement based on performance

Exposure to advanced LLM evaluation and AI quality workflows

Ideal Candidate

We are looking for young, dynamic professionals who have demonstrated strong academic excellence, exceptional linguistic expertise, and the ability to thrive in a fast-paced AI-driven environment with minimal supervision.