Before buying an AI tool for your training: the 5 questions nobody asks

Demos of AI tools for training are always convincing. The interface is clean, the response arrives quickly, and the generated content seems coherent. Most vendors have refined their demo journeys to the point where it is hard to see what does not work.

What a demo does not show: how your learners' data flows. Whether the tool genuinely helps them learn or simply helps them move faster. What happens during assessments. How the tool behaves when something falls outside the expected scenario.

Here are five questions to ask before signing. Not to catch out the salesperson, but to give yourself the basis for an informed decision.

Question 1 - Where does your learners' data go?

This is the most important question, and the least asked when purchasing AI tools.

Every message sent to an AI assistant integrated into your LMS goes somewhere. That somewhere may be a US cloud service, a generalist LLM with no clearly defined data-retention policy, or infrastructure whose exact location you do not know. Your data protection officer does know, or should.

The practical questions to ask:

  • Is the learner's identity pseudonymised before any data is sent to an external service?
  • Where is the processing infrastructure hosted? In France? In Europe? With which cloud provider?
  • How long is processed data retained, and can your institution administer that period?
  • How is a learner's request to delete their data handled?

Since 2 August 2026, the AI Act has been fully applicable. For limited-risk AI tools, which includes the great majority of training assistants, the transparency and traceability requirements are clear. A provider unable to answer these questions precisely cannot guarantee your compliance.

Question 2 - Is the tool designed to help people learn, or to reduce friction?

This distinction is fundamental, and a demo does not reveal it.

A tool designed to reduce friction responds quickly, avoids frustration and moves the learner through the course without resistance. In the short term, learner feedback is positive: the experience is smooth and comfortable. In the medium term, the real skills are not there.

Scientific evidence published in 2025 measures this gap. A study in the Proceedings of the National Academy of Sciences (Bastani et al., 2025) followed nearly a thousand secondary-school students using different versions of an AI mathematics assistant.

During exercises, with AI available, results jump: +48% for the standard-assistant group and +127% for the group using an assistant with progressive levels of help. Once access was removed and the exam taken alone, the first group scored 17% lower than those who had never had AI, while this penalty largely disappeared in the second group.

The guardrails therefore did not produce a better final result. They prevented it from deteriorating. That is already considerable, and it is what you should seek to verify in a product.

This is not a question of technology. It is a question of design. A tool that supports learning requires cognitive effort from the learner. A tool that reduces friction relieves them of it.

Practical questions to ask: Does the tool ask learners questions before giving them an answer? Are its levels of help configurable, or does the assistant always give the complete answer? What pedagogical basis informed the tutor's behaviour?

Question 3 - Do your instructors remain in the loop?

An AI tool integrated into your LMS will influence the content your learners receive, the explanations they get and the activities they are offered. The question is not "Can the instructor check everything?" No one can check everything. The question is whether they have the means to control what matters.

The controls that matter: Approval before insertion. With content-generation tools, nothing should be added to your LMS without an instructor's explicit confirmation. No automatic insertion, no silent publishing. Configuration per activity. AI behaviour must be able to vary by context: more guiding in a discovery exercise, more restrictive in assessment. Traceability. The instructor must be able to see what AI proposed, what was approved and what was rejected.

A tool that automates without approval, publishes without confirmation and operates out of the instructor's view is not an educational tool. It is a content-production tool with direct access to your course.

Question 4 - How does the tool behave during assessments?

This is the question nobody asks. Yet it is the most revealing.

If a learner can use the AI tutor while completing an assignment or taking a quiz, the grade does not measure their skills: it measures their ability to ask AI good questions. This is not necessarily a problem in every context. But if your programme aims to validate a genuine skill, it is a serious problem.

Findings from Liu and co-authors (2026), based on three randomised trials with 1,222 participants, add a further dimension: even a brief interaction with a tool that provides direct solutions reduces persistence when facing difficulty. These effects appear after 10 to 15 minutes of use. An assistant available during assessments does not merely enable occasional "cheating": it changes how learners approach difficult problems.

Questions to ask the provider: In which spaces is the assistant available? In assignments? In quizzes? Is this behaviour configurable or fixed? Can the instructor disable the assistant for specific activities?

Question 5 - Does the tool integrate with your existing LMS, or does it ask you to change it?

Some vendors make this promise: "Our AI is so powerful that you should migrate to our platform." The cost of that migration is rarely presented honestly.

Migrating an LMS means migrating content, users, learning pathways, learning histories, SSO integrations and configuration settings accumulated over years. For an institution that has invested in Moodle, the real cost of moving to a proprietary platform is not a monthly subscription: it is an information-system transformation.

The alternative is an AI tool that integrates natively into your existing LMS, without major reconfiguration, migration or loss of learning continuity.

Questions to ask: Is the tool native to your LMS or does it connect as an overlay (LTI, iframe and so on)? How deep is the integration: does it read course context and Moodle permissions, or ignore them? If the service stops or pricing changes, does your data remain accessible in your LMS?

How PimenkoAI answers these five questions

Let us be clear: we have an interest in presenting PimenkoAI favourably. But these five questions are useful regardless of our answers, and we encourage you to ask them of every tool you evaluate.

On data: learner identity is never transmitted in raw form to the processing service. It passes through a systematic anonymisation mechanism. Our application infrastructure is hosted by Scaleway, a French provider, and inference uses the API of Mistral, also a French company. Neither is subject to US law. The institution can set conversation retention from 30 to 365 days, with automatic deletion and compliance with Moodle Privacy requests.

On learning design: PimenkoAI's Tutor mode is designed with progressive levels of help (N0 to N3), a questioning system before answers are given, and four explicit sub-modes (understand, progress, revise, retrieve). When a learner asks for a direct answer to an assignment, the conversation is redirected.

On instructor control: no generated activity is added to Moodle without explicit confirmation. Creation mode proposes; the instructor approves.

On assessments: Tutor mode is hidden by default in Quiz, Assignment and Workshop activities. This is not an option to turn on: it is the product's default behaviour.

On integration: PimenkoAI relies exclusively on Moodle's native AI API (core_ai). It reads Moodle permissions, respects roles and requires no migration. It is a Moodle plugin, not a competing LMS.

See how PimenkoAI answers these questions →

References