Model Validation & Red Teaming

Overview

We follow specific processes to ensure that our model always responds using company knowledge and tools while adhering to all permissions and authentication protocols applied at the company’s end.

Grounding Mechanism

Leena AI employs a grounding mechanism that serves as a fundamental safeguard for the model's behavior. This mechanism functions as follows:

• Fetches Relevant Data: Retrieves relevant text from a knowledge base or relevant company systems/tools.
• Mitigates Injection Risks: Mitigates the risk of injection attacks by ensuring the user query is used as an input to the prompt rather than being treated as the prompt itself.
• Ensures Grounded Responses: Guarantees that responses are based strictly on grounded information.
• Fail-Safe Approach: Implements a strict protocol where, if grounded information is not present, the system does not provide an answer.

Security Testing

While the grounding mechanism provides a strong foundation for security, Leena AI has taken additional steps to ensure response integrity:

• Fact Check / Validation Models: These models check the LLM response against grounded information or tools to ensure that the LLM response is derived only from that grounded information.
• Enforcement: If the response is not fully supported by the grounded data, the system does not provide an answer.

This proactive approach helps identify and address potential vulnerabilities in the system.

Authentication and Information Access Control

To maintain data security and prevent unauthorized access to sensitive information, Leena AI has implemented the following measures:

• User Authentication: Authentication is required before passing any grounded information to the LLMs.
• Access Control Measures: We ensure that only information accessible to the specifically authenticated user is passed to the LLM's context.

This process prevents the inclusion of unauthorised or sensitive information in the model's responses.