# Safe Futures: Personalized Safety in Multimodal AI Systems

> Recent advancements in artificial intelligence (AI) have led to systems that can remember personal details and exhibit empathetic behavior, resulting in more engaging user experiences. However, as the...

**Source**: arxiv.org | **Published**: 2026-09-28 | **Type**: research

## Key Facts

- Implement personalized trajectory-level safety to enhance user interaction monitoring.
- Develop AI systems that prioritize cumulative user experience over isolated interactions.
- Utilize empathetic AI behavior to foster deeper user engagement and satisfaction.
- Conduct regular assessments of user-AI relationships to identify and mitigate risks.
- Train staff on new AI safety protocols to ensure ongoing compliance and user protection.

## Summary

**Paper:** [Sampling Safe Futures: Multimodal Trajectory Planning for Personalized Safety in Anthropomorphic AI](https://arxiv.org/abs/2609.30780)

**Authors:** Benedetta Picano, Dusit Niyato

\## Executive Summary

Recent advancements in artificial intelligence (AI) have led to systems that can remember personal details and exhibit empathetic behavior, resulting in more engaging user experiences. However, as these AI systems evolve into social counterparts, they pose new risks. Current safety measures mainly focus on individual interactions, lacking the ability to assess the cumulative effects of these exchanges over time. This limitation can lead to situations where users unknowingly move towards harmful interactions.

This research introduces a new approach called personalized trajectory-level safety, which aims to address these risks by treating the safety of interactions as an ongoing process rather than a series of isolated events. The framework analyzes the relationship between the user and the AI over time, identifying potential escalation points based on the user's messages and the system's responses.

At each interaction, the proposed method employs a screening step to eliminate any response strategies that do not ensure at least one safe continuation of the conversation, taking into account various potential user behaviors. This step is crucial as it enables the system to maintain a focus on user safety throughout the interaction.

The framework utilizes a Generative Flow Network to sample diverse potential future interactions, weighing their likelihood, safety, and usefulness. From these options, the system selects the strategy that maximizes safe and beneficial continuations of the dialogue. The evaluation of this framework was conducted through simulations, which were calibrated using statistics from real human-chatbot interactions. Additionally, response strategies were derived from existing public benchmarks.

The results from these simulations indicate that this trajectory-aware decision-making process significantly decreases the occurrence of harmful interactions while maintaining helpful exchanges. This represents a shift in how safety is conceptualized for anthropomorphic AI, moving from a reactive approach focused on individual responses to a proactive strategy that fosters safer long-term human-AI relationships.

This research could have implications for various sectors where AI is integrated as a conversational partner, such as customer service, mental health support, and education. By adopting a trajectory-level safety approach, organizations may enhance the safety and effectiveness of AI interactions, potentially leading to improved user trust and satisfaction. The source code for this framework is publicly available, allowing for further exploration and adaptation by interested parties.

\## Academic Abstract

Anthropomorphic artificial intelligence systems increasingly remember personal details, display empathy, and are engaged with as social counterparts, creating forms of risk that emerge from the evolution of the user-system relationship over time. Existing safeguards largely operate at the level of individual conversational turns and cannot determine whether a sequence of seemingly acceptable interactions is cumulatively moving a particular user toward harm. This paper introduces personalized trajectory-level safety, a framework that treats relational safety as a sequential decision problem over a latent escalation state inferred from the user's messages and influenced by the system's responses. At each turn, a screening step first discards any response strategy that does not preserve at least one safe continuation of the interaction under every plausible model of the user. Among the remaining strategies, we formulate action selection as multimodal trajectory sampling, and use a Generative Flow Network to generate diverse future evolutions in proportion to their plausibility, safety, and utility. The system then selects the strategy that preserves the largest fraction of safe and useful continuations. We evaluate the framework in simulation, calibrated on statistics reported for real human-chatbot interactions, and using response strategies derived from public benchmarks. Results show that trajectory-aware decision making substantially reduces the frequency of harmful states while keeping helpful interaction. This work reframes safety for anthropomorphic AI from response-level filtering to personalized control over the future evolution of human-AI relationships. The source code is available at https://github.com/benedettapicano/ANTHROPOMORPHIC_SAFETY_TRAJ.

## Frequently Asked Questions

**What business problems does this research solve?**

This research addresses the potential risks associated with AI systems that evolve into social counterparts, specifically the cumulative effects of user interactions that may lead to harmful outcomes. By introducing personalized trajectory-level safety, it helps ensure safer interactions over time.

**Which industries benefit most from this research?**

Industries that rely heavily on customer interaction through AI, such as customer service, healthcare, and education, could benefit most. These sectors could utilize the framework to enhance user safety and improve the quality of engagements through empathetic AI.

**What are the practical implementation considerations for businesses?**

Businesses would need to consider how to integrate the personalized trajectory-level safety framework into their existing AI systems. This could involve adjusting current safety measures to focus on ongoing user interactions rather than isolated instances, which may require significant changes to system architecture.

**What resources/expertise are needed to implement this research in a business context?**

Implementing this approach may require expertise in AI development, particularly in natural language processing and user experience design. Additionally, businesses may need resources for ongoing data analysis to monitor user interactions and system responses over time.

**What are the competitive advantages of utilizing this research?**

By adopting personalized trajectory-level safety, businesses could differentiate themselves by offering safer and more engaging AI interactions. This could lead to improved customer trust and satisfaction, potentially resulting in higher retention rates and a stronger brand reputation.

## Links

- [Read on Welcome.AI](https://welcome.ai/content/safe-futures-personalized-safety-in-multimodal-ai-systems)
- [Original source](https://arxiv.org/abs/2609.30780)

---

Source: Welcome.AI | https://welcome.ai/content/safe-futures-personalized-safety-in-multimodal-ai-systems