{"id":731,"date":"2025-07-01T10:53:10","date_gmt":"2025-07-01T08:53:10","guid":{"rendered":"https:\/\/luminous-horizon.eu\/?page_id=731"},"modified":"2025-07-02T14:50:53","modified_gmt":"2025-07-02T12:50:53","slug":"language-meets-rehabilitation-the-luminous-metacognition-chatbot-prototype","status":"publish","type":"page","link":"https:\/\/luminous-horizon.eu\/index.php\/blogs\/language-meets-rehabilitation-the-luminous-metacognition-chatbot-prototype\/","title":{"rendered":"Language Meets Rehabilitation: The LUMINOUS Metacognition Chatbot Prototype\u00a0"},"content":{"rendered":"\n<p class=\"has-x-large-font-size\">Language Meets Rehabilitation: The LUMINOUS Metacognition Chatbot Prototype&nbsp;<\/p>\n\n\n\n<p>As part of the LUMINOUS project\u2019s mission to create <strong>language-augmented XR systems<\/strong> that adapt intelligently to real-world complexity, we have developed a <strong>Metacognition Chatbot prototype<\/strong> \u2014an AI assistant designed to support <strong>neurorehabilitation and psychoeducation<\/strong> through natural, context-aware conversation.&nbsp;<\/p>\n\n\n\n<p>The chatbot is being integrated into the <strong>MindFocus<\/strong> virtual reality (VR) platform, which offers immersive cognitive rehabilitation experiences. This novel assistant helps patients and clinicians access <strong>personalized, accurate, and clinically grounded information<\/strong>, based on curated documents and user queries, in both English and French.&nbsp;<\/p>\n\n\n\n<p class=\"has-large-font-size\"><strong><span style=\"text-decoration: underline;\">Data-Driven Intelligence: The Metacognition Chatbot Dataset<\/span><\/strong><\/p>\n\n\n\n<p>To power the chatbot\u2019s contextual understanding, the team created a specialized <strong>Retrieval-Augmented Generation (RAG)<\/strong> dataset including:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Neurorehabilitation domain knowledge<\/strong>: Articles, books, infographics, and slides provided by CHUV, covering therapeutic principles and rehabilitation strategies.&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>MindFocus game manual<\/strong>: A comprehensive 70-page guide to the MindFocus software, enabling the chatbot to explain gameplay and functionality clearly.&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Expert-designed questions<\/strong>: Real-world queries from clinical professionals to evaluate system responses.&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>This multilingual dataset ensures that the chatbot responds accurately to queries about rehabilitation practices and system use.&nbsp;<\/p>\n\n\n\n<p class=\"has-large-font-size\"><span style=\"text-decoration: underline;\"><strong>How It Works: Knowledge Injection in Practice<\/strong>\u00a0<\/span><\/p>\n\n\n\n<p>The system uses a <strong>RAG pipeline<\/strong> built with modern LLM components:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Document embedding<\/strong>: PDF page chunks and user queries are embedded using BAAI\/bgm-m3.&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Retrieval<\/strong>: The top-matching content is retrieved dynamically.&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Response generation<\/strong>: Gemini-2.0-Flash generates fluent, grounded replies based on both the query and retrieved content.&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>This setup ensures that the chatbot\u2019s answers are <strong>not pre-scripted<\/strong>, but <strong>emerge from the model\u2019s reasoning<\/strong> over relevant, domain-specific information.&nbsp;<\/p>\n\n\n\n<p class=\"has-large-font-size\"><strong><span style=\"text-decoration: underline;\">Evaluation &amp; Insights<\/span><\/strong><\/p>\n\n\n\n<p>Performance is being assessed through both <strong>manual<\/strong> and <strong>automated<\/strong> evaluation:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Clinical experts assess responses using targeted questions.&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A <strong>RAG Triad<\/strong> evaluation pipeline, powered by an LLM, automatically scores:&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Context relevance<\/em>&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Groundedness<\/em> (i.e., how well the response is supported by retrieved data)&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Answer relevance<\/em>&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>This multi-layered strategy enables continuous improvement, even in the absence of standard ground-truth datasets\u2014an essential step for real-world deployments.&nbsp;<\/p>\n\n\n\n<p class=\"has-large-font-size\"><span style=\"text-decoration: underline;\"><strong>Strategic Alignment with LUMINOUS<\/strong><\/span><\/p>\n\n\n\n<p>The Metacognition Chatbot prototype embodies key LUMINOUS goals:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Multilingual, multimodal intelligence<\/strong>: It handles complex neurorehabilitation queries in both English and French, from textual prompts to VR-integrated usage.&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Generalization beyond training data<\/strong>: Rather than relying solely on prior training, the chatbot dynamically incorporates new factual information using the RAG framework.&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Support for personalized interaction<\/strong>: By responding to the user&#8217;s context and needs, the system supports psychoeducation and informed participation in therapy.&nbsp;<\/li>\n<\/ul>\n\n\n\n<p class=\"has-large-font-size\"><strong><span style=\"text-decoration: underline;\">Clinical Integration and Future Steps<\/span><\/strong><\/p>\n\n\n\n<p>The Metacognition Chatbot is being tested by neuropsychologists and speech therapists, providing valuable feedback. It is currently being embedded into <strong>pre- and post-therapy<\/strong> workflows in the MindFocus VR platform.&nbsp;&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"414\" src=\"https:\/\/luminous-horizon.eu\/wp-content\/uploads\/2025\/07\/image-10-1024x414.png\" alt=\"\" class=\"wp-image-732\" srcset=\"https:\/\/luminous-horizon.eu\/wp-content\/uploads\/2025\/07\/image-10-1024x414.png 1024w, https:\/\/luminous-horizon.eu\/wp-content\/uploads\/2025\/07\/image-10-300x121.png 300w, https:\/\/luminous-horizon.eu\/wp-content\/uploads\/2025\/07\/image-10-768x310.png 768w, https:\/\/luminous-horizon.eu\/wp-content\/uploads\/2025\/07\/image-10.png 1299w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Further work includes:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Testing with patients and real-time therapy sessions&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Expanding the evaluation dataset and refining grounding techniques&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Integrating general knowledge beyond domain-specific documents&nbsp;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Developing a more \u201creasoned\u201d version of the chatbot for deeper clinical interaction&nbsp;<\/li>\n<\/ul>\n\n\n\n<p class=\"has-large-font-size\"><strong><span style=\"text-decoration: underline;\">A Milestone in Human-Centered XR Intelligence<\/span><\/strong><\/p>\n\n\n\n<p>By enabling <strong>conversational access to domain-specific knowledge<\/strong> in dynamic XR environments, the Metacognition Chatbot marks an important step toward <strong>human-centered AI for healthcare<\/strong>. As LUMINOUS continues to push the boundaries of what is possible at the intersection of language, cognition, and extended reality, this prototype stands as a powerful demonstration of the project\u2019s potential.&nbsp;<\/p>\n\n\n\n<p>Stay tuned for upcoming results from clinical pilots and continued development of this promising tool.&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Language Meets Rehabilitation: The LUMINOUS Metacognition Chatbot Prototype&nbsp; As part of the LUMINOUS project\u2019s mission to create language-augmented XR systems that adapt intelligently to real-world complexity, we have developed a Metacognition Chatbot prototype \u2014an AI assistant designed to support neurorehabilitation and psychoeducation through natural, context-aware conversation.&nbsp; The chatbot is being integrated into the MindFocus virtual [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":696,"menu_order":5,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-731","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/pages\/731","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/comments?post=731"}],"version-history":[{"count":3,"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/pages\/731\/revisions"}],"predecessor-version":[{"id":866,"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/pages\/731\/revisions\/866"}],"up":[{"embeddable":true,"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/pages\/696"}],"wp:attachment":[{"href":"https:\/\/luminous-horizon.eu\/index.php\/wp-json\/wp\/v2\/media?parent=731"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}