【摘要】目的：自动远程评估和监测患者的神经和心理健康正日益成为数字诊所和远程医疗生态系统的重要组成部分，尤其是在COVID-19大流行之后。这篇评论文章回顾了各种健康信息模式，这些模式可用于在现实世界中大规模开发此类远程临床评估。方法：我们首先概述了各种健康信息模式——语音声学、自然语言、对话动态、口面或全身运动、眼神注视、呼吸、心肺和神经——它们都可以从各种信号源（音频、视频、文本或传感器）中提取。我们进一步通过示例来激发它们的临床效用，说明每种模式的信息如何帮助我们描述不同疾病如何影响患者口头交流的不同方面。然后，我们阐明了将一种或多种这些模式结合起来以实现更全面、信息丰富和稳健的评估的优势。发现：我们发现，结合多种健康信息模式可以提高科学可解释性，提高下游健康应用（如早期检测和进度监测）的性能，提高技术稳健性，并改善用户体验。我们使用 Modality 评估平台的真实案例研究来说明如何利用这些原则进行大规模远程临床评估。结论：这篇评论文章鼓励结合多种模式的以人为本的信息来衡量患者健康的各个方面，并认为整合这些互补信息的远程临床评估比单独使用任何一种数据流更有效，并能带来更好的临床结果。

    [Abstract] Purpose: Automated remote assessment and monitoring of patients' neurological and mental health is increasingly becoming an essential component of the digital clinic and telehealth ecosystem, especially after the COVID-19 pandemic. This review article reviews various modalities of health information that are useful for developing such remote clinical assessments in the real world at scale. Approach: We first present an overview of the various modalities of health information—speech acoustics, natural language, conversational dynamics, orofacial or full body movement, eye gaze, respiration, cardiopulmonary, and neural—which can each be extracted from various signal sources—audio, video, text, or sensors. We further motivate their clinical utility with examples of how information from each modality can help us characterize how different disorders affect different aspects of patients' spoken communication. We then elucidate the advantages of combining one or more of these modalities toward a more holistic, informative, and robust assessment. Findings: We find that combining multiple modalities of health information allows for improved scientific interpretability, improved performance on downstream health applications such as early detection and progress monitoring, improved technological robustness, and improved user experience. We illustrate how these principles can be leveraged for remote clinical assessment at scale using a real-world case study of the Modality assessment platform. Conclusion: This review article motivates the combination of human-centric information from multiple modalities to measure various aspects of patients' health, arguing that remote clinical assessment that integrates this complementary information can be more effective and lead to better clinical outcomes than using any one data stream in isolation.

    论文原文：Vikram Ramanarayanan (2024). Multimodal Technologies for Remote Assessment of Neurological and Mental Health. Journal of Speech, Language, and Hearing Research, Volume 67, Number 11, Pages 4233-4245. November，2024.

https://doi.org/10.1044/2024_JSLHR-24-00142