With the rapid digitization of healthcare, there are many applications for health management, monitoring, and decision support. These applications generate and collect multimodal data. While such heterogeneous data streams can enable intelligent decision support for patients, they can also be overwhelming for patients to process and analyze. This project aims to develop data-driven techniques to fuse multimodal health data to generate actionable information that can improve patient safety and awareness. Specifically, our goal is to develop novel natural language processing, time-series modeling, and machine learning techniques to enable intelligent health assistance. It requires deep semantic inference and alignment of heterogeneous data streams that vary in modality, format, and content. These tasks are particularly challenging as this domain is low-resource due to scarce training data and expensive annotation process.