Human Motion Analysis With Radar

Author : Mohammad Mahbubur Rahman
Publisher :
ISBN 13 :
Total Pages : 0 pages
Book Rating : 4.:/5 (139 download)

DOWNLOAD NOW!

Download or read book Physics-Aware Deep Learning for Radar-Based Cyber Physical Human Systems written by Mohammad Mahbubur Rahman and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Radar, an acronym for Radio Detection and Ranging, is an emerging sensing modality utilized in Cyber-Physical Systems (CPS), that blends physical processes with computational and communication components. Due to its ability to sense physical objects in challenging environments (like fog, rain, and darkness), cost-effectiveness, and privacy-preserving nature, radar has found applications in health and safety CPS, CPS for gesture/sign language-driven smart environments, and automotive CPS. Health and safety CPS relies on the radar for fall detection, gait analysis, vital sign detection, and human activity recognition. Gesture and sign-language-driven smart environments utilize radar to interact with assistive devices and household appliances through gesture and sign-language recognition. Automotive CPS utilizes radar for in-cabin sensing and external sensing, including pedestrian detection, self-driving car vs police communication, and off-road traffic gesture detection. In radar-enabled CPS, the efficacy of each sensing task hinges on the radar's perception of the human test subject. This can be expensive and often results in a scarcity of data. Additionally, multiple RF sensors with different bandwidths, frequencies, and waveforms are required for CPS to capture the variety of motions performed in a wider physical space. Diverse human motions, such as fine-grain or gross body movements, exhibit varying degrees of sensitivity to distinct frequency sensors. As a result, achieving interoperability among multiple-frequency sensors is crucial for delivering a seamless CPS experience. Furthermore, when implementing CPS for remote health monitoring, it is frequently necessary to comprehend the intricacies of human movement. For example, assessing gait abnormalities and the risk of falls necessitates the estimation of human gait parameters and an understanding of human posture. As a result, a critical sensing challenge is how to extract valuable insights from the complex and high-dimensional information contained in RF measurements, including range-Doppler, range-azimuth, and range-elevation heatmaps. These sensing challenges lead to various deep learning challenges, like training under insufficient support samples, sensors/ aspect angles invariant feature learning for RF sensors interoperability, open-set problems for recognizing diverse human motions without prior training data, and decoupling individual motions from a multi-person scenario. Conventional data-driven deep learning approaches require vast amounts of training data for each scenario, but gathering large volumes of RF data is a major challenge in itself. To address these issues, this dissertation proposes physics-aware solutions that leverage physical prior knowledge and operate effectively with limited measured data. To address the data insufficiency, this dissertation designed a novel Physics-aware Generative Adversarial Network (PhGAN), which synthesizes a large volume of kinematically accurate radar micro-Doppler data using a few measured radar data and knowledge of human motion kinematics. As the envelopes of micro-Doppler constrain the maximum velocity incurred during motion and capture the differences between human gaits, it is essential that the process for generating synthetic samples consistently and realistically replicates the envelopes of the motion classes. Thus, the proposed method precludes gross kinematic errors in synthetic samples by supplying the signature envelopes as inputs to additional branches in the discriminator and utilizing the envelope distance between the real and synthetic micro-Doppler as an additional physics-based loss term in the discriminator loss function. Radar-enabled CPS requires multiple RF sensors with different operating characteristics to capture a wide range of motions in a larger physical space. However, the differences in the time-frequency representation of acquired data from different RF sensors lead to poor classification performance. This thesis explores the question of how to achieve interoperability between multi-frequency RF sensors to mitigate training data deficiency. Two approaches namely, domain adaptation, and cross-modal fusion have been proposed to address this problem. The domain adaptation approach uses adversarial image-to-image translation techniques to adapt micro-Doppler signatures from one frequency sensor to another. The adapted signatures are then used to train the DNNs while real signatures are used during the inference phase. In cross-modal fusion, different RF sensors are treated as different modalities, and a proposed framework jointly exploits these disparate RF sensor data to improve target recognition. The efficacy of these solutions is demonstrated through examples of the gross body human activity, such as ambulatory human motion recognition for health and safety CPS applications, and fine-grained human motions, such as American Sign Language (ASL) recognition for smart deaf space design. Notably, this dissertation is the first to conduct a study on RF-based ASL recognition, achieving state-of-the-art classification performance for 100-word fluent ASL recognition. Finally, this dissertation proposes an RF-based human pose estimation framework that recognizes diverse human motions and facilitates gait analysis and fall risk assessments in elderly care and nursing homes. The accuracy of this framework is validated by gold standard Vicon motion capture measurements. In conclusion, this dissertation provides innovative and practical solutions to the challenges of radar-enabled CPS, leveraging physics-aware techniques to enable effective learning with limited data and advance the field toward more accurate and reliable sensing.