publications | Systems and Informatics Research Lab

2026

Journal AnxietyFaceTrack: A Smartphone-Based Non-Intrusive Approach for Detecting State Anxiety Using Facial Features

Nilesh Kumar Sahu, Snehil Gupta, and Haroon R Lone

IEEE Transactions on Affective Computing, 2026

Abs DOI

Social Anxiety Disorder (SAD) is a widespread mental health condition, yet its lack of objective markers hinders timely detection and intervention. While previous research has focused on behavioral and non-verbal markers of SAD in structured activities (e.g., speeches or interviews), these settings fail to replicate in real-world, unstructured social interactions. Identifying non-verbal markers in naturalistic, unstaged environments is essential for developing ubiquitous and non-intrusive monitoring solutions. To address this gap, we present AnxietyFaceTrack, a study leveraging facial video analysis to detect state anxiety in unstaged social settings. A cohort of 91 participants engaged in a social setting with unfamiliar individuals, and their facial videos were recorded using a low-cost smartphone camera. We examined facial features, including eye movements, head position, facial landmarks, and facial action units, and used self-reported survey data to establish ground truth for multiclass (anxious, neutral, non-anxious) and binary (e.g., anxious vs. neutral) classifications. Our results demonstrate that a Random Forest classifier trained on the top 20% of features achieved the highest accuracy of 91.0% for multiclass classification and an average accuracy of 92.33% across binary classifications. Notably, head position and facial landmarks yielded the best performance for individual facial regions, achieving 85.0% and 88.0% accuracy, respectively, in multiclass classification, and 89.66% and 91.0% accuracy, respectively, across binary classifications. Post-hoc analysis identified head rotation, facial edge features, and eye landmarks as key contributors to the detection of anxiety. This study introduces a non-intrusive, cost-effective solution that can be seamlessly integrated into everyday smartphones for continuous anxiety monitoring, offering a promising pathway for early detection and intervention.

2025

Journal MAD: A Multimodal Physiological and Self-Reported Dataset for Anxiety Research from a Low-to-Middle-Income Country

Nilesh Sahu, Snehil Gupta, and Haroon R. Lone

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2025

Abs DOI

Wearable sensors provide a promising approach for monitoring physiological parameters, offering valuable insights into mental health conditions, including Social Anxiety Disorder (SAD). Early detection of SAD using physiological data can facilitate timely interventions, yet the development of robust anxiety detection models requires high-quality datasets. While real-world studies provide ecological validity, controlled studies ensure structured, high-quality data with minimal missing values, making them ideal for developing generalized and personalized anxiety detection models.; AB@Existing publicly available datasets related to anxiety research are limited to developed nations and focus on one or two anxiety-inducing activities. However, cultural differences significantly influence how anxiety is experienced and expressed, highlighting the need for datasets from diverse populations. This work presents MAD, a novel dataset collected in a low-to-middle-income country that addresses this gap. Our study involved participants engaging in three anxiety-inducing activities—speech, group discussion, and interview—each structured into three phases: anticipation, performance, and reflection. Physiological data were collected using wearable sensors, including electrocardiogram, electrodermal activity, and photoplethysmography, along with self-reported anxiety levels.; AB@Our dataset (N = 97) is unique in its inclusion of multiple anxiety-inducing activities, comprehensive phase-wise assessment, and representation of an underrepresented population. It provides a valuable resource for developing generalizable anxiety detection models, designing personalized interventions, and studying cultural variations in anxiety responses. By making MAD available, we aim to facilitate future research in machine learning-based mental health analysis, cross-cultural studies, and privacy-preserving anxiety detection approaches.
Journal Intact but vulnerable: Affective working memory in social anxiety disorder

Arya Adyasha and Haroon R. Lone

PLOS Mental Health, 2025

Abs DOI

Social anxiety disorder (SAD) is increasingly recognized as a disorder of cognitive-affective dysregulation, with growing evidence implicating affective working memory (AWM) as a core mechanism. This narrative review synthesizes findings from behavioral, electrophysiological, and neuroimaging studies to examine how socially and emotionally salient stimuli interfere with working memory (WM) processes in individuals with SAD. While general WM capacity often appears intact in neutral contexts, impairments emerge consistently when tasks involve socially threatening content or require executive control under cognitive load. Key disruptions include reduced flexibility, impaired updating, and sustained attentional capture by threat. These deficits are further moderated by individual symptom dimensions and task demands. The review critically evaluates current theoretical models and highlights methodological limitations in the field, proposing directions for future research. Clinical implications include the use of AWM-sensitive assessments and interventions that target executive regulation of emotion. AWM thus provides a mechanistic bridge linking cognition, emotion, and social dysfunction in SAD.
Conference An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation

Vimaleswar A, Prabhu Nandan Sahu, Nilesh Kumar Sahu, and 1 more author

In Proceedings of the International Joint Conference on Natural Language Processing and the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL), 2025

Abs DOI

Mental health plays a crucial role in the overall well-being of an individual. In recent years, digital platforms have increasingly been used to expand mental health and emotional support. However, there are persistent challenges related to limited user accessibility, internet connectivity, and data privacy, which highlight the need for an offline, smartphone-based solutions. To address these challenges, we propose EmoSApp (Emotional Support App): an entirely offline, smartphone-based conversational app designed to provide mental health and emotional support. EmoSApp leverages a language model, specifically the LLaMA-3.2-1B-Instruct, which is fine-tuned and quantized on a custom-curated “Knowledge Dataset” comprising 14,582 mental health QA pairs along with multi-turn conversational data, enabling robust domain expertise and fully on-device inference on resource-constrained smartphones. Through qualitative evaluation with students and mental health professionals, we demonstrate that EmoSApp has the ability to respond coherently and empathetically, provide relevant suggestions to user’s mental health problems, and maintain interactive dialogue. Additionally, quantitative evaluations on nine commonsense and reasoning benchmarks, along with two mental health specific datasets, demonstrate EmoSApp’s effectiveness in low-resource settings. By prioritizing on-device deployment and specialized domain-specific adaptation, EmoSApp serves as a blueprint for future innovations in portable, secure, and highly tailored AI-driven mental health support.
Conference Investigating the Generalizability of ECG Noise Detection Across Diverse Data Sources

Sharmad Kalpande, Nilesh Kumar Sahu, and Haroon R. Lone

In Proceedings of the 2025 ACM International Symposium on Wearable Computers (ISWC), 2025

Abs DOI

Electrocardiograms (ECGs) are vital for monitoring cardiac health, enabling the assessment of heart rate variability (HRV), detection of arrhythmias, and diagnosis of cardiovascular conditions. However, ECG signals recorded from wearable devices are frequently corrupted by noise artifacts, particularly those arising from motion and large muscle activity, which distort R-peaks and the QRS complex. These distortions hinder reliable HRV analysis and increase the risk of clinical misinterpretation. Existing studies on ECG noise detection typically evaluate performance on a single dataset, limiting insight into the generalizability of such methods across diverse sensors and recording conditions. In this work, we propose an HRV-based machine learning approach to detect noisy ECG segments and evaluate its generalizability using cross-dataset experiments on four datasets collected in both controlled and uncontrolled settings. Our method achieves over 90% average accuracy and an AUPRC exceeding 90%, even on previously unseen datasets—demonstrating robust performance across heterogeneous data sources. To support reproducibility and further research, we also release a curated and labeled ECG dataset annotated for noise artifacts.
Conference Exploring Heart Rate Variability and Heart Rate Dynamics Using Wearables Before, During, and After Speech Activity: Insights from a Controlled Study in a Low-Middle-Income Country

Nilesh Kumar Sahu, Snehil Gupta, and Haroon R. Lone

In Proceedings of the ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies, 2025

Abs DOI

Conventional methods for diagnosing Social Anxiety Disorder (SAD), such as clinical interviews and self-reported questionnaires, often face accessibility barriers and subjective biases, underscoring the need for objective physiological markers. This study investigates heart rate (HR) and heart rate variability (HRV) as potential indicators of SAD by analyzing cardiovascular responses to anxiety-inducing speech tasks across four distinct phases: baseline, anticipation, speech activity, and reflection.In a controlled laboratory setting, we analyzed data from 51 participants and found that HRV decreased and HR increased during the anticipation and speech activity phases compared to baseline, while the reflection phase showed a reversal, with HRV increasing and HR decreasing. Participants with SAD exhibited lower HRV, higher HR, and greater self-reported anxiety than non-SAD participants across all phases. These findings enhance our understanding of the physiological signatures of social anxiety and have implications for developing wearable-based monitoring systems for SAD detection and intervention. To support further research, we also release an HRV dataset capturing multi-phase anxiety responses, advancing physiological-based mental health assessment.
Journal Thermal vision: Pioneering non-invasive temperature tracking in congested spaces

Arijit Samal and Haroon R. Lone

Smart Health, 2025

Abs DOI

Non-invasive temperature monitoring of individuals plays a crucial role in identifying and isolating symptomatic individuals. Temperature monitoring becomes particularly vital in settings characterized by close human proximity, often referred to as dense settings. However, existing research on non-invasive temperature estimation using thermal cameras has predominantly focused on sparse settings. Unfortunately, the risk of disease transmission is significantly higher in dense settings like movie theaters or classrooms. Consequently, there is an urgent need to develop robust temperature estimation methods tailored explicitly for dense settings. Our study proposes a non-invasive temperature estimation system that combines a thermal camera with an edge device. Our system employs YOLO models for face detection and utilizes a regression framework for temperature estimation. We evaluated the system on a diverse dataset collected in dense and sparse settings. Our proposed face detection model achieves an impressive mAP score of over 94 in both in-dataset and cross-dataset evaluations. Furthermore, the regression framework demonstrates remarkable performance with a mean square error of 0.18 °C and an impressive score of 0.96. Our experiments’ results highlight the developed system’s effectiveness, positioning it as a promising solution for continuous temperature monitoring in real-world applications. With this paper, we release our dataset and programming code publicly.
Conference The Impact of Acute Social Stress on Working Memory Updating in Social Anxiety Disorder

Arya Adyasha, Anushka Shelke, and Haroon R. Lone

In Proceedings of the Annual Meeting of the Cognitive Science Society, 2025

Abs DOI

Social anxiety disorder (SAD) is characterized by cognitive biases that impair emotional information processing. This study examines how acute social stress affects working memory (WM) updating in socially anxious individuals using an emotional 2-back task. A 2x2 factorial design (N = 137) categorized participants into socially anxious (SA) and non-anxious (NSA) groups, further divided into stressed and control conditions. Acute stress was induced via speech anticipation, and subjective stress ratings were recorded. Mixed-effects modelling revealed independent negative effects of both social anxiety and acute stress on WM updating, including reduced accuracy, increased false alarms, lower discriminability (d-prime), and increased inverse efficiency score. Notably, disgusted facial expressions enhanced task efficiency under stress. Findings suggest stress-related cognitive deficits in social anxiety are additive rather than interactive, highlighting potential targets for intervention. This study contributes to understanding emotion-cognition interactions and extends research to understudied cultural contexts.
Conference Leveraging Language Models for Summarizing Mental State Examinations: A Comprehensive Evaluation and Dataset Release

Nilesh Kumar Sahu, Manjeet Yadav, Mudita Chaturvedi, and 2 more authors

In , 2025

Abs

Mental health disorders affect a significant portion of the global population, with diagnoses primarily conducted through Mental State Examinations (MSEs). MSEs serve as structured assessments to evaluate behavioral and cognitive functioning across various domains, aiding mental health professionals in diagnosis and treatment monitoring. However, in developing countries, access to mental health support is limited, leading to an overwhelming demand for mental health professionals. Resident doctors often conduct initial patient assessments and create summaries for senior doctors, but their availability is constrained, resulting in extended patient wait times. This study addresses the challenge of generating concise summaries from MSEs through the evaluation of various language models. Given the scarcity of relevant mental health conversation datasets, we developed a 12-item descriptive MSE questionnaire and collected responses from 405 participants, resulting in 9720 utterances covering diverse mental health aspects. Subsequently, we assessed the performance of five well-known pre-trained summarization models, both with and without fine-tuning, for summarizing MSEs. Our comprehensive evaluation, leveraging metrics such as ROUGE, SummaC, and human evaluation, demonstrates that language models can generate automated coherent MSE summaries for doctors. With this paper, we release our collected conversational dataset and trained models publicly for the mental health research community.
Preprint Navigating in the Dark: A Multimodal Framework and Dataset for Nighttime Traffic Sign Recognition

Aditya Mishra, Akshay Agarwal, and Haroon R Lone

2025
Preprint Real-World Receptivity to Adaptive Mental Health Interventions: Findings from an In-the-Wild Study

Nilesh Kumar Sahu, Aditya Sneh, Snehil Gupta, and 1 more author

2025
Preprint Fairness-Aware Few-Shot Learning for Audio-Visual Stress Detection

Anushka Sanjay Shelke, Aditya Sneh, Arya Adyasha, and 1 more author

2025
Preprint HCFSLN: Adaptive Hyperbolic Few-Shot Learning for Multimodal Anxiety Detection

Aditya Sneh, Nilesh Kumar Sahu, Anushka Sanjay Shelke, and 2 more authors

2025
Preprint RestAware: Non-Invasive Sleep Monitoring Using FMCW Radar and AI-Generated Summaries

Agniva Banerjee, Bhanu Partap Paregi, and Haroon R Lone

2025
Preprint Are Anxiety Detection Models Generalizable? A Cross-Activity and Cross-Population Study Using Wearables

Nilesh Kumar Sahu, Snehil Gupta, and Haroon R Lone

2025
Preprint Interdisciplinary Insights into Social Anxiety Disorder: Bridging Computer Science and Mental Health, a Goal towards Digital mHealth

Nilesh Kumar Sahu, Snehil Gupta, and Haroon R Lone

2025
Journal Wearable Sensor-based Multimodal Physiological Responses of Socially Anxious Individuals in Social Contexts on Zoom

Emma R Toner, Mark Rucker, Zhiyuan Wang, and 8 more authors

IEEE Transactions on Affective Computing, 2025

Abs DOI

Correctly identifying an individual’s social context from passively worn sensors holds promise for delivering just-in-time adaptive interventions (JITAIs) to treat social anxiety disorder. In this study, we present results using passively collected data from a within-subject experiment that assessed physiological response across different social contexts (i.e, alone vs. with others), social phases (i.e., pre- and post-interaction vs. during an interaction), social interaction sizes (i.e., dyadic vs. group interactions), and levels of social threat (i.e., implicit vs. explicit social evaluation). Participants in the study (N=46) reported moderate to severe social anxiety symptoms as assessed by the Social Interaction Anxiety Scale (≥34 out of 80). Univariate paired difference tests, multivariate random forest models, and follow-up cluster analyses were used to explore physiological response patterns across different social and non-social contexts. Our results suggest that social context is more reliably distinguishable than social phase, group size, or level of social threat, but that there is considerable variability in physiological response patterns even among these distinguishable contexts. Implications for real-world context detection and deployment of JITAIs are discussed.

2024

Journal Indian Traffic Sign Detection and Classification Through a Unified Framework

Rishabh Uikey, Haroon R Lone, and Akshay Agarwal

IEEE transactions on intelligent transportation systems, 2024

Abs DOI

Traffic sign boards are vital in facilitating smart transportation systems. More than 90% of accidents happen due to drivers’ inattentiveness over these boards. Hence, relaying traffic sign board information automatically to drivers becomes crucial to avoid such accidents. While numerous traffic sign detection and classification systems exist, it is important to note that these automated systems have not been adequately assessed within the context of Indian settings. The task of traffic sign detection and classification presents unique challenges in the Indian context due to the presence of multiple variations for a single action. This paper proposes a robust methodology for detecting and classifying traffic signboards simultaneously using deep learning. The method uses convolutional models like AlexNet, VGG-19, ResNet-50, and EfficientNet v2 as the network backbone with different loss functions for bounding box regression and classification. Along with the proposed method, we collected the Indian traffic signs and information boards dataset. The collected dataset consists of 4300+ raw images without any augmentation. We evaluated our method on the collected dataset and found the detection and classification accuracies as 85.5% and 98.5%, respectively. With this paper, we release the dataset publicly.
Conference Mapping Thermal Footprints: Occupancy Estimation and Localization in Diverse Indoor Settings with Thermal Arrays

Soumya R Sahoo and Haroon R Lone

In Proceedings of the 7th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS), 2024

Abs DOI

Estimating and locating occupants indoors is crucial for automating operations within buildings. However, current privacy-preserving occupancy estimation systems using thermal cameras have not been thoroughly evaluated in dense settings, such as classrooms or movie theaters, where occupants sit close to one another. Estimating occupancy in such settings presents challenges, as nearby occupants often lead to clusters of thermal signatures, thereby affecting the accuracy of the estimation. In response to these challenges, our work proposes Machine Learning (ML) and Deep Learning (DL) based methods for occupancy estimation and localization, both in dense and sparse settings. The ML method complements existing occupancy estimation techniques by incorporating new manual features, while the DL method performs automatic feature extraction, accurate occupancy estimation and localization. Remarkably, the results demonstrate a significant enhancement in occupancy estimation accuracy of up to 10%, achieving an impressive overall accuracy rate of 97%. Moreover, our evaluation on an edge device confirms the practicality and relevance of the proposed methods in real-world applications.
Conference Towards Safer Roads: Deep Learning for Rash Driving Detection using Smartphone Sensors Data

Durgesh Mishra, Manoj Gulati, and Haroon R Lone

In Proceedings of the 7th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS), 2024

Abs DOI

Rash driving detection is vital to prevent accidents and improve public safety. Existing rash driving solutions using hand-crafted features have several limitations. We propose a simple yet efficient two-step process to overcome the limitations of the existing works by leveraging smartphone sensor (accelerometer and gyroscope) data. The first step filters out normal driving data and retains only the abnormal driving data with the proposed Adaptive Time Window (ATW) algorithm. This not only enhances the accuracy of detection but also reduces computation time, making our solution more efficient. Importantly, the proposed ATW algorithm completely eliminates window overlap redundancy and edge effects in the system. The second step classifies abnormal driving patterns with the proposed 1D CNN model. Our results demonstrate that the proposed solution is highly accurate and has a weighted accuracy of 97.14%. Additionally, as part of this research, we have curated and released a labeled Indian dataset comprising five distinct rash driving patterns: Lane Weaving, Lane Swerving, Hard Braking, Hard Cornering, and Quick U-turn. This dataset can be valuable for further studies and aid in developing multimodal rash driving detection systems.
Workshop Eyes Speak Louder: Harnessing Deep Features From Low-Cost Camera Video for Anxiety Detection

Nandigramam Sai Harshit, Nilesh Kumar Sahu, and Haroon R Lone

In Proceedings of the 10th Workshop on Body-Centric Computing Systems, colocated with ACM MobiSys 2024, 2024

Abs DOI

Social anxiety is a common mental health disorder, affecting approximately 36% of the world’s population. Currently, diagnosing social anxiety disorder relies on clinical interviews and self-reported questionnaires, which are prone to subjective biases. Discovering non-intrusive, objective markers of anxiety, such as eye gaze patterns, could enhance current diagnostic approaches and facilitate early assessment, thereby reducing the treatment gap. Previous research has explored predicting anxiety states using physiological signals and audio data, but conclusive results remain elusive. In this study, we propose leveraging eye gaze features to predict one’s anxiety levels using deep eye gaze features. One hundred eleven participants engaged in an anxiety provoking speech activity were recorded using a low-cost smartphone camera. Further, we used a variational autoencoder to extract deep eye gaze features from the raw video features, which were then fed into machine-learning models to predict participant’s anxiety levels. Our result shows that the Random Forest classification model achieved the highest average accuracy, precision, recall, & F1 score of 100% (averaged over five folds), suggesting its capability to detect participant’s anxiety in real-world settings.
Journal Wearable Technology Insights: Unveiling Physiological Responses During Three Different Socially Anxious Activities

Nilesh Kumar Sahu, Snehil Gupta, and Haroon R Lone

ACM Journal on Computing and Sustainable Societies, 2024

Abs DOI

Wearable technology holds promise for monitoring and managing Social Anxiety Disorder (SAD), yet the absence of clear biomarkers specific to SAD hampers its effectiveness. This paper explores this issue by presenting a study investigating variances in heart rate, heart rate variability, and skin conductance between socially anxious and non-anxious individuals. One hundred eleven non-clinical student participants participated in groups of three in three anxiety-provoking activities (i.e., speech, group discussion, and interview) in a controlled lab-based study. During the study, electrocardiogram (ECG) and electrodermal activity (EDA) signals were captured via on-body electrodes. During data analysis, participants were divided into four groups based on their self-reported anxiety level (“None”, “mild”, “moderate”, and “severe”). Between-group analysis shows that discriminating ECG features (i.e., HR and MeanNN) could identify anxious individuals during anxiety-provoking activities, while EDA could not. Moreover, the discriminating ECG features improved the classification accuracy of anxious and non-anxious individuals in different machine-learning techniques. The findings need to be further scrutinized in real-world settings for the generalizability of the results.
Journal Unveiling Social Anxiety: Analyzing Acoustic and Linguistic Traits in Impromptu Speech within a Controlled Study

Nilesh Kumar Sahu, Manjeet Yadav, and Haroon R Lone

ACM Journal on Computing and Sustainable Societies, 2024

Abs DOI

Early detection and treatment of Social Anxiety Disorder (SAD) is crucial. However, current diagnostic methods have several drawbacks, including being time-consuming for clinical interviews, susceptible to emotional bias for self-reports, and inconclusive for physiological measures. Our research focuses on a digital approach using acoustic and linguistic features extracted from participants’ “speech” for diagnosing SAD. Our methodology involves identifying correlations between extracted features and SAD severity, selecting the effective features, and comparing classical machine learning and deep learning methods for predicting SAD. Our results demonstrate that both acoustic and linguistic features outperform deep learning approaches when considered individually. Logistic Regression proves effective for acoustic features, while Random Forest excels with linguistic features, achieving the highest accuracy of 85.71%. Our findings pave the way for non-intrusive SAD diagnosing that can be used conveniently anywhere, facilitating early detection.
Preprint Beyond Questionnaires: Video Analysis for Social Anxiety Detection

Nilesh Kumar Sahu, Nandigramam Sai Harshit, Rishabh Uikey, and 1 more author

2024
Preprint Breaking the silence: validating social interaction anxiety scale (SIAS) and unveiling social anxiety trends among indian students in the post-COVID era

Nilesh Kumar Sahu, Bhagyalakshmi Ajith, Snehil Gupta, and 1 more author

2024
Preprint Harnessing Smartwatch Microphone Sensors for Cough Detection and Classification

Pranay Jaiswal and Haroon R. Lone

2024
Journal DECODE: Data-driven Energy Consumption Prediction leveraging Historical Data and Environmental Factors in Buildings

Aditya Mishra, Haroon R. Lone, and Aayush Mishra

Energy and Buildings, 2024

Abs DOI

Energy prediction in buildings plays a crucial role in effective energy management. Precise predictions are essential for achieving optimal energy consumption and distribution within the grid. This paper introduces a Long Short-Term Memory (LSTM) model designed to forecast building energy consumption using historical energy data, occupancy patterns, and weather conditions. The LSTM model provides accurate short, medium, and long-term energy predictions for residential and commercial buildings compared to existing prediction models. We compare our LSTM model with established prediction methods, including linear regression, decision trees, and random forest. Encouragingly, the proposed LSTM model emerges as the superior performer across all metrics. It demonstrates exceptional prediction accuracy, boasting the highest R2 score of 0.97 and the most favorable mean absolute error (MAE) of 0.007. An additional advantage of our developed model is its capacity to achieve efficient energy consumption forecasts even when trained on a limited dataset. We address concerns about overfitting (variance) and underfitting (bias) through rigorous training and evaluation on real-world data. In summary, our research contributes to energy prediction by offering a robust LSTM model that outperforms alternative methods and operates with remarkable efficiency, generalizability, and reliability.

2023

Journal Comparative Assessment of Smartwatch Photoplethysmography Accuracy

Pranay Jaiswal, Nilesh Kumar Sahu, and Haroon R Lone

IEEE Sensors Letters, 2023

Abs DOI

Nowadays, photoplethysmography (PPG) sensor integrated into smartwatches monitors heart health. Despite its convenience, PPG is susceptible to motion artifacts and noise. In this work, we rigorously investigate the smartwatch’s PPG accuracy against medical-grade electrocardiogram (ECG) and PPG devices during different activities (i.e., sitting, standing, and walking). We conducted a user study with 32 healthy student participants and computed different heart rate (HR) features from the devices. Our analysis shows that the smartwatch PPG features, pnn50, low-frequency power (LF), and high-frequency power (HF), differ significantly from the ECG device during the activities. However, other time-domain features, such as HR, mean normal, standard deviation of normal, and root mean square of successive differences do not differ significantly during different activities. Furthermore, both the medical-grade and smartwatch PPG measurements (pnn50, HF, and LF) differed significantly from the ECG during standing and walking. The smartwatch’s PPG accuracy compared with the ECG device reinforces its potential as an alternative health monitoring device.
Conference Occupancy counting in dense and sparse settings with a low-cost thermal camera

Soumya R Sahoo and Haroon R Lone

In 2023 15th International Conference on COMmunication Systems & NETworkS (COMSNETS), 2023

Abs DOI

Occupancy detection and counting are crucial to automate buildings’ lighting, heating, and cooling systems. However, most existing occupancy detection and counting techniques are underperforming or privacy-averse. In this work, we propose an adaptive blob filtering algorithm for processing thermal images and evaluate several classification algorithms for occupancy counting. We evaluate the performance of the proposed algorithm using a low-cost privacy-preserving thermal camera under both sparse and dense occupancy settings. Our results show that the proposed algorithm improves occupancy counting results significantly compared to two existing baselines, resulting in an average accuracy of 84.5% under dense classroom settings.

2020

Journal Predicting subjective measures of social anxiety from sparsely collected mobile sensor data

Haroon Rashid, Sanjana Mendu, Katharine E Daniel, and 4 more authors

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2020

Abs DOI

Exploiting the capabilities of smartphones for monitoring social anxiety shows promise for advancing our ability to both identify indicators of and treat social anxiety in natural settings. Smart devices allow researchers to collect passive data unobtrusively through built-in sensors and active data using subjective, self-report measures with Ecological Momentary Assessment (EMA) studies. Prior work has established the potential to predict subjective measures from passive data. However, the majority of the past work on social anxiety has focused on a limited subset of self-reported measures. Furthermore, the data collected in real-world studies often results in numerous missing values in one or more data streams, which ultimately reduces the usable data for analysis and limits the potential of machine learning algorithms. We explore several approaches for addressing these problems in a smartphone based monitoring and intervention study of eighty socially anxious participants over a five week period. Our work complements and extends prior work in two directions: (i) we show the predictability of seven different self-reported dimensions of social anxiety, and (ii) we explore four imputation methods to handle missing data and evaluate their effectiveness in the prediction of subjective measures from the passive data. Our evaluation shows imputation of missing data reduces prediction error by as much as 22%. We discuss the implications of these results for future research.

2019

Journal I-BLEND, a campus-scale commercial and residential buildings electrical energy dataset

Haroon Rashid, Pushpendra Singh, and Amarjeet Singh

Scientific data, Nature, 2019

Abs DOI

Efficient energy consumption at the building level is vital for sustainability. Providing energy efficient systems and solutions requires an understanding of how energy gets consumed. However, there is a general lack of large-scale open datasets about the energy consumption of buildings, which hinders the research. The recent emergence of smart energy meters makes it possible to collect such data, which can then be used for analysis. In this paper, we release I-BLEND, 52 months of electrical energy dataset at a one-minute sampling rate from commercial and residential buildings of an academic institute campus in an emerging economy, India. Also, we provide occupancy datasets at a 10-minute sampling rate for each of the campus buildings. To the best of our knowledge, this is the first such dataset from India. Public availability of such fine-granular data will allow users to perform different research tasks such as analyzing the impact of weather or occupancy schedule on energy consumption, detecting anomalies, and developing algorithms for predictive maintenance.
Journal Can non-intrusive load monitoring be used for identifying an appliance’s anomalous behaviour?

Haroon Rashid, Pushpendra Singh, Vladimir Stankovic, and 1 more author

Applied energy, 2019

Abs DOI

Identification of faulty appliance behaviour in real time can signal energy wastage and the need for appliance servicing or replacement leading to energy savings. The problem of appliance fault or anomaly detection has been tackled vastly in relation to submetering, which is not scalable since it requires separate meters for each appliance. At the same time, for applications such as energy feedback, Non-intrusive load monitoring (NILM) has been recognised as a scalable and practical alternative to submetering. However, the usability of NILM for anomaly detection has not yet been investigated. Since the goal of NILM is to provide energy consumption estimate, it is unclear if the signal fidelity of appliance signatures generated by state-of-the-art NILM is sufficient to enable accurate appliance fault detection. In this paper, we attempt to determine whether appliance signatures detected by NILM can be used directly for anomaly detection. This is carried out by proposing an anomaly detection algorithm which performs well for submetering data and evaluate its ability to identify the same faulty behaviour of appliances but with NILM-generated appliance power traces. Our results on a dataset of six residential homes using four state-of-the-art NILM algorithms show that, on average, NILM traces are not as robust to identification of faulty behaviour as compared to using submetered data. We discuss in detail observations pertaining to the reconstructed appliance signatures following NILM and their fidelity with respect to noise-free submetered data.
Conference Evaluation of non-intrusive load monitoring algorithms for appliance-level anomaly detection

Haroon Rashid, Vladimir Stankovic, Lina Stankovic, and 1 more author

In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019

Abs DOI

Appliance fault in buildings resulting in abnormal energy consumption is known as an anomaly. Traditionally, anomaly detection is performed either at aggregate, i.e., meter-level, or at appliance level. Meter-level anomaly detection does not identify the anomaly-causing appliance, while appliance-level detection requires submetering each appliance in the building. Non-Intrusive Load Monitoring (NILM) has been proposed as an alternative to submetering to detect when appliances are running as well as estimate the appliance energy consumption. So far, applications have revolved around meaningful energy feedback. In this paper, we assess whether NILM can indeed be used for anomaly detection, as an alternative to submetering. We propose a supervised anomaly detection approach, AEM, and evaluate the effectiveness of NILM for anomaly detection. The proposed approach first learns an appliance’s normal operation and then monitors its energy consumption for anomaly detection. We resort to real data, aggregate and subme-tered data from the two-year long REFIT dataset. We explain why anomaly detection performs worse with NILM data as compared to submetered data, highlighting the need for new, anomaly-aware NILM approaches.

2018

Conference Rimor: Towards identifying anomalous appliances in buildings

Haroon Rashid, Nipun Batra, and Pushpendra Singh

In Proceedings of the 5th Conference on Systems for Built Environments, 2018

Abs DOI

Buildings across the world contribute about one-third of the total energy consumption. Studies report that anomalies in energy consumption caused by faults and abnormal appliance usage waste up to 20% of energy in buildings. Recent works leverage smart meter data to find such anomalies; however, such works do not identify the appliance causing the anomaly. Moreover, most of these works are not real-time and report the anomaly at the end of the day. In this paper, we propose a technique named Rimor that addresses these limitations. Rimor predicts the energy consumption of a home using historical energy data and contextual information and flags an anomaly when the actual energy consumption deviates significantly from the predicted consumption. Further, it identifies anomalous appliance(s) by using easy-to-collect appliance power ratings. We evaluated it on four real-world energy datasets containing 51 homes and found it to be 15% more accurate in detecting anomalies as compared to four other baseline approaches. Rimor reports an appliance identification accuracy of 82%. In addition, we also release an anomaly annotated energy dataset for the research community.
Conference Monitor: An abnormality detection approach in buildings energy consumption

Haroon Rashid and Pushpendra Singh

In 2018 IEEE 4th international conference on collaboration and internet computing (CIC), 2018

Abs DOI

With the growth of smart cities, more buildings are now being instrumented with smart meters for providing better energy efficiency for sustainable development. Buildings consume around 39% of electrical energy worldwide and studies report that wasteful consumer behavior such as forgetting to switch off an appliance after use or using an appliance with misconfigured settings adds about one-third to buildings consumption. These instances result in deviations in energy consumption as compared to its normal consumption and are called as abnormalities. Detecting such abnormalities is important for reducing energy wastage. Existing methods detect abnormalities by analyzing smart meter data, however, they result in a high number of false positive alarms. This inaccuracy results in ignoring the alarms by building administrators which also affects genuine alarms. Thus, reducing the false positive alarms and making detection algorithms more accurate is a major aim. In this paper, we present our novel approach, called Monitor, which first identifies patterns in past consumption data and then uses these patterns to detect abnormalities. Our approach requires smart meter data only and reduces the rate of false positive alarms considerably. We have evaluated our approach on 16 weeks smart meter data of real world buildings. The comparison of this approach with existing approaches shows that our approach improves the accuracy by up to 24% in best scenario and on average by 14%. This improvement in accuracy reduces the rate of false positive alarms significantly and makes it more suitable for real-world deployments.
Conference Want to reduce energy consumption, whom should we call?

Priyanka Mary Mammen, Hareesh Kumar, Krithi Ramamritham, and 1 more author

In Proceedings of the Ninth International Conference on Future Energy Systems, 2018

Abs DOI

Power shortage is a serious issue in developing nations. During periods of high demand, utilities need to motivate the consumers to curtail their consumption for maintaining grid stability and avoiding blackouts or brownouts. Identification of suitable candidates is essential for such events, as the budget set aside by utilities for Demand Response (DR) events for providing incentives to the consumers should not exceed the added production cost due to peaks. Similarly, from the consumers’ point of view, participation comes with the compromise to their convenience. Hence, the selection criteria should be such that it minimizes the peaking cost to the utility without affecting consumer comfort. In this paper, we present SmarDeR, a smart DR consumer selection strategy which considers several factors and consolidates them into a single function which can work in different modes to strategically choose the candidates for the DR event based on the goals specified by the utility. We evaluate different policies and metrics for approaching the right consumers for participating in the DR events. Thereby, we can maintain a fair distribution of requests among the most relevant and reliable users. Experiments with smart-meter data from apartments in our campus demonstrates the effectiveness of our SmarDeR approach

2017

Conference Want to reduce energy consumption? don’t depend on the consumers!

Haroon Rashid, Priyanka Mary Mammen, Siddharth Singh, and 3 more authors

In Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments, 2017

Abs DOI

Motivating users to save energy is considered to be the holy grail of smart energy management. However, many studies have shown that changing user behavior from an energy standpoint is a very difficult problem. Furthermore, in countries such as the United States, users do not have sufficient monetary incentives to become energy conscious, given the low cost of electricity, and more generally, energy. In this paper, we study this issue in a developing economy and present a user study of 41 apartments in a high-rise apartment complex in India. Through a combination of fine-grain energy meter usage data and detailed user surveys, we find that these users may be no more energy conscious or motivated to adopt energy efficiency measures than their counterparts in Western nations. Our study challenges the belief that energy prices are higher in developing regions and hence, users in developing regions tend to be more energy-aware than those elsewhere. Consequently, and importantly, we argue that utility companies, rather than end-users, should be the vanguard for realizing energy efficiency improvement at consumer premises in order to obtain grid-wide benefits such as peak load reduction or avoiding blackouts. Towards this goal, we argue for a sustained research effort into utility-scale energy analytic approaches, for example, to identify end users who are large consumers along with the underlying causes of their consumption. Utilities can deploy such approaches and then aggressively target these users for energy efficiency improvements.
Conference Revisiting selection of residential consumers for demand response programs

Haroon Rashid, Pushpendra Singh, and Krithi Ramamritham

In Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments, 2017

Abs DOI

Electrical utilities depend on Demand Response programs to manage peak loads by incentivizing consumers to voluntarily curtail a portion of their load during a specified period. Utilities first categorize consumers based on their energy consumption patterns into different clusters and then request consumers of a particular cluster to participate in the demand response program. At a coarse level, clustering approaches do well, but we may not be able to correctly predict which cluster’s profile will fit that day’s power availability. We address this issue by examining the consistency of consumer’s consumption patterns across several consecutive days. We demonstrate that measuring consistency quantitatively helps to understand predictability of consumer’s energy consumption. In the rest of the paper, we provide details of our proposed consistency metric. Further, we propose a methodology to select a few consumers among the consistent ones such that they have a peak at the time specified by the demand response program. We validate our approach using real-world energy consumption data from residential buildings.

2015

Journal Dead reckoning localisation technique for mobile wireless sensor networks

Haroon Rashid and Ashok Kumar Turuk

IET Wireless Sensor Systems, 2015

Abs DOI

Localisation in wireless sensor networks (WSNs) not only provides a node with its geographical location but also a basic requirement for other applications such as geographical routing. Although a rich literature is available for localisation in static WSN, not enough work is done for mobile WSNs, owing to the complexity because of node mobility. Most of the existing techniques for localisation in mobile WSNs use Monte Carlo localisation (MCL), which is not only time consuming but also memory intensive. They, consider either the unknown nodes or anchor nodes to be static. In this study, the authors propose a technique called dead reckoning localisation for mobile WSNs (DRLMSN). In the proposed technique all nodes (unknown nodes as well as anchor nodes) are mobile. Localisation in DRLMSN is done at discrete time intervals called checkpoints. Unknown nodes are localised for the first time using three anchor nodes. For their subsequent localisations, only two anchor nodes are used. The proposed technique estimates two possible locations of a node using Bézout’s theorem. A dead reckoning approach is used to select one of the two estimated locations. The authors have evaluated DRLMSN through simulation using Castalia simulator, and is compared with a similar technique called received signal strength-MCL.

2013

Journal Localization of wireless sensor networks using a single anchor node

Haroon Rashid and Ashok Kumar Turuk

Wireless personal communications, 2013

Abs DOI

Localization of nodes in a sensor network is essential for the following two reasons: (i) to know the location of a node reporting the occurrence of an event, and (ii) to initiate a prompt action whenever necessary. Different localization techniques have been proposed in the literature. Most of these techniques use three location aware nodes for localization of an unknown node. Moreover, the localization techniques also differ from environment to environment. In this paper, we proposed a localization technique for grid environment. Sensor nodes are deployed in a grid pattern and localization is achieved using a single location aware or anchor node. We have identified three types of node in the proposed scheme: (i) Anchor node, (ii) Unknown node and (iii) Special node. First, the special nodes are localized with respect to the anchor node, then the unknown nodes are localized using trilateration mechanism. We have compared the proposed scheme with an existing localization algorithm for grid deployment called Multiduolateration. The parameters considered for localization are localization time and localization error. It is observed that localization time and error in the proposed scheme is lower than that of Multiduolateration.