Literature Review
What existing research says about pediatric upper-limb rehabilitation
Wearable Sensor-Based Upper Limb Rehabilitation
The ARMIGO glove is a rehabilitation device for children designed to support upper limb rehabilitation through sensors placed at each of the four joints — fingers, wrist, elbow, and shoulder. Wearable Sensing Technologies using Inertial Measurement Units (MPU-9250) and Flex Sensors are among the best technologies for collecting real-time kinematic data. Unlike haptic devices, the ARMIGO glove focuses solely on measuring joint movement without applying force. However, most current rehabilitation devices analyse single joints rather than assessing the interaction of multiple joints simultaneously.
Virtual Reality and Gamification in Pediatric Rehabilitation
Using virtual reality (VR) and serious gaming as rehabilitation methods for pediatric patients has proven effective in overcoming challenges of traditional approaches that struggle with patient motivation. VR makes repetitive exercises more engaging by delivering them as game-based therapies, leading to improved patient attention and compliance. Several studies show benefit in upper limb recovery through goal-directed activities in gaming. However, current VR systems focus primarily on gross motor movements, lack emotional adaptations for children, and are prohibitively expensive for widespread adoption.
Machine Learning for Movement Recognition & Quality Assessment
Machine learning is increasingly applied in rehabilitation to classify movement types using sensor data. Traditional approaches are limited in classifying upper extremity movements due to patient variability. Deep learning with Long Short-Term Memory (LSTM) networks has been shown to effectively classify complex sequential movement patterns in wearable sensor data. Research on LSTMs shows potential for improving classification of movement patterns and assessment of rehabilitation sessions. Key limitations include narrow datasets from adult patients and difficulty integrating ML into real-time immersive environments.
Tele-Rehabilitation and Remote Monitoring Systems
Tele-rehabilitation technologies have emerged as a new approach for therapist-led remote rehabilitation via IoT. Systems with continuous movement measurement improve quality of care for pediatric patients through web-based platforms that share progress between therapists and patients. However, existing products lack full integration of monitoring within the rehabilitation program itself. Additionally, resource constraints in developing nations restrict many children with motor disabilities from accessing adequate physiotherapy services.
Research Gap
Critical gaps identified in existing rehabilitation systems
The literature review reveals critical gaps in existing rehabilitation systems. Table 1 presents a structured comparison of existing systems against the ArmiGo platform, highlighting the specific capabilities that remain unaddressed.
Table 1.2.1 — Comparison of Existing Systems vs ArmiGo
Research Gap Analysis Table
Based on this analysis, the following specific research gaps are identified:
No existing published system integrates wearable sensor tracking for all four upper limb joints — shoulder, elbow, wrist, and fingers — simultaneously with joint-specific LSTM classification and dedicated therapeutic VR game environments in a single, affordable platform designed for children with hemiplegia.
LSTM models for therapeutic movement classification have not been deployed as embedded real-time ONNX Runtime inference components within a commercial VR game engine for paediatric rehabilitation at interactive frame rates.
No affordable, child-centred multi-joint rehabilitation system exists for the Sri Lankan clinical context, where therapist shortages, geographic barriers, and cost constraints severely limit access to conventional physiotherapy.
Multi-level monitoring ecosystems combining therapist clinical dashboard, parent mobile application, AI voice assistant, and four-joint wearable therapy have not been implemented in a unified system.
Most existing VR rehabilitation systems target adult stroke populations and lack the child-specific narrative, motivational, and cognitive design elements required for sustained engagement in children aged 5–15.
Research Problem
The central challenge ArmiGo addresses
Problem & Impact Overview
"Children with hemiplegia in Sri Lanka and similar low-resource settings lack access to an affordable, engaging, and clinically effective multi-joint upper limb rehabilitation system capable of delivering high-repetition therapeutic exercise across the shoulder, elbow, wrist, and finger joints, providing real-time intelligent movement classification, and supporting continuous remote therapist oversight — all within a home-deployable platform accessible to families regardless of geographic or economic barriers."
This problem manifests through four interconnected sub-problems:
The absence of a complete wearable sensor suite covering all four upper limb joints in a single integrated, child-comfortable, low-cost system capable of real-time kinematic capture in both clinical and home environments.
The lack of validated LSTM models for real-time, embedded classification of therapeutic movements for each joint — shoulder, elbow, wrist, and fingers — deployed within a VR game engine at interactive frame rates without dependency on cloud inference.
The non-existence of a multi-game VR rehabilitation ecosystem that maps each joint's therapeutic movement repertoire to distinct engaging in-game actions, providing immediate multi-sensory feedback that motivates sustained high-repetition practice in children.
The unavailability of a unified multi-level monitoring infrastructure that connects all four joint therapy components to a shared cloud database, therapist dashboard, and parent mobile application, enabling continuous remote clinical oversight across all joints simultaneously.
Research Objectives
What ArmiGo sets out to achieve
Research Objectives Overview
Main Objective
To design, develop, and evaluate an integrated four-joint wearable movement tracking, LSTM classification, and VR therapy system — ArmiGo — for upper limb rehabilitation in children with hemiplegia, enabling real-time per-joint movement classification and therapeutic VR gameplay across the shoulder, elbow, wrist, and finger joints within a unified, affordable, and home-deployable platform.
Specific Objectives by Joint Component:
Develop a wearable MPU-9250 IMU shoulder module with Kalman-filtered UDP streaming, session calibration, and an LSTM classifier achieving at least 80% average accuracy across six therapeutic shoulder movement classes, integrated via ONNX Runtime into an Unreal Engine VR game.
Develop a wearable MPU-6050 IMU elbow sleeve with real-time signal conditioning, LSTM-based classification achieving at least 90% mean accuracy, and integration with the "Knight's Quest: The Shield of Strength" VR game via ONNX Runtime.
Develop a wearable MPU-9250 IMU wrist module with Kalman filtering and complementary filter sensor fusion, LSTM-based classification of four wrist movements (flexion, extension, pronation, supination), integrated with the "Fishing Adventure" VR game.
Develop a flex-sensor glove with five per-finger sensors and ESP32 processing, implementing two-step calibration and LSTM-based classification of five therapeutic finger movements with at least 90% target accuracy, integrated with the "Magic Quest: The Enchanted Fingers" VR game.
Establish a shared WiFi WebSocket data pipeline from all four wearable modules to Unreal Engine 5, with a unified Azure Cosmos DB cloud backend, React.js therapist dashboard, and parent mobile application providing multi-level remote monitoring across all joints.
Evaluate the usability, comfort, clinical utility, and technical performance of the complete ArmiGo system through structured stakeholder feedback from parents, children, and a consulting physiotherapist.
Methodology
Research approach and development process
Requirement Analysis
Clinician interviews, literature synthesis, and parent/patient surveys to define functional and non-functional requirements for all four joint modules.
Iterative Design & Build
Agile sprints covering hardware prototyping (ESP32 + IMU sensors), LSTM model training, VR game development in Unreal Engine 5, and backend/mobile dashboard implementation.
Evaluation & Validation
Usability testing with target users (children, parents, therapists), clinical expert review, and quantitative performance metrics analysis across all four joint modules.
Technology Stack — Multi-Layer Architecture:
Hardware Layer
- ESP32 Microcontroller (Wi-Fi + Bluetooth)
- MPU-6050 / MPU-9250 IMU Sensors
- Flex Sensors (5-finger glove)
- Child-friendly elastic wearable sleeves
- Google Cardboard / Oculus Quest VR headsets
Firmware & Data Handling
- Arduino IDE & MicroPython
- Kalman Filter for noise reduction
- Moving average signal smoothing
- Wi-Fi / Serial communication protocols
- Real-time kinematic data preprocessing
Machine Learning Layer
- LSTM Networks (temporal movement classification)
- Support Vector Machines (static posture detection)
- TensorFlow Lite (embedded deployment)
- ONNX Runtime (VR game integration)
- Python · scikit-learn · Keras
Game Development Layer
- Unreal Engine 5 (primary VR platform)
- Unity 3D (alternative environments)
- 4 therapeutic joint-specific VR games
- Adaptive difficulty scaling
- Real-time ONNX Runtime gesture integration
Cloud & Monitoring Layer
- Azure Cosmos DB (unified cloud backend)
- Firebase / AWS S3 (media & session storage)
- React.js therapist clinical dashboard
- WebSocket real-time data pipeline
- AI Voice Assistant (Dialogflow / Rasa)
Mobile & Caregiver Interface
- React Native / Flutter parent app
- Progress tracking & analytics
- Teleconsultation feature
- Push notifications & session summaries
- Home rehabilitation continuity support