Vision-based Hand Gesture Virtual Keyboard-Mouse framework with Bilingual Next-word prediction

Touchless human-computer interaction (HCI) is increasingly sought for improved accessibility, yet existing gesture-driven keyboards remain hindered by limited vocabularies, sensitivity to illumination, high computational overhead, and inadequate support for multiple languages. To address these challenges, a modular, non-probabilistic pipeline is proposed that integrates real-time geometric encoding of MediaPipe hand landmarks for gesture recognition and two-layer long-short-term memory (LSTM) networks for multilingual next-word prediction in English and Bangla. A unified keyboard-mouse interface maps recognized gestures to on-screen key and pointer actions, while a lightweight performance monitor logs frame rate, latency, and resource usage. All components run sequentially in real time. Extensive evaluation on various hardware platforms demonstrates that nine static gestures are classified with at least 97.4% accuracy and under 0.03 ms per gesture latency, allowing frame rates up to 37.7 FPS. LSTM models achieve accuracies above 97% and perplexities below 1.15, resulting in character error rates of 3.43% for English and 8.34% for Bangla. Cross-platform statistical analysis confirms consistent usability and resource efficiency. The proposed system thus offers a low-latency scalable solution for multilingual and touchless text entry on desktop platforms, with potential extension to wearable devices in future work.