Watch & Do A Smart IoT Interaction System with Object Detection and Gaze Estimation
To detect target device is identified by various deep learning-based gaze estimation and object detection techniques and user can control the target IOT device by hand gesture.
The target device is identified by various Algorithms are used for gaze estimation and object detection techniques. Afterwards, hand gesture recognition is applied to generate an IoT device control command which is transmitted to the IoT platform. This algorithms results are not accuracy. The experimental results and case studies demonstrate the feasibility of the proposed system and imply the future research directions.
Proposed system works as follows. First, the Watch module records the opposite side of the user to detect and recognize the types of IoT devices installed in the room. Second, the Watch module detects the user’s head region and then computes a fine-grained head pose information (i.e., pitch, yaw, and roll) to estimate the user’s gaze position. With this information, the proposed system can identify the target device. Then, the Do module captures the user’s hand gestures. A combination of hand gesture information and the type of selected IoT device is then translated into an IoT command and transmitted to IoT platforms. The target device is identified by deep learning Algorithms are used for gaze estimation and object detection techniques