Hybrid Augmented Model for Detecting Pedestrian Attention

Hybrid Augmented Model for Detecting Pedestrian Attention

Wang Z. (ITMO, St. Petersburg, Russia)
Zhdanov D. D. (ITMO, St. Petersburg, Russia)
Zhdanov A. D. (ITMO, St. Petersburg, Russia)

Abstract

Pedestrian safety on the road largely depends on how accurately a pedestrian perceives the traffic situation and how reliably a vehicle can assess the pedestrian’s level of situational awareness. Estimating pedestrian gaze and attention in real-world street environments is therefore an important but challenging problem due to small face resolution, motion blur, occlusions, and extreme illumination variations. In previous studies, face-based gaze estimation and pupil tracking were shown to provide fine-grained attention cues under favorable visual conditions. However, these approaches remain fragile in unconstrained street scenes, where eye visibility and facial landmark detection frequently fail, leading to unreliable or missing outputs in safety-critical situations. In this paper, we present a robust hybrid framework that extends our earlier work by introducing a pipeline of pedestrian attention estimation methods with explicit error control at each stage to enable continuous gaze direction estimation under complex urban conditions. In addition to a learning-based gaze estimator, we incorporate geometric methods and a lightweight head pose estimation network to compensate for the breakdown of individual methods under noisy input data. The proposed framework integrates face detection, gaze estimation, geometric head pose computation, and learning-based head pose prediction, including the prediction of head position and orientation at the next time step. Unlike prior gaze-centric systems, neural networks in this design are not only used to improve accuracy but are elevated to ensure continuous video processing under real-world visual degradation associated with complex traffic scenarios. Extensive experiments on open datasets collected in real street environments demonstrate that the proposed method substantially enhances the success rate and accuracy of long-distance gaze recognition, thereby improving the feasibility of end-to-end assessment of pedestrian situational awareness while maintaining accuracy comparable to gaze estimation under favorable conditions.

Keywords

pedestrian attention; gaze estimation; head pose; autonomous driving.

Edition

Proceedings of the Institute for System Programming, vol. 38, issue 3, part 3, 2026, pp. 115-124

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2026-38(3)-39

For citation

Wang Z., Zhdanov D. D., Zhdanov A. D. Hybrid Augmented Model for Detecting Pedestrian Attention. Proceedings of the Institute for System Programming, vol. 38, issue 3, part 3, 2026, pp. 115-124 DOI: 10.15514/ISPRAS-2026-38(3)-39.

Full text of the paper in pdf Back to the contents of the volume