Deep Learning for Sidewalk Anomaly Detection

City sidewalks became unmanageable

The proliferation of micro-mobility - e-scooters, autonomous delivery bots, dockless bikes - has introduced a new layer of complexity to pedestrian zones. Traditional monitoring infrastructure, reliant on manual surveillance or simplistic motion detection, is ill-equipped to handle this dynamic mix. The result is a significant blind spot in urban safety management, leading to increased conflict between pedestrians and vehicles, and a lack of actionable data for urban planners.

Computer vision in the wild is hard

Deploying computer vision in uncontrolled outdoor environments presents non-trivial engineering challenges. Unlike controlled industrial settings, city streets are subject to extreme variability:

• High occlusion rates: In dense crowds, subjects are frequently partially or fully occluded, breaking standard tracking algorithms.
• Visual ambiguity: Distinguishing between a pedestrian running and a person on a kick-scooter requires fine-grained feature extraction that generic models miss.
• Latency constraints: For safety applications, the inference-to-alert pipeline must operate sub-100ms, requiring edge-compute rather than cloud-dependent architectures.

Edge-native deep learning architecture

TendersLab engineered a bespoke deep learning pipeline designed specifically for the chaotic nature of urban sidewalks - moving beyond simple object detection to behavioral understanding.

1. Fine-grained classification with YOLOv8

We trained a custom YOLOv8 architecture on a proprietary dataset of over 50,000 annotated urban images. The model distinguishes subtle class differences - differentiating a pedestrian from a scooter rider or cyclist with high precision, even under poor lighting conditions.

2. Temporal consistency via DeepSORT

To handle occlusion, we integrated DeepSORT (Simple Online and Realtime Tracking with a Deep Association Metric). This maintains unique IDs for each entity as they move through the frame, enabling trajectory-based behavioral inference - identifying, for instance, a vehicle moving at unsafe velocities in a pedestrian zone.

3. TensorRT optimization for the edge

The entire inference pipeline was optimized using NVIDIA TensorRT, deployed on resource-constrained edge devices (NVIDIA Jetson series) connected directly to street cameras. Processing locally reduced bandwidth usage by 95% and ensured privacy compliance by transmitting only metadata and anonymized alert clips.

From reactive to proactive urban safety

• 99.2% classification accuracy in real-world trials, effectively filtering false positives that plague traditional systems.
• Sub-50ms inference latency enabling immediate triggering of safety protocols - digital signage warnings, automated enforcement alerts.
• Rich usage analytics giving planners datasets on sidewalk usage patterns to inform infrastructure decisions.