14 KiB

Raw Permalink Blame History

Multi-Object Tracking with Ultralytics YOLO

Object tracking, a key aspect of video analytics, involves identifying the location and class of objects within video frames and assigning a unique ID to each detected object as it moves. This capability enables a wide range of applications, from surveillance and security systems to real-time sports analysis and autonomous vehicle navigation. Learn more about tracking on our tracking documentation page.

🎯 Why Choose Ultralytics YOLO for Object Tracking?

Ultralytics YOLO trackers provide output consistent with standard object detection but add persistent object IDs. This simplifies the process of tracking objects in video streams and performing subsequent analyses. Here’s why Ultralytics YOLO is an excellent choice for your object tracking needs:

Efficiency: Process video streams in real-time without sacrificing accuracy.
Flexibility: Supports multiple robust tracking algorithms and configurations.
Ease of Use: Offers straightforward Python API and CLI options for rapid integration and deployment.
Customizability: Easily integrates with custom-trained YOLO models, enabling deployment in specialized, domain-specific applications.

Watch: Object Detection and Tracking with Ultralytics YOLOv8.

✨ Features at a Glance

Ultralytics YOLO extends its powerful object detection features to deliver robust and versatile object tracking:

Real-Time Tracking: Seamlessly track objects in high-frame-rate videos.
Multiple Tracker Support: Choose from a selection of established tracking algorithms.
Customizable Tracker Configurations: Adapt the tracking algorithm to specific requirements by adjusting various parameters.

🛠️ Available Trackers

Ultralytics YOLO supports the following tracking algorithms. Enable them by passing the relevant YAML configuration file, such as tracker=tracker_type.yaml:

BoT-SORT: Use botsort.yaml to enable this tracker. Based on the BoT-SORT paper and its official code implementation.
ByteTrack: Use bytetrack.yaml to enable this tracker. Based on the ByteTrack paper and its official code implementation.

The default tracker is BoT-SORT.

⚙️ Usage

To run the tracker on video streams, use a trained Detect, Segment, or Pose model like Ultralytics YOLO11n, YOLO11n-seg, or YOLO11n-pose.

# Python
from ultralytics import YOLO

# Load an official or custom model
model = YOLO("yolo11n.pt")  # Load an official Detect model
# model = YOLO("yolo11n-seg.pt")  # Load an official Segment model
# model = YOLO("yolo11n-pose.pt")  # Load an official Pose model
# model = YOLO("path/to/best.pt")  # Load a custom trained model

# Perform tracking with the model
results = model.track(source="https://youtu.be/LNwODJXcvt4", show=True)  # Tracking with default tracker
# results = model.track(source="https://youtu.be/LNwODJXcvt4", show=True, tracker="bytetrack.yaml")  # Tracking with ByteTrack tracker

# CLI
# Perform tracking with various models using the command line interface
yolo track model=yolo11n.pt source="https://youtu.be/LNwODJXcvt4" # Official Detect model
# yolo track model=yolo11n-seg.pt source="https://youtu.be/LNwODJXcvt4"  # Official Segment model
# yolo track model=yolo11n-pose.pt source="https://youtu.be/LNwODJXcvt4" # Official Pose model
# yolo track model=path/to/best.pt source="https://youtu.be/LNwODJXcvt4" # Custom trained model

# Track using ByteTrack tracker
# yolo track model=path/to/best.pt tracker="bytetrack.yaml"

As shown above, tracking is available for all Detect, Segment, and Pose models when run on videos or streaming sources.

🔧 Configuration

Tracking Arguments

Tracking configuration shares properties with the Predict mode, such as conf (confidence threshold), iou (Intersection over Union threshold), and show (display results). For additional configurations, refer to the Predict mode documentation.

# Python
from ultralytics import YOLO

# Configure the tracking parameters and run the tracker
model = YOLO("yolo11n.pt")
results = model.track(source="https://youtu.be/LNwODJXcvt4", conf=0.3, iou=0.5, show=True)

# CLI
# Configure tracking parameters and run the tracker using the command line interface
yolo track model=yolo11n.pt source="https://youtu.be/LNwODJXcvt4" conf=0.3 iou=0.5 show

Tracker Selection

Ultralytics allows you to use a modified tracker configuration file. Create a copy of a tracker config file (e.g., custom_tracker.yaml) from ultralytics/cfg/trackers and adjust any configurations (except tracker_type) according to your needs.

# Python
from ultralytics import YOLO

# Load the model and run the tracker with a custom configuration file
model = YOLO("yolo11n.pt")
results = model.track(source="https://youtu.be/LNwODJXcvt4", tracker="custom_tracker.yaml")

# CLI
# Load the model and run the tracker with a custom configuration file using the command line interface
yolo track model=yolo11n.pt source="https://youtu.be/LNwODJXcvt4" tracker='custom_tracker.yaml'

For a comprehensive list of tracking arguments, consult the Tracking Configuration files in the repository.

🐍 Python Examples

Persisting Tracks Loop

This Python script uses OpenCV (cv2) and Ultralytics YOLO11 to perform object tracking on video frames. Ensure you have installed the necessary packages (opencv-python and ultralytics). The persist=True argument indicates that the current frame is the next in a sequence, allowing the tracker to maintain track continuity from the previous frame.

# Python
import cv2

from ultralytics import YOLO

# Load the YOLO11 model
model = YOLO("yolo11n.pt")

# Open the video file
video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

# Loop through the video frames
while cap.isOpened():
    # Read a frame from the video
    success, frame = cap.read()

    if success:
        # Run YOLO11 tracking on the frame, persisting tracks between frames
        results = model.track(frame, persist=True)

        # Visualize the results on the frame
        annotated_frame = results[0].plot()

        # Display the annotated frame
        cv2.imshow("YOLO11 Tracking", annotated_frame)

        # Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        # Break the loop if the end of the video is reached
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()

Note the use of model.track(frame) instead of model(frame), which specifically enables object tracking. This script processes each video frame, visualizes the tracking results, and displays them. Press 'q' to exit the loop.

Plotting Tracks Over Time

Visualizing object tracks across consecutive frames offers valuable insights into movement patterns within a video. Ultralytics YOLO11 makes plotting these tracks efficient.

The following example demonstrates how to use YOLO11's tracking capabilities to plot the movement of detected objects. The script opens a video, reads it frame by frame, and uses the YOLO model built on PyTorch to identify and track objects. By storing the center points of the detected bounding boxes and connecting them, we can draw lines representing the paths of tracked objects using NumPy for numerical operations.

# Python
from collections import defaultdict

import cv2
import numpy as np

from ultralytics import YOLO

# Load the YOLO11 model
model = YOLO("yolo11n.pt")

# Open the video file
video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

# Store the track history
track_history = defaultdict(lambda: [])

# Loop through the video frames
while cap.isOpened():
    # Read a frame from the video
    success, frame = cap.read()

    if success:
        # Run YOLO11 tracking on the frame, persisting tracks between frames
        result = model.track(frame, persist=True)[0]

        # Get the boxes and track IDs
        if result.boxes and result.boxes.is_track:
            boxes = result.boxes.xywh.cpu()
            track_ids = result.boxes.id.int().cpu().tolist()

            # Visualize the result on the frame
            frame = result.plot()

            # Plot the tracks
            for box, track_id in zip(boxes, track_ids):
                x, y, w, h = box
                track = track_history[track_id]
                track.append((float(x), float(y)))  # x, y center point
                if len(track) > 30:  # retain 30 tracks for 30 frames
                    track.pop(0)

                # Draw the tracking lines
                points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))
                cv2.polylines(frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

        # Display the annotated frame
        cv2.imshow("YOLO11 Tracking", frame)

        # Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        # Break the loop if the end of the video is reached
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()

Multithreaded Tracking

Multithreaded tracking allows running object tracking on multiple video streams simultaneously, which is highly beneficial for systems handling inputs from several cameras, improving efficiency through concurrent processing.

This Python script utilizes Python's threading module for concurrent tracker execution. Each thread manages tracking for a single video file.

The run_tracker_in_thread function accepts parameters like the video file path, model, and a unique window index. It contains the main tracking loop, reading frames, running the tracker, and displaying results in a dedicated window.

This example uses two models, yolo11n.pt and yolo11n-seg.pt, tracking objects in video_file1 and video_file2, respectively.

Setting daemon=True in threading.Thread ensures threads exit when the main program finishes. Threads are started with start() and the main thread waits for their completion using join().

Finally, cv2.destroyAllWindows() closes all OpenCV windows after the threads finish.

# Python
import threading

import cv2

from ultralytics import YOLO

# Define model names and video sources
MODEL_NAMES = ["yolo11n.pt", "yolo11n-seg.pt"]
SOURCES = ["path/to/video.mp4", "0"]  # local video, 0 for webcam


def run_tracker_in_thread(model_name, filename):
    """
    Run YOLO tracker in its own thread for concurrent processing.

    Args:
        model_name (str): The YOLO11 model object.
        filename (str): The path to the video file or the identifier for the webcam/external camera source.
    """
    model = YOLO(model_name)
    results = model.track(filename, save=True, stream=True)
    for r in results:
        pass


# Create and start tracker threads using a for loop
tracker_threads = []
for video_file, model_name in zip(SOURCES, MODEL_NAMES):
    thread = threading.Thread(target=run_tracker_in_thread, args=(model_name, video_file), daemon=True)
    tracker_threads.append(thread)
    thread.start()

# Wait for all tracker threads to finish
for thread in tracker_threads:
    thread.join()

# Clean up and close windows
cv2.destroyAllWindows()

This setup can be easily scaled to handle more video streams by creating additional threads following the same pattern. Explore more applications in our blog post on object tracking.

🤝 Contribute New Trackers

Are you experienced in multi-object tracking and have implemented or adapted an algorithm with Ultralytics YOLO? We encourage you to contribute to our Trackers section in ultralytics/cfg/trackers! Your contributions can help expand the tracking solutions available within the Ultralytics ecosystem.

To contribute, please review our Contributing Guide for instructions on submitting a Pull Request (PR) 🛠️. We look forward to your contributions!

Let's work together to enhance the tracking capabilities of Ultralytics YOLO and provide more powerful tools for the computer vision and deep learning community 🙏!

14 KiB Raw Permalink Blame History Unescape Escape