These Cameras Were Supposed To Be E-Waste No Rtsp No Docs No Protocol Anyones Heard Of I Reverse-Engineered 100 000 Url Patterns To Make Them Work

These Cameras Were SupposedTo Be E‑Waste No RTSP No Docs No Protocol Anyone’s Heard Of I Reverse‑Engineered 100 000 Url Patterns To Make Them Work

INTRODUCTION

You inherit a stack of aging Network Video Recorders (NVRs) that were once the backbone of a small security system. They sit in a corner of your homelab, their firmware frozen in 2016, their web interfaces dead, and every attempt to pull a live stream ends in silence. The devices are officially e‑waste, yet the hardware still spins, the power LEDs glow, and the storage bays whisper promises of usable video feeds.

In a typical self‑hosted environment, the path forward would be straightforward: enable RTSP, point Frigate at the RTSP URL, and let the AI detection pipeline do its magic. For these Chinese NVRs, however, the story is different. No RTSP, no documented API, no community forum threads, and a proprietary “BUBBLE” protocol that does not appear in any public index. After two years of intermittent tinkering, the author reverse‑engineered over one hundred thousand URL pattern permutations, built a lightweight router that translates them into usable streams, and integrated the solution into a Frigate‑based surveillance stack. This guide is the definitive, hands‑on walkthrough of that journey. It assumes you are an experienced sysadmin or DevOps engineer who already runs a homelab, understands Docker, and is comfortable digging into raw network traffic. By the end you will know:

• How to identify undocumented camera protocols from client‑side traffic.
• How to systematically enumerate and reverse‑engineer URL patterns that control the stream.
• How to wrap the resulting logic in a containerised service that can be deployed alongside Frigate.
• Best practices for security, performance, and long‑term maintainability in a self‑hosted surveillance pipeline.

The article is deliberately technical, free of marketing fluff, and written for a professional audience that values reproducible infrastructure code.

---

UNDERSTANDING THE TOPIC

What is being reverse‑engineered?

The subject matter is a family of low‑cost IP cameras sold under various private labels in the Chinese market. They ship with a minimalist web UI that only serves static pages and a single “live view” button. Behind the scenes, the Android companion app communicates with the device using a custom HTTP‑based protocol the author calls BUBBLE. The protocol is not documented anywhere, does not advertise itself via standard service discovery, and lives exclusively on port 80.

Instead of a clean RTSP endpoint, the app issues GET requests to URLs that look like:

/video?stream=main&size=1080p&token=xyz123

Each request returns a multipart MJPEG payload that can be re‑assembled into a video stream. The challenge is that the exact set of query parameters, their ordering, and the token generation algorithm are never disclosed. The only clue is the traffic captured from the Android app, which the author saved as a pcap file and dissected with Wireshark.

Historical context

When these cameras first appeared in 2014‑2016, manufacturers relied on proprietary HTTP APIs to avoid licensing fees for RTSP stacks. The approach worked for a few years, but as the market matured, the lack of standardisation made integration with modern surveillance software impossible. Community projects such as ZoneMinder and Shinobi never added support because the API surface was too opaque.

The author’s effort is part of a broader movement in the DIY surveillance community to breathe new life into legacy hardware. Similar reverse‑engineering projects have targeted Dahua, Hikvision, and even early Axis devices, often resulting in open‑source shim libraries that expose a standard RTSP endpoint.

Key features of the solution

• Pattern Enumeration Engine – a Python service that generates every plausible URL combination based on observed parameter names, default values, and token length constraints.
• Dynamic Token Generator – reproduces the HMAC‑based token algorithm observed in traffic, allowing the service to sign requests without manual intervention.
• BUBBLE‑to‑RTSP Bridge – a thin Docker container that receives the generated URLs, fetches the MJPEG stream, re‑packages it into an RTSP stream using ffmpeg, and publishes it on a configurable port.
• Integration Hooks – the bridge can expose the stream via a standard RTSP URL that Frigate can ingest, or directly push frames into a local WebRTC pipeline for low‑latency viewing.

Pros and cons

Advantage	Description
Full control	You own the entire stack, from packet capture to stream publishing, eliminating reliance on third‑party firmware updates.
Reusability	The pattern engine can be repurposed for other obscure IP cameras that use similar HTTP‑based control channels.
Open‑source	All components are released under an MIT‑style license, encouraging community contributions.
Low cost	No need to replace hardware; the only added cost is compute resources for the bridge container.

Drawback	Description
Maintenance overhead	The reverse‑engineered logic is brittle; firmware updates can break the URL schema, requiring regeneration of patterns.
Security surface	Exposing an HTTP control channel on port 80 can be abused if not properly firewalled.
Latency	The extra transcoding step (MJPEG → RTSP) adds a few hundred milliseconds of delay compared to native RTSP.
Documentation gap	Future engineers must rely on code comments and captured traffic; there is no official spec.

Use cases

• Homelab surveillance – integrate legacy cameras into a Frigate‑based AI detection pipeline.
• Edge‑to‑cloud bridging – forward video frames to a cloud analytics service when native APIs are unavailable.
• Research – study proprietary camera firmware behaviours without flashing custom firmware.

Current state and future trends

The bridge container is now stable enough for daily use in production‑grade homelabs. The author plans to open‑source the pattern engine as a standalone library, enabling other developers to plug it into their own surveillance stacks. Future work includes:

• Automatic schema discovery – using machine‑learning classifiers to predict new URL patterns from traffic bursts.
• TLS hardening – wrapping the control channel in mutual TLS to prevent man‑in‑the‑middle tampering.
• Multi‑camera orchestration – a scheduler that dynamically allocates bandwidth based on motion detection events. —

PREREQUISITES

Before you begin, verify that your environment satisfies the following baseline requirements.

Requirement	Details
Operating System	Ubuntu 22.04 LTS or Debian 12 with kernel 5.15+.
Hardware	Minimum 4 CPU cores, 8 GB RAM, and 200 GB of storage for video buffers.
Network	Ability to capture traffic on the LAN segment where the cameras reside (e.g., via a SPAN port).
Docker	Docker Engine 24.0+ with rootless mode enabled for security.
Python	Python 3.11 with pip and virtualenv support.
ffmpeg	Version 6.0 or newer, compiled with --enable-libx264 and --enable-librtsp.
Frigate	Version 1.36.0 or later, running in a separate Docker compose stack.
Permissions	User must belong to the docker group or have equivalent sudo rights for container management.
Security	Outbound firewall rules must allow TCP 80 to the camera IPs; inbound rules must restrict access to the bridge container’s RTSP port (default 8554).

Software versions (as of writing)

docker --version          # Docker Engine 24.0.7
docker-compose --version  # Docker Compose Plugin v2.24.5
python --version          # Python 3.11.9
ffmpeg -version           # ffmpeg version 6.0

Dependency checklist

1. Capture library – tcpdump or wireshark for pcap extraction. 2. Token library – cryptography Python package for HMAC‑SHA256 operations.
2. HTTP client – requests library for fetching MJPEG streams.
3. ffmpeg – already installed on the host; the bridge container will mount the binary via a volume.

---

INSTALLATION & SETUP

The following steps assume you have already captured a few minutes of traffic from the Android app and identified the base URL pattern. The example below uses a captured URL:

/stream?type=main&quality=high&auth=12345678

1. Clone the repository

git clone https://github.com/usmanmasoodashraf/camera-bridge.git
cd camera-bridge```

### 2. Create a Python virtual environment  

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

3. Generate URL patterns

The core of the reverse‑engineering effort is the PatternGenerator module. It reads a JSON file containing observed parameter names and their possible values, then outputs a combinatorial list of URLs.

# generator.py
import itertools
import json
import re

def load_params(path):
    with open(path) as f:
        return json.load(f)

def build_urls(params):
    keys = list(params.keys())
    values = [params[k] for k in keys]
    for combo in itertools.product(*values):
        query = '&'.join(f"{k}={v}" for k, v in zip(keys, combo))
        yield f"/stream?{query}"

Explanation * load_params() reads a JSON file like params.json where each key maps to a list of possible values.

• build_urls() uses itertools.product to generate every permutation.
• The resulting URLs are written to patterns.txt for downstream processing.

Create params.json with observed values:

{
  "type": ["main", "sub"],
  "quality": ["high", "low", "medium"],
  "auth": ["12345678", "87654321"]
}

Run the generator:

python generator.py```

The script will produce `patterns.txt` containing every plausible URL. In practice, the author observed roughly 100 000 unique permutations, which is why the project’s title mentions “100 000 Url Patterns”.  

### 4. Deploy the bridge container  

The bridge is packaged as a Docker image that bundles the token generator and ffmpeg transcoder. Build it locally:  

```bash
docker build -t camera-bridge:latest .

Dockerfile (excerpt):

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY bridge.py /app/
COPY token.py /app/
COPY ffmpeg.sh /app/

EXPOSE 8554
ENTRYPOINT ["python", "bridge.py"]

bridge.py (simplified):

#!/usr/bin/env python3
import sys
import requests
from token import generate_token
from subprocess import run

PATTERNS = open("patterns.txt").read().splitlines()
CAMERA_IP = sys.argv[1]
RTSP_PORT = int(sys.argv[2]) if len(sys.argv) > 1 else 8554

for pattern in PATTERNS:
    url = f"http://{CAMERA_IP}{pattern}"
    token = generate_token(url)
    full_url = f"{url}&token={token}"
    # Fetch MJPEG stream
    resp = requests.get(full_url, stream=True, timeout=5)
    if resp.status_code == 200:
        # Pipe to ffmpeg for RTSP output
        run([
            "ffmpeg",
            "-i", "-",
            "-c:v", "copy",
            "-f", "rtsp",
            f"rtsp://0.0.0.0:{RTSP_PORT}/stream"
        ], input=resp.raw, check=True)
        break

Explanation of Docker command

When you run the container, you must pass the camera’s IP address and the desired RTSP port as arguments. The container uses $CONTAINER_ID for internal bookkeeping, but the Docker CLI does not expose templating syntax.

docker run -d \
  --name camera_bridge_1 \
  -e CAMERA_IP=192.168.1.45 \
  -p 8554:8554 \
  camera-bridge:latest \
  192.168.1.45 8554

Note: The placeholder $CONTAINER_ID is not used in the command; it only appears in internal logs.

5. Verify the RTSP stream

ffprobe rtsp://localhost:8554/stream

You should see a description of the video stream, confirming that ffmpeg successfully re‑wrapped the MJPEG feed into an RTSP URL.

6. Hook the stream into Frigate

Add the following entry to your Frigate configuration.yaml:

cameras:
  legacy_nvr_1:
    ffmpeg:
      inputs:
        - camera: