BB.Camera.FFmpeg.Jpeg (bb_camera_ffmpeg v0.1.0)

Copy Markdown View Source

Structural scanning of a JPEG byte stream.

An MJPEG stream is a concatenation of complete JPEG images, so the obvious framing strategy is to split on the end-of-image marker FF D9. That is wrong in two directions and both of them happen in practice:

  • FF D9 appears inside an EXIF thumbnail, which is a whole JPEG embedded in an APP1 segment. Plenty of UVC cameras emit one, and -c:v copy passes it through untouched.
  • A truncated frame — the device was unplugged mid-write, the pipe was cut — leaves a partial image that never terminates, and a splitter will happily glue it to the front of the next one.

So this module walks the marker structure instead: segments are skipped by their declared length, and entropy-coded data is scanned for the next real marker, skipping stuffed FF 00 pairs, fill bytes and RST markers the way the format requires. A frame ends at the EOI that the structure says is the end, not at the first byte pair that looks like one.

Walking the structure also means the frame header is parsed on the way past, so info/0 carries the image's actual dimensions and coding process, read from its SOF segment. That matters because a camera is free to hand back something other than what it was asked for, and the only way to report the truth about a frame is to read it out of the frame.

Resumable scanning

Capture delivers a JPEG across several reads, so scan/3 is written to resume. It returns a state/0 marking how far it got, and the caller hands that back with more bytes appended. A frame is therefore scanned once in total rather than once per read, which keeps framing linear in the size of the stream rather than quadratic.

BB.Camera.FFmpeg.Framer wraps this with the buffer and the size caps; callers reading a stream want that rather than this.

Summary

Types

How far a scan got, to be handed back to scan/3 once more bytes arrive.

Functions

The scan state for a buffer that has not been looked at yet.

Find the next start-of-image marker at or after from.

Scan buffer for the end of the JPEG that starts at its first byte.

Does this buffer begin with a start-of-image marker?

Types

error()

@type error() ::
  :not_jpeg
  | {:expected_marker, non_neg_integer()}
  | {:unexpected_soi, non_neg_integer()}
  | {:bad_segment_length, non_neg_integer()}
  | {:truncated_frame_header, non_neg_integer()}

info()

@type info() :: BB.Camera.FFmpeg.Jpeg.Info.t()

state()

@type state() :: {:marker, non_neg_integer()} | {:entropy, non_neg_integer()}

How far a scan got, to be handed back to scan/3 once more bytes arrive.

{:marker, offset} means the next thing in the stream is a marker starting at offset; {:entropy, offset} means the scan is inside entropy-coded data and should resume looking for a marker at offset.

Functions

initial_state()

@spec initial_state() :: state()

The scan state for a buffer that has not been looked at yet.

The two bytes of the SOI marker are consumed up front, so callers must check for it with soi?/1 first.

next_soi(buffer, from)

@spec next_soi(binary(), non_neg_integer()) :: {:ok, non_neg_integer()} | :none

Find the next start-of-image marker at or after from.

Used to resynchronise after a malformed frame: everything before the returned offset is unparseable and should be dropped.

iex> BB.Camera.FFmpeg.Jpeg.next_soi(<<0x00, 0x00, 0xFF, 0xD8, 0x01>>, 0)
{:ok, 2}

iex> BB.Camera.FFmpeg.Jpeg.next_soi(<<0xFF, 0xD8, 0xFF, 0xD8>>, 1)
{:ok, 2}

iex> BB.Camera.FFmpeg.Jpeg.next_soi(<<0x00, 0x01, 0x02>>, 0)
:none

scan(buffer, state, info \\ nil)

@spec scan(binary(), state(), info() | nil) ::
  {:complete, pos_integer(), info() | nil}
  | {:incomplete, state(), info() | nil}
  | {:error, error()}

Scan buffer for the end of the JPEG that starts at its first byte.

state is initial_state/0 for a fresh buffer, or the state returned by a previous :incomplete. info is likewise threaded through: it is nil until the scan walks past the frame header, and non-nil from then on.

Returns one of:

  • {:complete, length, info} — the first length bytes of buffer are one whole JPEG. Anything after that belongs to the next image.
  • {:incomplete, state, info} — the image runs past the end of the buffer.
  • {:error, reason} — the bytes are not a well-formed JPEG, and the caller should resynchronise rather than trust any of them.

soi?(arg1)

@spec soi?(binary()) :: boolean()

Does this buffer begin with a start-of-image marker?

false for a buffer shorter than the marker, which is the "not yet, keep reading" answer as much as it is the "this is not a JPEG" one — callers with a byte or less in hand should wait for more before concluding anything.

iex> BB.Camera.FFmpeg.Jpeg.soi?(<<0xFF, 0xD8, 0x00>>)
true

iex> BB.Camera.FFmpeg.Jpeg.soi?(<<0x00, 0xFF, 0xD8>>)
false

iex> BB.Camera.FFmpeg.Jpeg.soi?(<<0xFF>>)
false