Structural scanning of a JPEG byte stream.
An MJPEG stream is a concatenation of complete JPEG images, so the obvious
framing strategy is to split on the end-of-image marker FF D9. That is
wrong in two directions and both of them happen in practice:
FF D9appears inside an EXIF thumbnail, which is a whole JPEG embedded in anAPP1segment. Plenty of UVC cameras emit one, and-c:v copypasses it through untouched.- A truncated frame — the device was unplugged mid-write, the pipe was cut — leaves a partial image that never terminates, and a splitter will happily glue it to the front of the next one.
So this module walks the marker structure instead: segments are skipped by
their declared length, and entropy-coded data is scanned for the next real
marker, skipping stuffed FF 00 pairs, fill bytes and RST markers the way
the format requires. A frame ends at the EOI that the structure says is the
end, not at the first byte pair that looks like one.
Walking the structure also means the frame header is parsed on the way past,
so info/0 carries the image's actual dimensions and coding process,
read from its SOF segment. That matters because a camera is free to hand
back something other than what it was asked for, and the only way to report
the truth about a frame is to read it out of the frame.
Resumable scanning
Capture delivers a JPEG across several reads, so scan/3 is written to
resume. It returns a state/0 marking how far it got, and the caller hands
that back with more bytes appended. A frame is therefore scanned once in
total rather than once per read, which keeps framing linear in the size of
the stream rather than quadratic.
BB.Camera.FFmpeg.Framer wraps this with the buffer and the size caps; callers
reading a stream want that rather than this.
Summary
Functions
The scan state for a buffer that has not been looked at yet.
Find the next start-of-image marker at or after from.
Scan buffer for the end of the JPEG that starts at its first byte.
Does this buffer begin with a start-of-image marker?
Types
@type error() :: :not_jpeg | {:expected_marker, non_neg_integer()} | {:unexpected_soi, non_neg_integer()} | {:bad_segment_length, non_neg_integer()} | {:truncated_frame_header, non_neg_integer()}
@type info() :: BB.Camera.FFmpeg.Jpeg.Info.t()
@type state() :: {:marker, non_neg_integer()} | {:entropy, non_neg_integer()}
How far a scan got, to be handed back to scan/3 once more bytes arrive.
{:marker, offset} means the next thing in the stream is a marker starting
at offset; {:entropy, offset} means the scan is inside entropy-coded data
and should resume looking for a marker at offset.
Functions
@spec initial_state() :: state()
The scan state for a buffer that has not been looked at yet.
The two bytes of the SOI marker are consumed up front, so callers must
check for it with soi?/1 first.
@spec next_soi(binary(), non_neg_integer()) :: {:ok, non_neg_integer()} | :none
Find the next start-of-image marker at or after from.
Used to resynchronise after a malformed frame: everything before the returned offset is unparseable and should be dropped.
iex> BB.Camera.FFmpeg.Jpeg.next_soi(<<0x00, 0x00, 0xFF, 0xD8, 0x01>>, 0)
{:ok, 2}
iex> BB.Camera.FFmpeg.Jpeg.next_soi(<<0xFF, 0xD8, 0xFF, 0xD8>>, 1)
{:ok, 2}
iex> BB.Camera.FFmpeg.Jpeg.next_soi(<<0x00, 0x01, 0x02>>, 0)
:none
@spec scan(binary(), state(), info() | nil) :: {:complete, pos_integer(), info() | nil} | {:incomplete, state(), info() | nil} | {:error, error()}
Scan buffer for the end of the JPEG that starts at its first byte.
state is initial_state/0 for a fresh buffer, or the state returned by a
previous :incomplete. info is likewise threaded through: it is nil
until the scan walks past the frame header, and non-nil from then on.
Returns one of:
{:complete, length, info}— the firstlengthbytes ofbufferare one whole JPEG. Anything after that belongs to the next image.{:incomplete, state, info}— the image runs past the end of the buffer.{:error, reason}— the bytes are not a well-formed JPEG, and the caller should resynchronise rather than trust any of them.
Does this buffer begin with a start-of-image marker?
false for a buffer shorter than the marker, which is the "not yet, keep
reading" answer as much as it is the "this is not a JPEG" one — callers with
a byte or less in hand should wait for more before concluding anything.
iex> BB.Camera.FFmpeg.Jpeg.soi?(<<0xFF, 0xD8, 0x00>>)
true
iex> BB.Camera.FFmpeg.Jpeg.soi?(<<0x00, 0xFF, 0xD8>>)
false
iex> BB.Camera.FFmpeg.Jpeg.soi?(<<0xFF>>)
false