Behaviour and API for cameras in the BB framework.
A camera is an ordinary BB.Sensor that happens to produce rather more data
than a sensor usually does. It is declared in the robot DSL like any other
sensor, supervised like any other sensor, and bb itself knows nothing about
cameras.
topology do
link :head do
sensor :eye, {BB.Camera.Synthetic, width: 640, height: 480}
end
endFrames do not go on PubSub
Everything else in bb publishes its readings and lets subscribers sort it
out. Cameras cannot: a 1080p MJPEG stream is tens of megabytes a second, and
a fan-out registry with no backpressure turns one slow subscriber into an
unbounded mailbox. So BB.PubSub carries camera lifecycle only - capture
started and ended - and frames travel over the bounded interface
BB.Camera.Hub serves.
The split is not an optimisation. It is what lets a robot keep running when a consumer stalls.
Consuming a camera
Two shapes over the same mechanism. Pull, which suits anything with its own loop - an HTTP route writing chunks, say:
{:ok, session} = BB.Camera.Media.open(MyRobot, :eye, format: :jpeg)
{:ok, frame} = BB.Camera.Media.next(session, 2_000)
:ok = BB.Camera.Media.close(session)And demand, which suits anything with its own notion of backpressure - a Membrane pipeline, or a GenServer that wants frames as messages:
{:ok, sub} = BB.Camera.subscribe(MyRobot, :eye)
:ok = BB.Camera.request(sub)
receive do
{:bb_camera, ref, %BB.Camera.Frame{} = frame} when ref == sub.ref -> frame
endIn both, a request yields exactly one frame, strictly newer than the last one
you were given. A consumer that asks in a loop therefore runs at the camera's
frame rate; one that asks slowly gets the latest frame each time and misses
the ones in between, which for independently decodable frames is lossless for
what survives. That is also why this package refuses interframe codecs
outright - see BB.Camera.Format.
Writing a driver
One module, use BB.Sensor and use BB.Camera. One callback of your own,
then two messages from the hub to answer. BB.Camera.Synthetic is a
complete worked example; the short version is:
defmodule MyCamera do
use BB.Sensor, options_schema: [...]
use BB.Camera
@impl BB.Camera
def stream_info(opts) do
BB.Camera.StreamInfo.new(format: :jpeg, width: opts[:width], height: opts[:height])
end
@impl BB.Sensor
def init(opts) do
{:ok, info} = stream_info(opts)
{:ok, hub} = BB.Camera.Hub.start_link(bb: opts[:bb], stream_info: info)
{:ok, %{bb: opts[:bb], hub: hub}}
end
@impl BB.Sensor
def handle_info({:bb_camera, :start_capture}, state), do: ...
def handle_info({:bb_camera, :stop_capture}, state), do: ...
endA camera is configured like any other component: through its
BB.Sensor.options_schema/0, which the robot's DSL can fill from
parameters with param/1. Changing one of those parameters at runtime
re-validates the options and calls BB.Sensor.handle_options/2, which is
where a driver reconfigures the device.
Nothing in this interface assumes where frames come from. A driver reading a device through a subprocess, one calling a platform capture API from a NIF, and one synthesising frames all meet it identically.
Summary
Callbacks
What this camera will produce, derived from its declared options alone.
Functions
Withdraw an outstanding request.
Every camera robot declares, whether or not it is running.
The hub serving a camera, if it's running.
Whether frames in format decode without reference to each other.
Ask for one frame.
What a camera is producing.
Become a consumer of a camera's capture.
Stop consuming. Capture stops when the last subscription goes.
Callbacks
@callback stream_info(opts :: keyword()) :: {:ok, BB.Camera.StreamInfo.t()} | {:error, struct()}
What this camera will produce, derived from its declared options alone.
This must be pure. It is called during discovery, against robots that are not running, with the options exactly as written in the DSL - parameter references unresolved, hardware unopened, possibly absent. Describe what was asked for.
What the camera turns out to actually produce is reported at runtime through
BB.Camera.Hub.capture_started/2 or BB.Camera.Hub.update_stream_info/2,
and that value is what consumers see once it exists.
Return {:error, %BB.Error.Invalid.Camera.UnknownStreamInfo{}} when the
options genuinely cannot imply a format; discovery then reports
stream_info: nil rather than inventing one.
Functions
@spec cancel_request(BB.Camera.Subscription.t()) :: :ok | {:error, struct()}
Withdraw an outstanding request.
Leaves the subscription in place. Use it when you've stopped waiting - after a timeout, say - so the hub isn't holding demand nothing will collect.
@spec discover(module(), keyword()) :: [BB.Camera.Info.t()]
Every camera robot declares, whether or not it is running.
Cameras that are declared but have no live hub are reported with
status: :declared rather than omitted.
Options
:status- keep only cameras in this state (:capturing,:runningor:declared). Defaults to all of them.
The hub serving a camera, if it's running.
Whether frames in format decode without reference to each other.
The predicate this package's format support is gated on. See
BB.Camera.Format.
@spec request(BB.Camera.Subscription.t()) :: :ok | {:error, struct()}
Ask for one frame.
You will be sent {:bb_camera, subscription.ref, %BB.Camera.Frame{}} once a
frame strictly newer than your last one exists - immediately if one already
does.
At most one request is outstanding at a time; asking again before delivery changes nothing. Demand does not accumulate, so a consumer that stops asking simply stops receiving, and the camera's memory use doesn't depend on how long it takes to come back.
These also arrive on a subscription, regardless of outstanding demand:
{:bb_camera, ref, {:stream_info, info}}immediately before the first frame a changed format describes.{:bb_camera, ref, :end_of_capture}when capture stopped cleanly.{:bb_camera, ref, {:capture_failed, error}}when it failed.
@spec stream_info(module(), atom()) :: {:ok, BB.Camera.StreamInfo.t()} | {:error, struct()}
What a camera is producing.
The live format when the camera is running, the declared one when it isn't.
@spec subscribe(module(), atom(), keyword()) :: {:ok, BB.Camera.Subscription.t()} | {:error, struct()}
Become a consumer of a camera's capture.
Subscribing delivers nothing by itself - call request/1 for each frame you
want. It does start capture if nobody else was watching, and the camera stops
capturing when the last subscription goes.
Options
:format- assert the camera produces this format, failing loudly if it doesn't. A consumer that can only handle JPEG should say so here rather than discover it from the bytes.:owner- the process frames are sent to, defaulting to the caller. The hub monitors it and unsubscribes automatically when it dies, so a crashed consumer releases the capture rather than pinning it.
The returned subscription's monitor watches the hub and belongs to the
calling process, which is not necessarily :owner. Subscribing on one
process's behalf from another is supported - the frames go where you said -
but the owner won't learn of a dead camera from a monitor it doesn't hold.
Subscribe from the process that will consume unless you have a reason not to.
@spec unsubscribe(BB.Camera.Subscription.t()) :: :ok
Stop consuming. Capture stops when the last subscription goes.