Migrate Iot.Devices.Camera to use new application naming

[mg-yolo-enterprises]

Per https://github.com/raspberrypi/rpicam-apps/blob/main/README.md the libcamera-* name has been updated to rpicam-* and "Users are encouraged to adopt the new application and library names as soon as possible".

libcamera is still working with the current version of applications distributed with Raspberry Pi OS.

Further discussion: https://forums.raspberrypi.com/viewtopic.php?t=359569

[raffaeler]

Thank you for catching this.
I just opened an issue to the relevant repository asking the correct way to detect the presence of the correct tool set.
We must ensure that any upgrade to our library does not break the compatibility of the app.

[naushir]

What about possibly using the rpicam-apps version string? The API function RPiCamAppsVersion() will return a string representing the version number in the form major.minor.patch. You can test for version > 1.4.0 when we switch to rpicam-*. Would that work for you?

[raffaeler]

I am not familiar with the RPiCamAppsVersion() API.

• What library does expose it?
• Is it available in both libcamera and rpi-* OS versions?

From the sources, I see it is a C++ library. This is a problem because C++ does not have a standard ABI. The mangled name of the function may change in future recompilation and break the library. Such APIs should be exposed using the "C" contract which always work.
Is there a "C" binding too?

Thanks

[naushir]

In libcamera-* land, this API function used to be called LibcameraAppsVersion(). So you would have to open up the rpicam-apps.so/libcamera-apps.so, and look for the symbol LibcameraAppsVersion() or RPiCamAppsVersion() with dlsym, which is not ideal. Also, this is using C++ mangling, but I can fix that easily enough to use a C contract.

Instead of this, how about using apt-policy to check the package name:

$ apt policy rpicam-apps 2>/dev/null | grep 'Candidate:' | awk '{print $2}'
1.5.3-1

Would that be more suitable?

[raffaeler]

Unfortunately Libcamera C++ bindings are causing a lot of problems to everyone producing a library in any language.

Running the apt-policy involves running a bash command from the library which may slow down the operatings requesting to open a channel with the camera. I would prefer something different, if possible.

The problem is that this operation may eventually be run many times when someone is using the library to open/close the camera multiple times.
While we can create a sort of detection call at the beginning, this would break our current API.

[naushir]

Perhaps you can explain how you are using the library .so file in more detail? This would help reach a better solution if possible.

[raffaeler]

@naushir sure, let me explain.
In the legacy configuration, we could easily use the libraries to bind .NET (C#) to the .so libraries and invoke the APIs to acquire a photo or a video stream.

When Raspbian transitioned to the libcamera stack, I had a long conversation with Kieran Bingham to understand how to use the Libcamera library. You can see a (long) summary of the problems here: #1875

But let me write a TLDR here:
Libcamera is a C++ library only and there is no "C" binding. This prevents any reliable compilation to make interop from any language. Every time a compilation happens (for example when a Raspbian version is published), a different C++ compiler version may be used and this may produce different mangled names for the C++ exports. This automatically breaks anyone relies on mangled names.

BTW the Python bindings provided by the Raspberry PI foundation do not suffer from this problem because they are generated and shipped along with the OS. If you always recompile the binaries and the interoperability code, you can safely use the mangled names.

To overcome this problem, I created a "binding" that invokes the libcamera apps and capture the stdout/stderr. It's far from being perfect but is a great workaround to avoid the problems described above. I hardcoded the libcamera tool names because I could never image that they could change at any time.

We are not alone. The entire community around the Raspberry PI has been hit by this same problem. There are a lot of libraries that are still recommending to switch to the legacy stack for this reason. But with the latest 64 bits OS versions the legacy stack is not provided anymore and therefore all those libraries are broken.
The renaming of the tools is breaking us now, but I bet we are not alone again.

What do we need (and not just us)?

1. In the short term, we need reliable detection method to understand whether we should run the libcamera or the newer rpicam- tools
2. In the long term, a stable "C" ABI to allow the rest of the world to safely inteop with those libraries
3. A safe stable "C" ABI for the HW H.264 encoder that used to work in the legacy stack and that is no more used by the Libcamera stack (at least this is what Kieran told me).

Thanks

[naushir]

We are not alone. The entire community around the Raspberry PI has been hit by this same problem. There are a lot of libraries that are still recommending to switch to the legacy stack for this reason. But with the latest 64 bits OS versions the legacy stack is not provided anymore and therefore all those libraries are broken.

I won't go into the details as this is been discussed many times over the years, but the legacy stack was a dead end for Raspberry Pi. It's fully closed source (so no 3rd party camera support), extremely difficult to maintain because of closed binary blobs, and would not gain any advanced camera features (e.g. PDAF or HDR). Using libcamera alleviates all these issues and adds significantly more control of the imaging system to our users.

• In the short term, we need reliable detection method to understand whether we should run the libcamera or the newer rpicam- tools

I could declare RPiCamAppsVersion() as extern "C" and you can then use dlsym() to search for the presence of the symbol. If present, assume rpicam-* else use libcamera-*. Would that work?

• In the long term, a stable "C" ABI to allow the rest of the world to safely inteop with those libraries

I know the libcamera development team have plans for C bindings for the API, but cannot comment on timeframes for when this may be available. But rpicam-apps will remain a C++ API even if/when a C API for libcamera is available. We don't have the engineering resources to rework this ourselves.

• A safe stable "C" ABI for the HW H.264 encoder that used to work in the legacy stack and that is no more used by the Libcamera stack (at least this is what Kieran told me).

This is already available as the codecs go through a standard V4L2 memory-to-memory interface. You can see the interface in rpicam-apps here.

[raffaeler]

I won't go into the details as this is been discussed many times over the years

Sure, I just wanted to start from a common acknowledgement. I fully share the reasons for the migration, but my personal opinion is that the choice to have a C++ library only is causing a huge damage to the ecosystem.

I could declare RPiCamAppsVersion() as extern "C" and you can then use dlsym() to search for the presence of the symbol. If present, assume rpicam-* else use libcamera-*. Would that work?

Yes, this would be a great way to identify the correct stack.
Does the older releases of the OS (the one with libcamera) have this library?
If yes, will they get this export too by apt updating the OS?

I know the libcamera development team have plans for C bindings for the API, but cannot comment on timeframes for when this may be available. But rpicam-apps will remain a C++ API even if/when a C API for libcamera is available. We don't have the engineering resources to rework this ourselves.

Kieran told me this as well a long time ago telling also that is not a priority. This means that it will take years before seeing something like that. Among the others, the rust, telephony and robotics communities are also struggling on this, but nothing happened. Libcamera is too vast to think that anyone else from the community will start wrapping the C++ library, not to count on the efforts required to keep it updated over the libcamera future changes.

This is already available as the codecs go through a standard V4L2 memory-to-memory interface. You can see the interface in rpicam-apps here.

I've see that in the past, but the VL42 compatibility layer for libcamera does not work as I expected (I already tried that) and I am not sure how to pipe the video into the encoder to get the H.264 stream out of a libcamera stack.
Also, I am not sure that VL42 works when libcamera has been switched on (my ignorance BTW).
Could you please elaborate on this?

Thank you very much!

[naushir]

Does the older releases of the OS (the one with libcamera) have this library?

The updates will only be available on Bookworm and later. So you will have to do a dlsym to check if the symbol exists.

I've see that in the past, but the VL42 compatibility layer for libcamera does not work as I expected (I already tried that) and I am not sure how to pipe the video into the encoder to get the H.264 stream out of a libcamera stack.
Also, I am not sure that VL42 works when libcamera has been switched on (my ignorance BTW).
Could you please elaborate on this?

There may be some confusion here. libcamera does indeed have a compatibility layer, but my link in the comment above points to a V4L2 H.264 codec API used by rpicam-apps. This is entirely separate and unrelated to the libcamera V4L2 compatibility layer. Note that libcamera only deals with uncompressed image streams. Compression (either still image/JPEG, or video/H.264) happens outside the libcamera domain - in our case at the rpicam-apps application layer.

[raffaeler]

The updates will only be available on Bookworm and later. So you will have to do a dlsym to check if the symbol exists.

Got it:

• looking for the rpicam-apps.so/libcamera-apps.so file and loading it
• dlsym to check for the symbol that you will tell us

At this point, beyond just seeing if the symbol exists, do you see any value in getting the version number? I am currently using the libcamera command line. If anything is different there, I should understand it from the version number, right?

Compression (either still image/JPEG, or video/H.264) happens outside the libcamera domain - in our case at the rpicam-apps application layer.

Right.
Please correct me if this is correct: even with libcamera support, I can still use VL42 interfaces to access the H.264 decoding, without loading any library. Correct?
Does the H.264 code you linked me have any "C" exports so that I can use the encoder without VL42? Would this make any sense?

Thanks!

[naushir]

At this point, beyond just seeing if the symbol exists, do you see any value in getting the version number? I am currently using the libcamera command line. If anything is different there, I should understand it from the version number, right?

The version number is probably not useful for what you want to test.

Please correct me if this is correct: even with libcamera support, I can still use VL42 interfaces to access the H.264 decoding, without loading any library. Correct?

For V4L2 H.264 encoding that's correct, it's completely unrelated to libcamera.

Does the H.264 code you linked me have any "C" exports so that I can use the encoder without VL42? Would this make any sense?

It does not, but it can be used as simple example of how you can drive the video encoder through the V4L2 API. It's not meant to be called externally from outside rpicam-apps.