Buildstream#4149
Conversation
|
I'm seeing the following error when buildstream tries to call bubblewrap to run a build |
|
@abderrahim Yeah I ran into that and various other errors when I was still using bubblewrap in mkosi as well. I consider it to be a totally broken check in bubblewrap. It should not fail when running unprivileged with capabilities, it's a totally valid use case. But the upstream is pretty much dead unfortunately. We can avoid the check by pretending we're root (while still setting HOME to the user home directory to reuse the cache). I'll push an update to that effect, please try again then. |
7519755 to
0496128
Compare
|
Hi, I was asked to look into this and give it a go as I have been looking into working on something very similar. 👍 I was wondering what the ultimate goal is here. As I understand in here we are only changing the need for having a script such as: into just calling It also occurs to me this means we are relying on |
In a way, maybe. https://gitlab.com/freedesktop-sdk/mkbstosi was an attempt at such a script. Ultimately, it would be nice to have things directly in mkosi and not have to install a third project. There are also a few complications with trying to call mkosi on a directory structure.
Do you often need to call Either way, if it's something that's useful I guess it can be added.
Yeah, mkosi would be another dependency in addition to buildstream.
That would be nice, but
So in a way, it's a trade off and we (GNOME OS) are pretty happy with it. I thought it might be useful to you too :) |
Since we don't require the version to be in the kernel image filename anymore, let's not insist on the - anymore.
In weird CI setups or such, there might be stuff in non-canonical directories in /. This stuff only has a minimal chance to affect the image build accidentally, so let's mount them in by default in case the image build actually does need to look up stuff in these directories.
The GnomeOS folks are looking into mkosi to build their images instead of BuildStream. While BuildStream will still take care of providing the rootfs tree, mkosi would take over the responsibility of packaging that directory tree into a disk image. Let's add support for BuildStream to mkosi to make this possible. Unlike the other supported distributions, BuildStream is not intended to be consumed by installing individual packages. Instead, BuildStream elements should be exposed which provide the full rootfs that should go into the image. That's why we limit the number of packages that can be specified to a single one, which should always provide all contents that should go into the image.
0496128 to
6744421
Compare
|
@daandemeyer I'm noticing a weird behaviour with this MR. When I set |
| ) -> None: | ||
| options = [ | ||
| "--same-dir", | ||
| *context.rootoptions(), | ||
| # bst might need to lookup files/paths across the user's home directory so make sure it is |
There was a problem hiding this comment.
| ) -> None: | |
| options = [ | |
| "--same-dir", | |
| *context.rootoptions(), | |
| # bst might need to lookup files/paths across the user's home directory so make sure it is | |
| ) -> None: | |
| options = [ | |
| "--same-dir", | |
| *context.rootoptions(), | |
| # we are passing --same-dir. Ensure the current working directory to make chdir() reliable. | |
| "--bind", Path.cwd(), Path.cwd(), | |
| # bst might need to lookup files/paths across the user's home directory so make sure it is |
I needed this to run it properly, I'm running it from distrobox and it does weird things to the home directory. Regardless, passing the directory we run it from seems sound.
| [Content] | ||
| # BuildStream is a generic distribution so we don't know which package to install, hence override to the | ||
| # empty list. | ||
| Packages= |
There was a problem hiding this comment.
I found the issue: this file gets loaded last (after mkosi-initrd/mkosi.conf and after the user provided mkosi.conf and mkosi.initrd.conf) so it overrides the packages to the empty list.
(as I workaround I removed this line, and the Packages list from mkosi-initrd/mkosi.conf, which obviously breaks other distros)
There was a problem hiding this comment.
Following the suggestion from @behrmann on matrix, I made a fix
Details
diff --git a/mkosi/resources/mkosi-initrd/mkosi.conf b/mkosi/resources/mkosi-initrd/mkosi.conf
index ab76f73a..ca1afddb 100644
--- a/mkosi/resources/mkosi-initrd/mkosi.conf
+++ b/mkosi/resources/mkosi-initrd/mkosi.conf
@@ -10,12 +10,6 @@ SplitArtifacts=
Bootable=no
MakeInitrd=yes
CleanPackageMetadata=yes
-Packages=
- systemd # sine qua non
- udev
- bash # for emergency logins
- less # this makes 'systemctl' much nicer to use ;)
- gzip # For compressed keymap unpacking by loadkeys
RemoveFiles=
# we don't need this after the binary catalogs have been built
diff --git a/mkosi/resources/mkosi-initrd/mkosi.conf.d/buildstream.conf b/mkosi/resources/mkosi-initrd/mkosi.conf.d/buildstream.conf
deleted file mode 100644
index 1e76355e..00000000
--- a/mkosi/resources/mkosi-initrd/mkosi.conf.d/buildstream.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-# SPDX-License-Identifier: LGPL-2.1-or-later
-
-[Match]
-Distribution=buildstream
-
-[Content]
-# BuildStream is a generic distribution so we don't know which package to install, hence override to the
-# empty list.
-Packages=
diff --git a/mkosi/resources/mkosi-initrd/mkosi.conf.d/default-packages.conf b/mkosi/resources/mkosi-initrd/mkosi.conf.d/default-packages.conf
new file mode 100644
index 00000000..5dc12408
--- /dev/null
+++ b/mkosi/resources/mkosi-initrd/mkosi.conf.d/default-packages.conf
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: LGPL-2.1-or-later
+
+[Match]
+# BuildStream is a generic distribution so we don't know which package to install.
+Distribution=!buildstream
+
+Packages=
+ systemd # sine qua non
+ udev
+ bash # for emergency logins
+ less # this makes 'systemctl' much nicer to use ;)
+ gzip # For compressed keymap unpacking by loadkeys
diff --git a/mkosi/resources/mkosi-tools/mkosi.conf b/mkosi/resources/mkosi-tools/mkosi.conf
index b597d0a0..5ccf8f46 100644
--- a/mkosi/resources/mkosi-tools/mkosi.conf
+++ b/mkosi/resources/mkosi-tools/mkosi.conf
@@ -2,22 +2,3 @@
[Output]
Output=mkosi.tools
-
-[Content]
-Packages=
- bash
- ca-certificates
- coreutils
- cpio
- curl
- dosfstools
- e2fsprogs
- keyutils
- kmod
- mtools
- opensc
- openssl
- systemd
- tar
- xfsprogs
- zstd
diff --git a/mkosi/resources/mkosi-tools/mkosi.conf.d/buildstream.conf b/mkosi/resources/mkosi-tools/mkosi.conf.d/buildstream.conf
deleted file mode 100644
index 1e76355e..00000000
--- a/mkosi/resources/mkosi-tools/mkosi.conf.d/buildstream.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-# SPDX-License-Identifier: LGPL-2.1-or-later
-
-[Match]
-Distribution=buildstream
-
-[Content]
-# BuildStream is a generic distribution so we don't know which package to install, hence override to the
-# empty list.
-Packages=
diff --git a/mkosi/resources/mkosi-tools/mkosi.conf.d/default-packages.conf b/mkosi/resources/mkosi-tools/mkosi.conf.d/default-packages.conf
new file mode 100644
index 00000000..e20e4bd4
--- /dev/null
+++ b/mkosi/resources/mkosi-tools/mkosi.conf.d/default-packages.conf
@@ -0,0 +1,22 @@
+[Match]
+# BuildStream is a generic distribution so we don't know which package to install.
+Distribution=!buildstream
+
+[Content]
+Packages=
+ bash
+ ca-certificates
+ coreutils
+ cpio
+ curl
+ dosfstools
+ e2fsprogs
+ keyutils
+ kmod
+ mtools
+ opensc
+ openssl
+ systemd
+ tar
+ xfsprogs
+ zstd
There was a problem hiding this comment.
If that works for you, that looks like a reasonable workaround to me, though that empty Packages= doesn't reset the package list to empty, seems like a bug in our config parsing that needs investigating.
|
For the record, I have an example using this MR in freedesktop-sdk: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/merge_requests/31101 It'll fail with the tip of this branch (as commented earlier) but the patch #4149 (comment) makes it work. |
Ah, thank you for this link! Looking at that and rereading your comment this finally clicked for me! Empty |
It is indeed intended behaviour that it overrides the packages defined in mkosi itself, but I don't think it's intended behaviour that it overrides the packages defined in the user's mkosi.conf.
It is a buildstream element that I added in the same MR.
The implementation in this PR adds BuildStream as a distro / package manager, so that we can use BuildStream elements as packages in mkosi. Looking at |
Ah, I wasn't entirely sure about the implementation here. So a
Well here it actually is. This is the input for
Sorry for being a bit dense. Misunderstanding stuff while travelling was a bit too easy, it seems. |
Correct.
Yeah, I understand that there is a mismatch here. However, if we're trying to fit buildstream in a distro-package-manager-shaped hole, we should be prepared to cut some corners to make it fit. Also the comment seemed to imply (at least to me) that we want to start with a clean slate and allow the user to set their packages, rather than preventing the user from using packages at all. btw, this also affects the tools tree. On GNOME OS, I don't have access to any other package manager (and none of them seemed to be a
This could certainly work, but I feel it would lose features that we already have for other distros / package managers.
I think that would be kinda weird, as each buildstream project is different. But ultimately, it would be pretty similar to:
Having no package by default and letting the user set their own wouldn't be very different from having a well-known package name. @daandemeyer @AdrianVovk: opinions? |
|
I looked at the code some more and we are running into an edge case of the config parsing. In That doesn't jive with the semantics of empty keys just resetting to their value to the empty list (for list-valued things), if some package is supposed to be installed. I think either way to circumvent this edge case in our logic would be fine:
The latter would be just a shorthand for the lists we use for the package-based distributions. That being said, not having to reset the list and thus having a generally additive semantic, as in the patch you linked above from our prior discussion on Matrix, seems preferable to me, since it's more flexible and should allow adding more buildstream inputs if necessary instead of having a single defined point. Ultimately I know too little of buildstream to say whether this is actually preferable for buildstream. |
4 participants
This will stay in draft until gnomeos has the necessary elements in place to allow properly integrating with mkosi so I can test this end-to-end.
cc @AdrianVovk