Skip to content

Add more specificity to Singularity Hub Pull Names#1059

Closed
vsoch wants to merge 2 commits into
apptainer:developmentfrom
vsoch:development
Closed

Add more specificity to Singularity Hub Pull Names#1059
vsoch wants to merge 2 commits into
apptainer:developmentfrom
vsoch:development

Conversation

@vsoch

@vsoch vsoch commented Oct 15, 2017

Copy link
Copy Markdown
Collaborator

This PR will add more specificity to how users can ask for and name containers. Specifically, since the new version of Singularity Hub allows for multiple container builds associated with a commit and branch, we need to add this level of specificity to the naming of the files. Here is a review:

Old Singularity Hub

The old Singularity Hub used a commit as a version, and provided the hash only as
a special / different preference specified by the user:

singularity pull --hash shub://vsoch/hello-world
Progress |===================================| 100.0% 
Done. Container is at: abba5a1f91b2e98ebe125fc7a5151b33.img

This still works. The hash is indeed the hash of the file:

md5sum abba5a1f91b2e98ebe125fc7a5151b33.img 
abba5a1f91b2e98ebe125fc7a5151b33  abba5a1f91b2e98ebe125fc7a5151b33.img

The hash itself was extracted from the filename in storage. Note that we do better now - it's stored in the database and delivered in the manifest. You can also still pull by commit

singularity pull --commit shub://vsoch/hello-world
Progress |===================================| 100.0% 
Done. Container is at: f9cd90a7b030a9c794bfe76c123da5a518ad9962.simg

and default naming goes to as it was before, only the branch (aka "tag" for Singularity Hub v1.0)

singularity pull shub://vsoch/hello-world
Progress |===================================| 100.0% 
Done. Container is at: vsoch-hello-world-master.simg

Of course we see the issues with this - given two different commands with default naming, we could return different containers (but call them the same thing). For the new version of Singularity Hub, this would produce the same file (but different containers)

singularity pull shub://vsoch/hello-world:apps
singularity pull shub://vsoch/hello-world:latest

Thus, we need to integrate the tag (AND the branch) into the default name. I will describe the changes below.

New Singularity Hub

The new Singularity Hub uses version as the content hash, and commit as commit. As it should have been.

Here is an example pulling by commit

singularity pull --commit shub://doc.fish/vsoch/hello-world
Progress |===================================| 100.0% 
Done. Container is at: e279432e6d3962777bb7b5e8d54f30f4347d867e.simg
singularity pull --hash shub://doc.fish/vsoch/hello-world
Progress |===================================| 100.0% 
Done. Container is at: 865f2d7bc8eb64e454162955d4c35bb2.simg

checking...

md5sum 865f2d7bc8eb64e454162955d4c35bb2.simg 
865f2d7bc8eb64e454162955d4c35bb2  865f2d7bc8eb64e454162955d4c35bb2.simg

Note for default naming belong, we now have a branch and tag.

singularity pull shub://doc.fish/vsoch/hello-world
Progress |===================================| 100.0% 
Done. Container is at: vsoch-hello-world-master-latest.simg

So for our conflict above, the two containers would be appropriately named:

singularity pull shub://vsoch/hello-world:apps --> vsoch-hello-world-master-apps.simg
singularity pull shub://vsoch/hello-world:latest --> vsoch-hello-world-master-latest.simg

I've written about the naming and recommendation to always name by hash here, and will also link this issue there.

Attn: @singularityware-admin

@vsoch vsoch requested review from GodloveD and gmkurtzer October 18, 2017 03:04
@vsoch

vsoch commented Oct 18, 2017

Copy link
Copy Markdown
Collaborator Author

I'm not sure why I don't see the changes I just made - I'm going to close and re-open.

@vsoch

vsoch commented Oct 18, 2017

Copy link
Copy Markdown
Collaborator Author

If anyone is able to close this, please go ahead. For some reason I cannot!

@vsoch vsoch closed this Oct 18, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant