Generating a list of non-uploaded dependencies to build to pass to Carthage

[erichoracek]

Rome looks like an awesome tool! Quick question about its usage:

In the example workflow, it would appear that there is an assumption that the entire contents of the current Cartfile.resolved will either be fully available for download or not. However, in the case where you have internal frameworks that are constantly being updated, it could likely be the case that all frameworks but the one you just updated are available in your s3 cache.

Would it be possible to have a command like the following to allow building only the dependencies that are currently uncached? e.g.:

rome download && carthage build `rome unavailable`

Say for example that APIClient was the only library that was uncached. This would evaluate to:

rome download && carthage build APIClient

Let me know if this is already possible or isn't needed for another reason. Thanks!

[tmspzz]

Hi, this is an interesting addition and totally possible. I will look into it.

There is no assumption that all the contents are on the S3 cache as downloading a frameworks package can fail and rome will continue trying the next one in the list.

[tmspzz]

Opened branch https://github.com/blender/Rome/tree/feature/list

Proposed command and modifiers:

$ rome list
Alamofire v3.2.1 ✔︎
APIClient c07a98aa38a9be8d47104f4e8c3dea75ec652368 ✘
Swiftz v0.0.5 ✘

$ rome list --missing
APIClient Swiftz

$ rome list --present
Alamofire

[erichoracek]

Awesome, thanks! A little more context on our setup:

We'd like our CI machine to be the sole producer of all precompiled binaries. This prevents developers from uploading a bad binary that could potentially be used in a production build (e.g. from a beta Xcode version, an in-progress artifact from development on an internal dependency etc.). As such, whenever a developer does an update locally, they'll ideally be able to pull down the cached dependencies and then build the remaining ones locally. When doing so, we'd want only rebuild the necessary dependencies so that our local carthage bootstrap doesn't take any longer than it needs to.

Once the developer integrates their changes into master, we'd like the CI machine to build any new frameworks and then upload them to the cache so that it's available to all subsequent developers.

We're currently accomplishing this in part with https://github.com/guidomb/carthage_cache, but unfortunately its cache is keyed by the entirety of the Cartfile.resolved, so in the case of a cache miss you're forced to do a full carthage bootstrap, which can be quite time consuming.

Thinking a little more about it—we'd need one additional piece of functionality: a command that is able to evaluate whether or not all dependencies are available in the cache to prevent invoking carthage build with no parameters (in the case of all dependencies being available). It would likely look something like:

rome download && rome is-cache-complete || carthage build `rome unavailable`

Unfortunately I don't have any familiarity with Haskell, or else I'd be willing to take a shot at implementing this.

[erichoracek]

Ah—didn't see that you replied while I was typing out this response—it should be totally possible to implement the rome is-cache-complete command in terms of rome list --missing. It should just be a matter of parsing whether or not there is not any output.

Thanks again for taking a shot at this!

[tmspzz]

You will probably need to write a small script to do so. If xargs on mac supported --no-run-if-empty flag it would have worked out of the box.

[tmspzz]

@erichoracek please check if this pre-release meets your needs https://github.com/blender/Rome/releases/tag/v0.3.0.1

[tmspzz]

Updated pre-release: https://github.com/blender/Rome/releases/tag/v0.3.0.2

[erichoracek]

This seems to be working as intended in this release as well. Thanks!