Stop rounding up individual jobs to the nearest minute for billing

[splitice]

A good well structured deployment script in Github Actions can involve many quick jobs. In our case we are aproaching 16 (typical execution time of 4-6 seconds), it's a bit unreasonable to be paying for 16 minutes per job for a total of just over 1 minute of CPU time.

This encourages us to create larger monolithic worflows rather than clean and re-usable ones. If per minute rounding was instead applied on the entire workflow this would encourage correct usage and enable more use cases.

I understand the need to make money, just wish you could be fairer with the distribution of costs.

[ethomson]

Hi @splitice, I understand where you're coming from - certainly the current setup for GitHub Actions workflows is optimized for longer-running jobs. But I'm not sure that multiple jobs is more "correct" unless you actually need a separate virtual machine for each step.

I'd love to know a bit more about what a well-structured deployment script looks like to you... is this something that you're willing to share? If not, I'd love to get the highlights - what are these jobs doing? Philosophically, why do you think of them as separate jobs instead of separate steps within a job? How are you sharing data between them (or are you not)? Are you using jobs as an isolation environment? If so, could you use containers as that isolation environment or do you require really strict isolation? Is it about the parallelization of jobs?

Regarding the costs, it's important to remember that each job runs on a separate virtual machine, and each new virtual machine takes time to set up and tear down. We keep runners in the set-up state so that your jobs can start on them as quickly as possible. We only bill for the time that your job is actually executing on our runners, and the remainder of the time we retain as costs.

[splitice]

@ethomson I can't share the workflow of the job that inspired this however I can share s screenshot of a similar one. It's a CD job for an In Development application (and it's already at 12 jobs - expected to be over 30 when complete).

Could I make these into a single job? Probably but not without sacrificing alot of the provided functionality. For example:

• Would it take longer per deploy (likely with sequential execution of dependencies or stages)? certainly
• Would it be more monolithic and less portable? certainly
• Would it be more difficult to troubleshoot, less visual and require more engineer effort to understand? certainly

I understand you have to spin up / spin down runners (unfortunately for workloads like this an overkill) but it doesnt change my opinion regarding how billing rounding should be implemented. A change like this would allow people to really put the runner side ("if", matrix builds, etc) to work for lightweight CD jobs rather than encouraging designing to minimising cost (expecially long term as the workflow grows).

To directly answer your questions:

what are these jobs doing?

Continuous Deployment for staging / production environments. Sometimes to one target, sometimes we do matrix builds on top of these jobs as well (e.g to deploy to multiple datacenters).

Philosophically, why do you think of them as separate jobs instead of separate steps within a job?

Non-linear dependencies, different tooling requirements and (in some cases) sometimes different destination clusters (e.g some services are k8s, some bare metal or not software related).

How are you sharing data between them (or are you not)?

Not applicable for this example each step is stateless.

Are you using jobs as an isolation environment?

Not particularly. Without isolation some of the jobs would be 1-2s faster (as most jobs have a small amount of overhead for initialization of tooling / environment) however that's "fast enough" as is.

If so, could you use containers as that isolation environment or do you require really strict isolation?

Certainly containers would be more than sufficient

Is it about the parallelization of jobs?

Parallel execution when conditions are met is necessary to ensure timely execution of complex jobs. Although these jobs require little CPU (at-least when compared to serious unit tests) they do require authentication and communication with remote services with the latencies inherent with such actions (e.g comparing of service versions or pulling a docker image).

In situations like this linear execution of jobs would be significantly slower and less optimal. The example I provided would take ~2:30min vs ~1 minute at it's current size.

[ethomson]

I really appreciate your detailed feedback. This is really useful to understand for our planning.

Having a runner for each of your jobs is indeed overkill in this scenario. Right now a job is affined to a runner, and a runner is expensive, which doesn't really fit in well with what you're trying to do here. We've talked a bit about how to support these sorts of lighter-weight jobs, and but we don't have anything to announce yet.

[maxisam]

Can we at least use 30s instead of a minute? It doesn't address the issue completely but at least it is not as bad.

[bencodner]

I am having the same issue. I created a matrix job in a pipeline that packages umbrella helm charts (around 90). Each matrix takes roughly 30 seconds but I'll be changing this to a script because this matrix approach is using too many minutes.

[nhuray]

I might understand that Github actions billable time was rounded to the minute 3 years ago when the service was still young, but 3 years later I just don't understand why Github does not charge a fair price based on real usage.

AWS for instance is now charging to the milliseconds for Lambda. I understand it's not the same architecture but paying 10 minutes for 10 jobs running 5 sec seems to me a non-sense.

[shealavington]

I've just experienced a very similar thing — Now moving into 2023 🤦‍♂️

I run two jobs, one for Javascript Linting, and one for PHP Linting... In total, it performs on average 30 seconds of work between them, but I'm charged 2 minutes every time 👀 It's on the small scale, but I can see this piling up over time. That's a time charge of 4x my actual usage.

Seems silly for me to crunch this into a single Linting job, because then I also can't keep track of which job/task is taking longer if runtime was to start to spike and they also have different requirements to run.

Below is another part of my workflow, it's a task for compiling assets, ready to cache the data for subsequent jobs that may or may not need to run. The Testing jobs run parallel because, well, they're testing jobs and either may or may not be needed.

In this setup I'm charged 3 minutes for less than a minutes work! Excuse me too for trying to be efficient and only having my tests run when they're needed. Instead of this nice workflow, I'll now be forced to always run at least my unit test suite, and design the job to handle caching of assets for any other jobs. At least then I can shave a guaranteed minute off every time.

It's just not as clean, I suppose I'll save 7 seconds doing so, but I'm forced to always run my tests even if not needed now.

Surely Github should have an efficient way of containerizing for a few seconds and only charging me for actual usage by now?

[dataplor-mike]

Came across this thread while trying to figure out how 11s resulted in a 54x charge of 10 minutes. Not sure if it's related to rounding or if it's a bug that overcharges for reruns or unmet conditions:

[james-callahan]

We hit this issue fairly hard, e.g.

--> in many circumstances we're getting charged over 60x more than we would be otherwise.

I'd love to know a bit more about what a well-structured deployment script looks like to you... is this something that you're willing to share? If not, I'd love to get the highlights - what are these jobs doing? Philosophically, why do you think of them as separate jobs instead of separate steps within a job?

@ethomson in our case we have many jobs as they are generated from a matrix (one for each folder in the repository).

[thomasschaller1]

We hit the same issue. We are evaluating different solutions, and this issue does not help using concurrency. Because instead of just building something, and then running multiple job with the result of the build, we will have to put everything in the same job to reduce our costs, and this will double the time of the whole pipeline (1min vs 2min).

In 2023, we can expect to have something at the seconds level, when you see that AWS is doing it even in the milliseconds level.

[sigi-tw]

How 'un'funny: just splitted up my workflow in a pipeline with sub jobs (github docu also says its a good way to reuse workflows) and now i'm discovering this specific issue.

The main reason why i splitted it up was because i have two images which need to be build and successful and then i need to do an e2e test.

[dhouglasoliveiraatpn]

Started on use of matrix stragegy to reuse jobs, but with this in mind, is a NO-GO!

[ollytheninja]

Reason number 1 why GitHub likes matrix jobs so much!

[piotrekkr]

Yeah they like matrix so much that they cannot fix matrix job outputs so people can actually use them in some sane way.

[olitomlinson]

Yeah this rounding-up billing model is really anti-customer. Do better.

[dataplor-mike]

5 months later and still overbilling

[gtirloni]

I set up a simple workflow to check stale issues and PRs and it runs for 4 seconds. I'm billed for the full minute. Should I hide this check in some other workflow instead?

It was easy to create the workflow that I wanted that 100% solved my needs but now I'm having to re-think how I organize this because of billing issues that I shouldn't have to think about in such simple scenario, IMHO.

[alexking]

This is frustrating because by splitting up jobs to be more efficient and take less overall time, I'm actually using more billable minutes?

[jcmv2019]

Let's also take note that aside from overbilling customers with inaccurate billable minutes, GitHub is also applying minutes multiplier to Windows and MacOS Docker machines.

x2 for Windows and x10 for MacOS (seriously?!) It would have been understandable if MacOS' multiplier is just x5, but x10? That's way too aggressive for a service that can't accurately bill its users for the exact time it was used.

[nanderv]

Makes sense, because github runs on actual apple hardware, which requires a lot more effort to set up compared to standardised servers. Apple just doesn't support building runners, so it becomes awkward for github to do.

[mukulseth]

I just came across this, and really surprised this still hasn't been addressed in any way

[localota-travel]

Yep, we just got set up using this system and got billed a whole bunch of minutes we didn't use, for processes that took just a few seconds each. Looks like we won't be expanding on this system until this is addressed.

[Kitketovsky]

It was all easy and pleasant to setup GitHub Actions pipeline until I decided to split one job to multiple for targeting the separation of concerns and DRY, until I found out GitHub's method to bill their customers. Most likely will be looking for another CI/CD service.

[zhaoyuheng200]

I guess nobody here has came across 0 second round up to 1 minute yet, but it's true, it can happen :)

[dcmbrown]

Well.... I wrote a test action which would spin up as many runners as I specified a week or two back. It was to definitively validate a marketplace action and also separately the service that it was using to determine which of them kept failing on an intermittent basis and specifically from GitHub. After several tests and using up about 300 runner minutes in about 5 minutes of actual run time, I can report this is a thing.

In the last of those tests, which the screenshot above is a part of, I have two runners which take 0 seconds, four 1 second runners, twenty seven 2 second runners and the rest are all +3 seconds to 2m17s (for the longest).

[frankiesimon]

This really kills the idea of using this as a cheap method to run short scripts often. Too bad, I thought the setup was pretty neat.

[jpmaniqis]

This really kills the idea of using this as a cheap method to run short scripts often. Too bad, I thought the setup was pretty neat.

[piotrekkr]

Something is quite off with how Github calculate run time of a job I have this job run:

It is basically 1s based on times from this screenshot. But then in summary page I have 12 seconds duration and billable time 1 minute. Also summary page shows 0 seconds for this job in dependency chart.

Is there like 11s tax added to each job run? Why three different job duration displayed? Anyone can explain this?

[MatthewRyanRead]

Time waiting for a runner, etc., is included in the overall duration. Basically, the overall workflow is more than just its jobs.

I don't know whether that extra runtime affects the billable time though.

[piotrekkr]

GH should include this additional time somewhere in summary or billing information. How could I know how much time does it take to boot runner? Billing should be as clear as possible and displaying three different run times for same job is only confusing. Also if there can be fractions of seconds rounded up then they should also print those fractions or rounded value. Not 0s and we pay for 1m billable time. This is just confusing.

[mrchief]

3 years and still no recourse... I think it might be time to move to AWS Codebuild or GCP Cloudbuild or similar. As much as I love Github Actions and as much as I'm a big proponent of using them, this billing nonsense is hard to justify.

[ekwoka]

Even just billing for all of the "getting the runner setup/killing it" would be reasonable. Just arbitrarily rounding up is kind of nonsense.

It's quite useful in PR checks to have different things be unique jobs, so that you don't have one check failing preventing other checks from even running. It's nice to clearly see the specific thing that failed quickly, and also see that the other stuff is good

[almightyju]

It's insanity that this is still unchanged after all these years.

I just split my workflow up to run some stuff in parallel to speed up the overall deployments and now I'm seeing the billable time 6-8 mins higher than the total duration, mostly because a lot of my jobs just happen to go over the minute mark by a few seconds

[SteveChurch]

I couldn't agree more. It has forced many people to create unbearable monolithic jobs and remove the use of the matrix to save them hundreds if not thousands of $$ per month. It doesn't really promote best practices being forced into doing this to avoid un-reasonable bills.

I appreciate the comments about scale up/down however even still, the billing around these actions seem way off point. I guess if people are racking up huge bills and it's becoming a problem, that becomes the time we move to self hosted runners.

I haven't tried this yet, but going to take a look. I have been playing with Blacksmith' runners too, they are nice, quite early days with supporting a wider range of runners, but rather quick in my tests and cost effective.

[aandis]

this is essentially robbery.

[lkfortuna]

Hi there from Actions 👋 - I understand there is a lot of community disappointment over the billing model of Actions. I understand the frustration when you see <1min of runtime in the UI and then are charged for 1 minute. I hear the feedback on being able to clearly see what is billed vs not billed, I will take that back to the team and evaluate how we can make clear the billed time for a run better. We are hoping to alleviate a bit of the pain by offering lower cost ARM runners for the community at end of year.

Just to be transparent, we have this billing model to cover the costs on our end of setting up/tearing down VMs, to keep queue times low, and to offer runners for free on public repositories. At this time we do not have plans to change to a per-second billing model. I know this will cause community frustration, but that is our reality. The way that our service works, we are not able to offer per second billing like a large scale cloud provider can, since we have to stand the VM up, install runner on the machine, install the image, etc. We are a business and we need to be able to run a net zero margin at least. I want to provide a clear answer here, so hopefully this helps to understand why we are not doing this.

[jaisw7]

@crohr The article talks about a virtual machine image which has some packages installed (At large scale, we can't simply install operating systems and packages on every new server machine/rack the company bought yesterday - Having a VM image ensures uniformity). It also states that "containers" come pre-bundled with VM. So, a CI job run inside a container in the virtual machine which makes sense (container's can get updated whenever Github releases a new version of action runner without updating the underlying VM. Since a minor update of a reasonably stable piece of software happens every 7 days, the containers/VM need to be updated every 7 days). The point is: You do not "start/stop" VM for every CI job (only a container which takes less than 2 seconds), and therefore the extra overhead cost (rounding to a minute) is unjustified.

[crohr]

@jaisw7 you are wrong, GHA runners are not launched within a container. And yes they are most likely cycled (i.e. destroyed & recreated) after every job, to ensure security. The link you mention is for docker support within the VM, with some base images (node, ruby, etc) preloaded on the VM, not some runner container, since the runner is a binary installed ON THE VM. I'll stop the discussion here because I don't think you are really interested in how it all works.

[jaisw7]

@crohr

you are wrong, GHA runners are not launched within a container

Even the third-party services such as CircleCI or self-hosted runners make use of containers. It's a pretty basic tribal knowledge.

yes they are most likely cycled (i.e. destroyed & recreated) after every job, to ensure security

That is again a speculation. Containers (by the very definition are containers) are secure and it is easy to create/destroy them as opposed to VMs. Fun fact: This is primarily the reason (including fast start/stop time) for proliferation of containers over VM in the current decade.

The link you mention is for docker support within the VM, with some base images (node, ruby, etc) preloaded on the VM, not some runner container, since the runner is a binary installed ON THE VM

A docker daemon running inside a VM can pull containers (via internet). These containers have the runner binary installed (which typically get started when containers start).

I'll stop the discussion here because I don't think you are really interested in how it all works

Sure buddy! In the meantime, consider reading about how self-hosted runners work.

[dserodio]

@jaisw7 self-hosted runners can be safely run in containers because you're the single tenant, and you're not worried about other clients reading information from your containers. GitHub is a public service and has a free tier, and as such is subject to all kinds of abuse, like attackers running malicious GHA jobs trying to steal information from GitHub customers. VMs offer a greater level of isolation, if you search for "container escape exploit" you will find a bunch of vulnerabilities that allow these kinds of attacks to be sucessful.

[PWrzesinski]

Hi there from Actions 👋 - I understand there is a lot of community disappointment over the billing model of Actions. I understand the frustration when you see <1min of runtime in the UI and then are charged for 1 minute. I hear the feedback on being able to clearly see what is billed vs not billed, I will take that back to the team and evaluate how we can make clear the billed time for a run better. We are hoping to alleviate a bit of the pain by offering lower cost ARM runners for the community at end of year.

Just to be transparent, we have this billing model to cover the costs on our end of setting up/tearing down VMs, to keep queue times low, and to offer runners for free on public repositories. At this time we do not have plans to change to a per-second billing model. I know this will cause community frustration, but that is our reality. The way that our service works, we are not able to offer per second billing like a large scale cloud provider can, since we have to stand the VM up, install runner on the machine, install the image, etc. We are a business and we need to be able to run a net zero margin at least. I want to provide a clear answer here, so hopefully this helps to understand why we are not doing this.

Hello, thought I'd share my opinion as well, because it seems to differ from most other responses. I came here also angry about being billed for the full minute when my quick health-check job takes like 10 seconds, but having read your explanation about the time required to set everything up I can understand it and I think it's fair. It's great that you care enough about the users to give a thorough explanation. Thanks!

[ekwoka]

@lkfortuna It's helpful to know that it is being looked at.

I think, at a minimum, having the UI do better to communicate overhead costs, at least on <1m runs could probably help reduce the feeling of being "robbed". It does make sense that there are costs to actions not directly covered by what we as users choose to run in an action, but those never being mentioned anywhere certainly makes a feeling of "what the heck is going on?"

[oleksandr-kinship]

@lkfortuna I think me same as most community can understand that GA runners have to be scaled up/down and that have to be charged. Github/Microsoft isn't charity company and you should have profit definitely, but the current billing approach looks as scam.
For example my job running 5 second. but Im charged for 1min=60sec. means 55 second you charge for instance up/down, really? I don't think that truth in 2024. Just imagining you call for Uber and instead of paying 5$ for ride you pay 60$ because your driver have to go to you some miles(but you don't know how much is it 500 meters or it's 5 miles, never mind pay 60$)
So options that could be improved:

• make visible up/down time for instance and add these seconds to job, e.g. 5 seconds to make instance ready -> 5 seconds job running -> 3 seconds to shut down. pay for 13 seconds. (NOT 60 seconds)
• change price to include up/down charge into price. (when I buying bread in store I don't care about time and electricity that were spent to make it done, but that price included into final charge already). You should have statistics that show average start time for a server. e.g I will pay for 10 second of runtime not 0.10 but 0.11. Just an example. It could be even bargain for you because big % isn't short but you will charge more.

Anyway the model have to be changed. Charge with seconds is the right way, I want to pay for what I use, but not for something that hidden from me under hood

[nanderv]

It's a business with a pricing model. It's not a scam, because it's written down quite clearly in the docs.

Price of a service doesn't need to correlate with actual costs; the price is based on what they think us customers are willing to pay for it. They have a service, and a pricing model. If you dislike the service/pricing model proposition, you are under no obligation to keep using this service.
As-is, there's no right way, there's just the way that github/microsoft wants to monetize their service. And from their perspective I do get that I'd wanna have fewer actions runs that run for longer, instead of micro-actions.

[ben-codner-dx]

@nenderv, I understand your point, and you're not wrong about the service and pricing model being clearly outlined. However, the issue being raised here—especially with matrix jobs—creates a significant pain point for many users. When running multiple tasks concurrently, the cost quickly becomes disproportionately high due to the way the billing is structured. For example, if each task only takes a few seconds to run but you're billed for a full minute, the total cost for running a matrix can escalate rapidly, even though the actual compute time is much lower.

This also affects how pipelines are designed. For instance, if you have a pipeline with, say, 30 jobs, each running for 5 seconds, you could end up paying for 30 minutes of compute time. That kind of pricing structure forces you to rethink how you design your pipelines, often leading to unnecessary compromises just to minimize costs.

As a paid customer, it's hard not to feel frustrated when the billing doesn't align with the actual usage. I think most of us here are just looking for more transparency and fairness in the billing process.

[ekwoka]

Yeah, I think at the core, the initial "correction" is to better communicate and clarify the billing, and potentially allow sub-minute chunks. Better communicating the "per run" cost in addition to the time user code runs. This is overall the "simple" part, since it's about communication, not how the systems actually work.

Separately, a way to potentially have matrix reuse containers/runners in a smarter way, where maybe we can say "do this matrix, but only 2 at a time, and reuse them", so that the cost of create/kill is not for every single option in the matrix (maybe still with some amount of context switching cost). Or something else kind of similar. Hell, maybe something that could even better run a matrix inside a single run instance, where a step could be matrixed without matrixing the entire job (which can reuse some steps more efficiently).

While of course you could do things like that now, they don't show as separate "jobs" in many places that it might matter. So even maybe just having ways to have checks and stuff link to a specific task. A benefit I see in heavy matrix use is getting simpler to review PR checks, where you might have multiple separate things (like a multi package monorepo) that you'd want separate checks for.

[jeremyjpj0916]

2026 and nothing changed lol. GG.

[Animalmix55]

The house always wins!