size-label action fails all PRs

[pcj]

@alexeagle The size-label github action does not appear to work with pull requests. It appears to immediately fail I think due to the GITHUB_TOKEN permissions.

Ref: #1782 (your PR) and #1784 (mine)

[gravypod]

I've setup a revert PR. If someone knows a better solution that would be preferable. I think it's a nice to have feature but I'd rather disable it than opening up a possible security risk or something.