Revert "Engine reference BUILD tarball contexts mangled"

[mdlinville]

Reverts #175

This removed actual content, and also we don't maintain these docs here, but in docker/docker.

[mdlinville]

@thaJeztah PTAL

[thaJeztah]

looks like this one should stay :)

[thaJeztah]

oh wait, erm, perhaps the fix should be in here, idk

[thaJeztah]

With context of #175 looks like those lines were indeed duplicated, so up to you what's best here; we can carry the change in docker/docker perhaps, or sync?

[mdlinville]

I think docker/docker looks fine as it, unless I am missing something.

[johndmulhausen]

@thaJeztah @mstanleyjones I'd like to have a discussion on why simple fixes can't live in our repo.

Absolutely no publishing of Engine content should be happening that doesn't create a pull request to docker/docker.github.io:master, which will automatically merge with commits that have happened in docker/docker.github.io. So there's certainly no danger of fixes that happen on docker/docker.github.io getting lost or overwritten.

It seems we have to choose between one of two maintenance tasks until such time as reference docs are auto-generated:

1. We can allow edits in docker/docker.github.io, giving us the maintenance task to refresh docker/docker's .md files from time to time
2. We don't allow such edits, thus creating the need to intercept every single attempt to edit the files on docker/docker.github.io and turn it down, and relying on a two-step process (edit the file on docker/docker and cherry-pick over to docker/docker.github.io) for publishing on any edit we want to go live.

Right now we are trying to enforce option #2 and I am unconvinced that it's worth it. Thoughts?

[mdlinville]

If we try to do the first way, there will be style changes in only our repo, and content changes (like updated flags) in only the upstream repo, and I imagine "periodically refreshing docker/docker with our version" will be a reconciliation nightmare. But I could be wrong....

[johndmulhausen]

@mstanleyjones I think we can mitigate the reconciliation nightmare with good timing. For example, at the moment where we know we need to publish changes that have originated from docker/docker we'll end up with a converged version on docker/docker.github.io after the publishing is over. At that moment, we refresh docker/docker. There would only be a few minutes between the time those files come over from docker/docker and come back from docker/docker.github.io with the upstream changes merged in.

[dnephin]

"a reconciliation nightmare" is exactly what we will have with 1.

which will automatically merge with commits that have happened in docker/docker.github.io

Not at all. It will conflict, and the person doing the merge will have to look through the history of two repos and attempt to guess at the intent of dozens of changes. It's just not a workable solution.

We don't allow such edits, thus creating the need to intercept every single attempt to edit the files on docker/docker.github.io and turn it down

This is exactly why so many people have raised concerns about the docs existing in two repos. It is very difficult to communicate that to contributors, even internal and frequent contributors can easily forget.

However, I still believe reference docs should be generated from the code repos, so we still have the problem of having to check in generated reference docs into a repo where they can be edited, but any edits will be lost as soon as we regenerate the docs.

Would it be possible to publish the docs from a non-master branch that refuses all PRs? The publish process would generate the docs, and check them into on the publish branch, so they would never exist in the master branch for someone to accidentally edit. It is much easier to communicate "branch publish is auto-generated and should never receive PRs" (instead of the current "arbitrary directory paths should never be edited").

[johndmulhausen]

@dnephin I hear you. We never wanted the docs to exist in two repos. The only reason we allowed anyone to keep docs in their repo after our migration was they intended to eventually auto-generate them.

There are possibilities like a "publish" branch, certainly, but I think the correct course of action is, once we have automatically-generated reference files, they are published directly to the docs repo, and remaining docs in docker/docker are removed. Further, those reference files wouldn't even be docs -- we've designed a YAML format for expressing the reference information, so that the source will 100% be the code inside docker/docker. That way there will truly be a single source of truth for the reference information, and nothing we do in the docs repo is even "docs" for the reference, it'll just be shell files that process the YAML.

[dsheets]

@mstanleyjones docker/docker has the code fence fixed but still duplicates the (first time) incorrect label:

Instead of specifying a context, you can pass a single Dockerfile in the URL
or pipe the file in via STDIN. To pipe a Dockerfile from STDIN:

Current source:

If you pass an URL to a remote tarball, the URL itself is sent to the daemon:

Instead of specifying a context, you can pass a single Dockerfile in the URL
or pipe the file in via STDIN. To pipe a Dockerfile from STDIN:

$ docker build http://server/context.tar.gz

The download operation will be performed on the host the Docker daemon is
running on, which is not necessarily the same host from which the build command
is being issued. The Docker daemon will fetch context.tar.gz and use it as the
build context. Tarball contexts must be tar archives conforming to the standard
tar UNIX format and can be compressed with any one of the 'xz', 'bzip2',
'gzip' or 'identity' (no compression) formats.

Text files

Instead of specifying a context, you can pass a single Dockerfile in the
URL or pipe the file in via STDIN. To pipe a Dockerfile from STDIN:

$ docker build - < Dockerfile

[joaofnfernandes]

I absolutely agree with @dnephin, and also brought the same point several times.
The same situation is happening with UCP and DTR docs.

For me it's not acceptable to have reference documentation living in two places for the reasons already mentioned above.

When the docs were migrated to this tree, I agreed on temporarily having the reference docs here because I was told that github pages could only deploy content that was on master.
But my understanding is that we would not allow reference docs to be edited here. Contributors would update them upstream, and we would import those updates somehow into this repo to stage them.

There are possibilities like a "publish" branch, certainly, but I think the correct course of action is, once we have automatically-generated reference files, they are published directly to the docs repo, and remaining docs in docker/docker are removed

@johndmulhausen looks like to me you are baiting and switching.
Yes, it would be good to have reference docs being automatically generated. When that happens we can follow the strategy you're suggesting.
In the meantime, while we are not doing that we should not accept changes to reference docs in this tree.

[dsheets]

Half of the original issue remains.