puppet master subcommand is not available with puppet 6

[vinzent]

Running octocatalog-diff with puppet 6 from git master branch, fails with Error: Unknown Puppet subcommand 'master'.

I think this is related to ticket SERVER-990. The proposed alternative is the command puppet catalog find.

full octocatalog-diff output:

W, [2018-05-31T07:28:38.544598 #16441]  WARN -- : Puppet command failed: STDOUT:
  Error: Unknown Puppet subcommand 'master'
  See 'puppet help' for help on available puppet subcommands
STDERR:
W, [2018-05-31T07:28:38.545095 #16441]  WARN -- : Failed build_catalog for HEAD validation: OctocatalogDiff::Errors::CatalogError Catalog failed:
/home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/catalogs.rb:259:in `catalog_validator': Catalog failed:  (OctocatalogDiff::Errors::CatalogError)
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:39:in `call'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:39:in `validate'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:202:in `execute_task'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:119:in `block (2 levels) in run_tasks_parallel'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:117:in `fork'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:117:in `block in run_tasks_parallel'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `each'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `each_with_index'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `run_tasks_parallel'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:94:in `run_tasks'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/catalogs.rb:92:in `build_catalog_parallelizer'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/catalogs.rb:29:in `catalogs'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/api/v1/catalog-diff.rb:34:in `catalog_diff'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/api/v1.rb:19:in `catalog_diff'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/cli.rb:119:in `cli'
        from /home/user/workspace/odiff/bundle/ruby/gems/octocatalog-diff-1.5.3/bin/octocatalog-diff:34:in `<top (required)>'
        from ../odiff/bin/octocatalog-diff:12:in `load'
        from ../odiff/bin/octocatalog-diff:12:in `<main>'
[user@server puppetenv (master)]$ /opt/octocatalog-diff-suite/bin/puppet-dev --version
6.0.0

[vinzent]

unfortunately it's not just s/master --compile/catalog find/.

running with catalog find does not set $facts (at least).

[vinzent]

sent a mail to the puppet-dev mailinglist https://groups.google.com/forum/#!topic/puppet-dev/5yLKqn-DIIQ

[kpaulisse]

I haven't looked at Puppet 6 yet. Regarding the YAML files maybe this could work if we structure them as facts in Puppet's data types? In the past I have tried to keep the YAML files as a plain old hash, because to do otherwise would require importing the puppet gem, and you could run into issues of version conflicts and such.

[vinzent]

@kpaulisse haven't tried, but I imaging it could work, if the yaml files would be the "ruby yaml objects" not the plain yamls.

there is some discussion going on on the ticket now: https://tickets.puppetlabs.com/browse/SERVER-990

[vinzent]

@kpaulisse The ticket https://tickets.puppetlabs.com/browse/PUP-9055 (migrated from SERVER-990) fix-version was set to 6.1.0 - but status still ready for engineering

[shanemadden]

For those not closely tracking Puppet JIRA, https://tickets.puppetlabs.com/browse/SERVER-2434 is planned for 6.3.0 and should be able to be used for octocatalog-diff's needs, but there will likely be some big changes needed to support the new endpoint.

[vinzent]

Release notes:

Puppet Server now has a new endpoint for catalog retrieval that allows many more options than the
previous endpoint. By default, this This endpoint is controlled by tk-auth, and by default is not
generally accessible , and . It is intended for use by other puppet services (like CD4PE). For details
on the API, see https://github.com/puppetlabs/puppetserver/blob/master/documentation/puppet-api/v4/catalog.markdown.

https://tickets.puppetlabs.com/browse/SERVER-2434

[BarnumD]

Perhaps this change will be of use: https://tickets.puppetlabs.com/browse/PUP-9055

[joshcooper]

FYI, puppet 6.5 should be released tomorrow (June 19th) and will contain puppet catalog compile that behaves the same as puppet master --compile did.

[jvperrin]

I've got this working with puppet 6.7.2 (the latest puppet release), and it might work with other releases by applying this diff (a more advanced diff would be needed to be able to support both puppet 6.x with older puppet releases at the same time, but this is really only a proof of concept anyway):

diff --git a/lib/octocatalog-diff/catalog-util/command.rb b/lib/octocatalog-diff/catalog-util/command.rb
index edb8c39..27e279f 100644
--- a/lib/octocatalog-diff/catalog-util/command.rb
+++ b/lib/octocatalog-diff/catalog-util/command.rb
@@ -56,7 +56,7 @@ module OctocatalogDiff

         # Node to compile
         cmdline = []
-        cmdline.concat ['master', '--compile', Shellwords.escape(@node)]
+        cmdline.concat ['catalog', 'compile', Shellwords.escape(@node)]

         # storeconfigs?
         if @options[:storeconfigs]
@@ -95,7 +95,6 @@ module OctocatalogDiff
           --no-daemonize
           --no-ca
           --color=false
-          --config_version="/bin/echo catalogscript"
         )

         # Add environment - only make this variable if preserve_environments is used.

I'm not entirely sure why the removal of --config_version="/bin/echo catalogscript" was needed, but without removing it, I'd always get an error like this:

W, [2019-07-31T23:43:57.537375 #29182]  WARN -- : Puppet command failed: STDOUT:
STDERR:
  Error: no implicit conversion of Symbol into Integer
  Error: Try 'puppet help catalog compile' for usage
W, [2019-07-31T23:43:57.537839 #29182]  WARN -- : Failed build_catalog for . validation: OctocatalogDiff::Errors::CatalogError Catalog failed: Error: no implicit conversion of Symbol into Integer
Error: Try 'puppet help catalog compile' for usage

/home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/catalogs.rb:259:in `catalog_validator': Catalog failed: Error: no implicit conversion of Symbol into Integer (OctocatalogDiff::Errors::CatalogError)
Error: Try 'puppet help catalog compile' for usage
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:39:in `call'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:39:in `validate'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:202:in `execute_task'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:119:in `block (2 levels) in run_tasks_parallel'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:117:in `fork'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:117:in `block in run_tasks_parallel'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:114:in `each'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:114:in `each_with_index'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:114:in `run_tasks_parallel'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/parallel.rb:94:in `run_tasks'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/catalogs.rb:92:in `build_catalog_parallelizer'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/util/catalogs.rb:29:in `catalogs'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/api/v1/catalog-diff.rb:34:in `catalog_diff'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/api/v1.rb:19:in `catalog_diff'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/cli.rb:151:in `run_octocatalog_diff'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/lib/octocatalog-diff/cli.rb:125:in `cli'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/gems/octocatalog-diff-1.5.4/bin/octocatalog-diff:34:in `<top (required)>'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/bin/octocatalog-diff:22:in `load'
        from /home/j/jv/jvperrin/.gem/ruby/2.3.0/bin/octocatalog-diff:22:in `<main>'

which came directly from the puppet catalog compile command.

[kozl]

@jvperrin thank you! Had the same problem with this error above.

[joshcooper]

I'd recommend passing --trace when building the puppet command line. That should show where the no implicit conversion of Symbol into Integer error is coming from in puppet.

[wiebe]

I can confirm the patches by @jvperrin also seem to work with Puppet 6.10.1 :-)
Probably the config options passed should be reviewed too...

[kozl]

Maybe we should do PR and invite someone from maintainers?