Skip to content

Commit 65e4457

Browse files
authored
Merge pull request #46 from github/kpaulisse-yaml-diff
Add option to ignore whitespace in yaml file diff
2 parents 6df2819 + 40cfc74 commit 65e4457

36 files changed

Lines changed: 575 additions & 2 deletions

.version

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
0.6.0
1+
0.6.1

doc/CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,15 @@
88
</tr>
99
</thead><tbody>
1010
<tr valign=top>
11+
<td>0.6.1</td>
12+
<td>2017-01-07</td>
13+
<td>
14+
<ul>
15+
<li><a href="https://github.com/github/octocatalog-diff/pull/46">#46</a>: Add option to ignore whitespace in yaml file diff</li>
16+
</ul>
17+
</td>
18+
</tr>
19+
<tr valign=top>
1120
<td>0.6.0</td>
1221
<td>2017-01-04</td>
1322
<td>

doc/advanced-filter.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Additional output filters
2+
3+
It is possible to enable additional filters for output results via the `--filters` command line option. This command line option accepts a comma-separated list of additional filters, and applies them to the results in the order you specify. The default behavior is not to use any of these filters.
4+
5+
Please note that there are other options to ignore specified diffs, including:
6+
7+
- [Ignoring certain changes via command line options](/doc/advanced-ignores.md)
8+
- [Dynamic ignoring of changes via tags in Puppet manifests](/doc/advanced-dynamic-ignores.md)
9+
10+
Here is the list of available filters and an explanation of each:
11+
12+
- [YAML](#YAML) - Ignore whitespace/comment differences if YAML parses to the same object
13+
14+
## YAML
15+
16+
#### Usage
17+
18+
```
19+
--filters YAML
20+
```
21+
22+
#### Description
23+
24+
If a file resource has extension `.yml` or `.yaml` and a difference in its content is observed, YAML objects are constructed from the previous and new values. If these YAML objects are identical, the difference is ignored.
25+
26+
This allows you to ignore changes in whitespace, comments, etc., that are not meaningful to a machine parsing the file. Please note that by filtering these changes, you are ignoring changes to comments, which may be meaningful to humans.

doc/advanced.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ See also:
2727
### Controlling output
2828

2929
- [Ignoring certain changes via command line options](/doc/advanced-ignores.md)
30+
- [Additional output filters](/doc/advanced-filter.md)
3031
- [Dynamic ignoring of changes via tags in Puppet manifests](/doc/advanced-dynamic-ignores.md)
3132
- [Output formats](/doc/advanced-output-formats.md)
3233
- [Useful output hacks](/doc/advanced-output-hacks.md)

doc/optionsref.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ Usage: octocatalog-diff [command line options]
4646
--no-hiera-path-strip Do not use any default hiera path strip settings
4747
--ignore-attr "attr1,attr2,..."
4848
Attributes to ignore
49+
--filters FILTER1[,FILTER2[,...]]
50+
Filters to apply
4951
--[no-]display-source Show source file and line for each difference
5052
--[no-]validate-references "before,require,subscribe,notify"
5153
References to validate
@@ -424,6 +426,20 @@ on which this is running. (<a href="../lib/octocatalog-diff/catalog-diff/cli/opt
424426
</td>
425427
</tr>
426428

429+
<tr>
430+
<td valign=top>
431+
<pre><code>--filters FILTER1[,FILTER2[,...]]</code></pre>
432+
</td>
433+
<td valign=top>
434+
Filters to apply
435+
</td>
436+
<td valign=top>
437+
Specify one or more filters to apply to the results of the catalog difference.
438+
For a list of available filters and further explanation, please refer to
439+
[Filtering results](/doc/advanced-filter.md). (<a href="../lib/octocatalog-diff/catalog-diff/cli/options/filters.rb">filters.rb</a>)
440+
</td>
441+
</tr>
442+
427443
<tr>
428444
<td valign=top>
429445
<pre><code>-f FROM_BRANCH
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# frozen_string_literal: true
2+
3+
# Specify one or more filters to apply to the results of the catalog difference.
4+
# For a list of available filters and further explanation, please refer to
5+
# [Filtering results](/doc/advanced-filter.md).
6+
# @param parser [OptionParser object] The OptionParser argument
7+
# @param options [Hash] Options hash being constructed; this is modified in this method.
8+
OctocatalogDiff::CatalogDiff::Cli::Options::Option.newoption(:filters) do
9+
has_weight 199
10+
11+
def parse(parser, options)
12+
parser.on('--filters FILTER1[,FILTER2[,...]]', Array, 'Filters to apply') do |x|
13+
options[:filters] ||= []
14+
options[:filters].concat x
15+
end
16+
end
17+
end

lib/octocatalog-diff/catalog-diff/differ.rb

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
require 'stringio'
88

99
require_relative '../catalog'
10+
require_relative 'filter'
1011

1112
module OctocatalogDiff
1213
module CatalogDiff
@@ -153,6 +154,9 @@ def catdiff
153154
# out any such parameters from the result array.
154155
filter_diffs_for_absent_files(result) if @opts[:suppress_absent_file_details]
155156

157+
# Apply any additional pluggable filters.
158+
OctocatalogDiff::CatalogDiff::Filter.apply_filters(result, @opts[:filters])
159+
156160
# That's it!
157161
@logger.debug "Exiting catdiff; change count: #{result.size}"
158162
result
@@ -175,7 +179,7 @@ def filter_diffs_for_absent_files(result)
175179
absent_files = Set.new
176180
result.each do |diff|
177181
next unless diff[0] == '~' || diff[0] == '!'
178-
next unless diff[1] =~ /^File\f(.+)\fparameters\fensure$/
182+
next unless diff[1] =~ /^File\f([^\f]+)\fparameters\fensure$/
179183
next unless ['absent', 'false', false].include?(diff[3])
180184
absent_files.add Regexp.last_match(1)
181185
end
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
require_relative 'filter/yaml'
2+
3+
module OctocatalogDiff
4+
module CatalogDiff
5+
# Filtering of diffs, and parent class for inheritance.
6+
class Filter
7+
# Public: Apply multiple filters by repeatedly calling the `filter` method for each
8+
# filter in an array. This method returns nothing.
9+
#
10+
# @param result [Array] Difference array (mutated)
11+
# @param filter_names [Array] Filters to run
12+
# @param options [Hash] Options for each filter (hashed by name)
13+
def self.apply_filters(result, filter_names, options = {})
14+
return unless filter_names.is_a?(Array)
15+
filter_names.each { |x| filter(result, x, options[x] || {}) }
16+
end
17+
18+
# Public: Perform a filter on `result` using the specified filter class.
19+
# This mutates `result` by removing items that are ignored. This method
20+
# returns nothing.
21+
#
22+
# @param result [Array] Difference array (mutated)
23+
# @param filter_class_name [String] Filter class name (from `filter` subdirectory)
24+
# @param options [Hash] Additional options (optional) to pass to filtered? method
25+
def self.filter(result, filter_class_name, options = {})
26+
filter_class_name = [name.to_s, filter_class_name].join('::')
27+
clazz = Kernel.const_get(filter_class_name)
28+
result.reject! { |item| clazz.filtered?(item, options) }
29+
end
30+
31+
# Inherited: Construct a default `filtered?` method for the subclass via inheritance.
32+
# Each subclass must implement this method, so the default method errors.
33+
def self.filtered?(_item, _options = {})
34+
raise "No `filtered?` method is implemented in #{name}"
35+
end
36+
end
37+
end
38+
end
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# frozen_string_literal: true
2+
3+
require 'yaml'
4+
5+
module OctocatalogDiff
6+
module CatalogDiff
7+
class Filter
8+
# Filter based on equivalence of YAML objects for file resources with named extensions.
9+
class YAML < OctocatalogDiff::CatalogDiff::Filter
10+
# Public: Actually do the comparison of YAML objects for appropriate resources.
11+
# Return true if the YAML objects are known to be equivalent. Return false if they
12+
# are not equivalent, or if equivalence cannot be determined.
13+
#
14+
# @param diff [Array] Difference
15+
# @param _options [Hash] Additional options (there are none for this filter)
16+
# @return [Boolean] true if this difference is a YAML file with identical objects, false otherwise
17+
def self.filtered?(diff, _options = {})
18+
# Skip additions or removals - focus only on changes
19+
return false unless diff[0] == '~' || diff[0] == '!'
20+
21+
# Make sure we are comparing file content for a file ending in .yaml or .yml extension
22+
return false unless diff[1] =~ /^File\f([^\f]+)\.ya?ml\fparameters\fcontent$/
23+
24+
# Attempt to convert the old (diff[2]) and new (diff[3]) into YAML objects. Assuming
25+
# that doesn't error out, the return value is whether or not they're equal.
26+
obj_old = ::YAML.load(diff[2])
27+
obj_new = ::YAML.load(diff[3])
28+
obj_old == obj_new
29+
rescue # Rescue everything - if something failed, we aren't sure what's going on, so we'll return false.
30+
false
31+
end
32+
end
33+
end
34+
end
35+
end
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"document_type": "Catalog",
3+
"data": {
4+
"tags": ["settings"],
5+
"name": "my.rspec.node",
6+
"version": "production",
7+
"environment": "production",
8+
"resources": [
9+
{
10+
"type": "File",
11+
"title": "/tmp/foo.yaml",
12+
"exported": false,
13+
"parameters": {
14+
"content": "foo:\n bar: baz\n"
15+
}
16+
}
17+
]
18+
},
19+
"metadata": {
20+
"api_version": 1
21+
}
22+
}

0 commit comments

Comments
 (0)