Freeze - hash / sort / compare / diff anything

freeze.freeze(data_structure) Freeze tries to convert any data-structure in a hierarchy of tuples.
freeze.vformat(*args, **kwargs) A pformat wrapper that produces narrow representations of data-structures.
freeze.dump(data_structure) Dump will create a human readable version of your data-structure.
freeze.tree_diff_assert(a, b[, n, sort]) User tree_diff() to assert a equals b.
freeze.tree_diff(a, b[, n, sort]) Dump any data-structure or object, traverse it depth-first in-order and apply a unified diff.
freeze.stable_hash(data_structure) Stable hash does: hash(recursive_sort(freeze(data_structure)))
freeze.recursive_hash(data_structure) Recursive hash does: hash(freeze(data_structure))
freeze.recursive_sort(data_structure) Sort a recursive data_structure.
freeze.transparent_repr(string) The result is __repr__ transparent.
freeze.traverse_frozen_data(data_structure) Yields the leaves of the frozen data-structure pre-order.
freeze.TraversalBasedReprCompare(payload) Implements the comparison method for frozen data-structures based on traverse_frozen_data.
freeze.object_to_items(data_structure) Converts a object to a items list respecting also slots.

Freeze the state of data-structures and objects for data-analysis or testing (diffing data-structures). Frozen data-structures consist of only tuples and these are comparable/sortable/hashable. The freeze() function can be used for many purposes for example implement __hash__() for your complex object very fast. dump() is intended for testing and analysis.

Authors: Jean-Louis Fuchs <ganwell@fangorn.ch> https://github.com/ganwell

Run doctests with “python -m freeze”

freeze.xfreeze.freeze(data_structure)[source]

Freeze tries to convert any data-structure in a hierarchy of tuples. Tuples are comparable/sortable/hashable, you can use this with with recursive_sort(). freeze has no recursion detection.

Parameters:data_structure – The structure to convert.
>>> recursive_sort(freeze(_TestClass(True)))
(('a', 'huhu'), ('sub', (('a', 'slot'), ('b', (1, (1, 2, 3), 2, 3)))))
>>> dump((None, (None, None)))
(None, (None, None))
freeze.xfreeze.vformat(*args, **kwargs)[source]

A pformat wrapper that produces narrow representations of data-structures. The result is __repr__ transparent. Non-printable characters won’t be escaped

freeze.xfreeze.dump(data_structure)[source]

Dump will create a human readable version of your data-structure. It will try to dump almost anything, it has recursion detection and will try to display the recursion in a meaningful way.

Parameters:data_structure – The structure to convert.

When you freeze only content counts, same content same hash

>>> a = hash(freeze(_TestClass(True)))
>>> b = hash(freeze(_TestClass(True)))
>>> b == a
True
>>> a = freeze(_TestClass(True))
>>> b = freeze(_TestClass(True))
>>> b == a
True
>>> x = _TestClass(True)
>>> a = freeze(dump(x))
>>> b = freeze(dump(x))
>>> b == a
True

When you dump-freeze only content/type counts, same content/type same hash

  • Two object of the same type with same content will be equal
  • Two object of the different type with same content will be different
>>> a = hash(freeze(dump(_TestClass(True))))
>>> b = hash(freeze(dump(_TestClass(True))))
>>> b == a
True
>>> a = freeze(dump(_TestClass(True)))
>>> b = freeze(dump(_TestClass(True)))
>>> b == a
True
>>> a = hash(freeze(dump(_TestClass(True))))
>>> b = hash(freeze(dump(_TestClass2(True))))
>>> b != a
True
>>> a = freeze(dump(_TestClass(True)))
>>> b = freeze(dump(_TestClass2(True)))
>>> b != a
True
>>> _py2_to_py3(vformat(dump([1, {'a' : 'b'}])))
[1,
 ["<class 'dict'>",
  {'a': 'b'}]]
>>> vformat(recursive_sort(dump(_TestClass(True))))
["<class 'freeze.xfreeze._TestClass'>",
 (('a',
   'huhu'),
  ('sub',
   ["<class 'freeze.xfreeze._TestSlots'>",
    (('a',
      'slot'),
     ('b',
      (1,
       (1,
        2,
        3),
       2,
       3)))]))]
>>> a = _TestSlots()
>>> b = [a, 1, 2, [a, "banane"]]
>>> _no_null_x(vformat(dump(b)))
  {'a': 'slot',
   'b': [1,
         2,
         3,
         (1,
          2,
          3)]}],
 1,
 2,
  'banane']]
>>> a = [1, 2]
>>> _no_null_x(vformat(dump((a, (a, a)))))
([1,
  2],
 ([1,
   2],
  [1,
   2]))
>>> recursive_sort(dump(freeze(_TestClass(True))))
(('a', 'huhu'), ((('a', 'slot'), ('b', (1, (1, 2, 3), 2, 3))), 'sub'))
>>> dump((None, (None, None)))
(None, (None, None))
>>> s = _TestClassWithLen()
>>> a = [s, s]
>>> _no_null_x(vformat(dump(a)))
  {'a': 'huhu'}],
>>> s = (1, 2)
>>> a = [s, s]
>>> _no_null_x(vformat(dump(a)))
[(1,
  2),
 (1,
  2)]
freeze.xfreeze.tree_diff_assert(a, b, n=5, sort=False)[source]

User tree_diff() to assert a equals b. Dump any data-structure or object, traverse it depth-first and apply a unified diff, to display the result.

Parameters:
  • a – data_structure a
  • b – data_structure b
  • n (int) – lines of context
  • sort – sort the data-structure

ATTENTION: Sorting means changing the data-structure. The test-result may differ. But in case of dictionaries the results become comparable because the sorting negates the hash-algorithms “de-sorting”.

>>> a = [
...     'a',
...     [3, 4],
...     {'a': [3, {'w' : set([4, 'tree', frozenset([3,5,2])])}]},
...     []
... ]
>>> b = [
...     'a',
...     [4, 3],
...     {'a': [3, {'w' : set([4, '3', frozenset([2,5,3])])}]},
...     []
... ]
>>> try:
...     tree_diff_assert(a, b, sort=True)
... except:
...     "GOT IT"
'GOT IT'
>>> a = [
...     'a',
...     [3, 4],
...     {'a': [3, {'w' : set([4, '3', frozenset([3,5,2])])}]},
...     []
... ]
>>> b = [
...     'a',
...     [4, 3],
...     {'a': [3, {'w' : set(['3', 4, frozenset([2,5,3])])}]},
...     []
... ]
>>> tree_diff_assert(a, b, sort=True)
>>> a = [
...     'a',
...     [3, 4],
...     {'a': [3, {'w' : set([4, '3', frozenset([3,5,2])])}]},
...     []
... ]
>>> b = [
...     'a',
...     [4, 3],
...     {'a': [3, {'w' : set(['3', 4, frozenset([2,5,3])])}]},
...     []
... ]
>>> try:
...     tree_diff_assert(a, b, sort=False)
... except:
...     "GOT IT"
'GOT IT'
freeze.xfreeze.tree_diff(a, b, n=5, sort=False)[source]

Dump any data-structure or object, traverse it depth-first in-order and apply a unified diff.

Depth-first in-order is just like structure would be printed.

Parameters:
  • a – data_structure a
  • b – data_structure b
  • n (int) – lines of context
  • sort – sort the data-structure

ATTENTION: Sorting means changing the data-structure. The test-result may differ. But in case of dictionaries the results become comparable because the sorting negates the hash-algorithms “de-sorting”.

>>> a = recursive_sort(freeze([
...     'a',
...     [3, 4],
...     {'a': [3, {'w' : set([4, '3', frozenset([3,5,2])])}]},
...     []
... ]))
>>> b = recursive_sort(freeze([
...     'a',
...     [7, 3],
...     {'a': [3, {'w' : set([4, '3', frozenset([2,5,3])])}]},
...     []
... ]))
>>> transparent_repr("\n".join(tree_diff(a, b).split("\n")[2:]))
@@ -7,6 +7,6 @@
       'w'),),
     3),
    'a'),),
  'a',
  (3,
-  4))
+  7))
>>> a = [
...     'a',
...     [3, 4],
...     {'a': [3, {'w' : set([4, '3', frozenset([3,5,2])])}]},
...     []
... ]
>>> b = [
...     'a',
...     [7, 3],
...     {'a': [3, {'w' : set([4, '3', frozenset([2,5,3])])}]},
...     []
... ]
>>> transparent_repr("\n".join(
...     tree_diff(a, b, sort=True
... ).split("\n")[2:]))
@@ -11,6 +11,6 @@
           '3',
           4)]),)],
      3)),)],
  'a',
  (3,
-  4))
+  7))
freeze.xfreeze.stable_hash(data_structure)[source]

Stable hash does: hash(recursive_sort(freeze(data_structure)))

>>> a = stable_hash(_TestClass(True))
>>> b = stable_hash(_TestClass(True))
>>> a == b
True
freeze.xfreeze.recursive_hash(data_structure)[source]

Recursive hash does: hash(freeze(data_structure))

>>> a = recursive_hash(_TestClass(True))
>>> b = recursive_hash(_TestClass(True))
>>> a == b
True
freeze.xfreeze.recursive_sort(data_structure)[source]

Sort a recursive data_structure.

Parameters:data_structure – The structure to convert.

data_structure must be already sortable or you must use freeze() or dump(). The function will work with many kinds of input. Dictionaries will be converted to lists of tuples.

>>> _py2_to_py3(vformat(recursive_sort(dump(
...     [3, 1, {'c' : 'c', 'a' : 'b', 'b' : 'a'}]
... ))))
(["<class 'dict'>",
  (('a',
    'b'),
   ('b',
    'a'),
   ('c',
    'c'))],
 1,
 3)
>>> recursive_sort([3, 1, {'c' : 'c', 'a' : 'b', 'b' : 'a'}])
((('a', 'b'), ('b', 'a'), ('c', 'c')), 1, 3)
>>> recursive_sort(_TestClass())
(('a', 'huhu'),)
freeze.xfreeze.transparent_repr(string)[source]

The result is __repr__ transparent. Non-printable characters won’t be escaped

>>> transparent_repr(3)
3
freeze.xfreeze.traverse_frozen_data(data_structure)[source]

Yields the leaves of the frozen data-structure pre-order.

It will produce the same order as one would write the data-structure.

class freeze.xfreeze.TraversalBasedReprCompare(payload)[source]

Implements the comparison method for frozen data-structures based on traverse_frozen_data.

>>> cm = TraversalBasedReprCompare
>>> cm(3) < cm(4)
True
>>> cm(4) > cm(3)
True
>>> cm(3) > cm(4)
False
>>> cm(3) == cm(3)
True
>>> cm(3) == cm(4)
False
>>> cm((3, 3)) > cm((3,))
True
>>> cm((3, 3)) == cm((3, 3))
True
>>> cm((3,)) > cm((3, 3))
False
>>> cm((3,)) == cm((3, 3))
False
freeze.xfreeze.object_to_items(data_structure)[source]

Converts a object to a items list respecting also slots.

Use dict(object_to_items(obj)) to get a dictionary.

Indices and tables