ir: Always use canonical types in comparison when possible

After staring at DWARF and ABIXML dumps for a while, I realized that
in the current (non-perfect) state of things, comparing type A and
type B can be slightly different from comparing canonical_type_of(A)
against type B, for instance.  This is essentially because comparing
canonical_type_of(A) against B rather than A against B changes the
order in which the nodes of the graph of types are visited.  Because
that graph has cycles, the order of visiting would essentially change
the value of the hashes that we compute during those visits.

This in turn changes the value of the comparisons depending on the
order of the comparisons.

So, to avoid those subtle changes, this patch ensures that whenever a
type has a canonical type, then it's that canonical type that is
always used in comparison.  This ensures that types coming from ABIXML
(as they are all canonical types) are compared in the same order as
types coming from the ELF binaries.

This fixes the self-comparison of the infinipath-psm package from
fc36, referenced in bug https://sourceware.org/bugzilla/show_bug.cgi?id=29413.

	* src/abg-ir.cc (maybe_get_canonical_type): Define new function
	template.
	(try_canonical_compare): Use the above function to always use
	canonical types in comparison.
	* tests/data/test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64-self-check-report.txt:
	New test reference output.
	* tests/data/test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm:
	Add new binary input.
	* tests/data/test-diff-pkg/infinipath-psm-debuginfo-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm:
	Likewise.
	* tests/data/Makefile.am: Add the new test material above to
	source distribution.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>

src/abg-ir.cc

@@ -878,6 +878,28 @@ notify_equality_failed(const type_or_decl_base *l __attribute__((unused)),
 #define ABG_RETURN_EQUAL(l, r) return ((l) == (r));
 #endif
 
+/// Get the canonical type of a given type T* as a T*.
+///
+/// Note that normally, canonical types are returned as @ref
+/// type_base* (un-typed form, kind of).  This function returns the
+/// canonical type as a T*, just like the T* it is looking at.
+///
+///
+/// @param t the type to consider.
+///
+/// @return either the canonical type of @p t or @p t itself if it
+/// doesn't have any canonical type.
+template<typename T>
+T*
+maybe_get_canonical_type(T* t)
+{
+  if (!t)
+    return nullptr;
+  if (type_base* type = t->get_naked_canonical_type())
+    return dynamic_cast<T*>(type);
+  return t;
+}
+
 /// Compare two types by comparing their canonical types if present.
 ///
 /// If the canonical types are not present (because the types have not
@@ -941,6 +963,11 @@ try_canonical_compare(const T *l, const T *r)
     if (hash_t r_hash = peek_hash_value(*r))
       if (l_hash != r_hash)
 	ABG_RETURN_FALSE;
+
+  // If a type has a canonical type, use its canonical type, always.
+  l = maybe_get_canonical_type(l);
+  r = maybe_get_canonical_type(r);
+
   return equals(*l, *r, 0);
 #endif
 }

tests/data/Makefile.am

@@ -2412,6 +2412,9 @@ test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64-self-check-report.txt \
 test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64.rpm \
 test-diff-pkg/libgm2-debuginfo-14.2.1-1.fc40.x86_64.rpm \
 test-diff-pkg/gcc-debuginfo-14.2.1-1.fc40.x86_64.rpm \
+test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm \
+test-diff-pkg/infinipath-psm-debuginfo-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm \
+test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64-self-check-report.txt \
 \
 test-diff-pkg-ctf/tarpkg-0-dir1.tar \
 test-diff-pkg-ctf/dirpkg-0-report-0.txt \

tests/data/test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64-self-check-report.txt

@@ -0,0 +1,2 @@
+==== SELF CHECK SUCCEEDED for 'libpsm_infinipath.so.1.16' ====
+==== SELF CHECK SUCCEEDED for 'libinfinipath.so.4.0' ====

tests/test-diff-pkg.cc

@@ -785,6 +785,18 @@ static InOutSpec in_out_specs[] =
     "data/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64-self-check-report.txt",
     "output/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64-self-check-report.txt"
   },
+  {
+    "data/test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm",
+    "data/test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm",
+    "--self-check",
+    "",
+    "data/test-diff-pkg/infinipath-psm-debuginfo-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm",
+    "data/test-diff-pkg/infinipath-psm-debuginfo-3.3-26_g604758e_open.6.fc36.5.x86_64.rpm",
+    "",
+    "",
+    "data/test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64-self-check-report.txt",
+    "output/test-diff-pkg/infinipath-psm-3.3-26_g604758e_open.6.fc36.5.x86_64-self-check-report.txt"
+  },
 #endif // WITH_RPM_ZSTD
 #endif //WITH_RPM