It seems validatetrans support was never added to libselinux, despite being added to
selinuxfs in kernel version 4.5
There is a utility to test, however the targeted policy has no validatetrans rules so some must be added:
$ cat validatetrans.cil
(mlsvalidatetrans db_table (and (or (or (or (eq l1 l2) (and (eq t3 unconfined_t) (domby l1 l2))) (and (eq t3 unconfined_t) (dom l1 l2))) (and (eq t3 unconfined_t) (incomp l1 l2))) (or (or (or (eq l1 h2) (and (eq t3 unconfined_t) (domby h1 h2))) (and (eq t3 unconfined_t) (dom h1 h2))) (and (eq t3 unconfined_t) (incomp h1 h2)))))
$ sudo semodule -i validatetrans.cil
$ ./validatetrans system_u:system_r:kernel_t:s0 system_u:system_r:init_t:s0:c0 db_table system_u:system_r: # invalid context here
opening /sys/fs/selinux/validatetrans
security_validatetrans returned -1 errno: Invalid argument
$ ./validatetrans system_u:system_r:kernel_t:s0 system_u:system_r:init_t:s0:c0 db_table system_u:system_r:init_t:s0
opening /sys/fs/selinux/validatetrans
security_validatetrans returned -1 errno: Operation not permitted
$ ./validatetrans system_u:system_r:kernel_t:s0 system_u:system_r:init_t:s0:c0 db_table system_u:system_r:unconfined_t:s0
opening /sys/fs/selinux/validatetrans
security_validatetrans returned 0 errno: Success
Signed-off-by: Joshua Brindle <joshua.brindle@crunchydata.com>