I would also suggest easier tasks to see what is feasible (or not) with a transformer:
- RegEx recognition: try to detect binary strings matching a particular RegEx. Some simple examples could be ((10)^*(101)^*)^* for strings of the form 101010101101101010110 (where every two successive occurences of 0 are separated by either 1 or 11), or (10(0)^*)^* for strings with no two consecutive 1's, starting by 1 and ending by 0. This test seems particularly relevant for the 5 vs 16 task according to this post.
- Modulo operations: try to learn modular arithmetic, like the computation of n \pmod 3 from the base-2 representation of n (that would be super easy in base 24 …). These congruence classes modulo 3 are not regular expressions, but they can be classified by a three-state automaton.