Proving Collatz is "Baker-Hard" (third try)

Ok, hopefully third time’s the charm… (Whoa, simultaneous with @oros post!)

Tao’s Proposition 6 gives a very interesting, sobering reason why the Collatz conjecture is so hard.

Consider:

Proposition A. 2^k - 3^x > x/2, for all integers x and k.
(Let’s restrict to cases where 2^k - 3^x is positive.)

Can you prove Proposition A?

If you can, then you know why 2^{3489238474} - 3^{93849899} \neq 28492791 without having to work out the arithmetic.

Proposition 6. If you can’t prove proposition A, then you can’t prove the Collatz conjecture.

This is sobering, because Proposition A is known to be very hard. It’s a watered-down version of Baker’s theorem showing that 2^k - 3^x grows exponentially in x. But not too watered-down, says Tao – it requires the same transcendental math that won Baker the Fields Medal in the 1960s.

Proposition 6 basically says the Collatz conjecture is “Baker-hard”. The rest of this post justifies it.

Tao’s idea is to show how solving the Collatz conjecture automatically establishes Proposition A … as a side-effect. Meaning, proving Collatz is even harder than proving Proposition A.

Tao does this by taking any claim of the form 2^k - 3^x = n (where n < x/2) and converting it into a Collatz counterexample cycle. Supposing you’ve already proved the Collatz conjecture, that generates a contradiction, and therefore the claim is false. In this way, a proof of the Collatz conjecture establishes Proposition A, by showing all such claims to be false.

Important concepts first:

The way we’ll create a Collatz cycle is to first create a parity sequence v (such as 1001011), a sequence “up” (1) and “down” (0) Collatz steps. Given such a sequence, there’s only one number m that returns to itself after following that sequence of (3n+1)/2 and n/2 operations.

v = a parity sequence

k = the length of v

x = the number of 1s in v

m = \cfrac{\beta(v)}{2^k - 3^x} = number that loops back on itself.

\beta(v) depends on where the 1s lie amongst the 0s in v. It’s a nasty function, but if you master it, you’ll be able to do all kinds of Collatz stuff.

Let p_i be the (zero-based) position of the ith 1.

\beta(v) = \sum_{i=0}^{x-1} 2^{p_i} 3^{x-i-1}

That is, \beta(v) is a sum of products of the form 2^a 3^b, where the a exponents increase as the b exponents decrease.

Some examples:

\beta(11100) = 19
\beta(11010) = 23
\beta(11011010) = 319

If \beta(v) is a multiple of 2^k - 3^x, then m is an integer, and is a member of a Collatz counterexample cycle. (The reason no one has found a Collatz counterexample in real life is that m has always turned out to be a fraction, not an integer.)

That’s it for the preliminaries.

Now suppose we receive a claim of 2^k - 3^x = 5, where 5 < x/2.

We create a Collatz counterexample cycle like this:

If x is even, v = 1^{x} 0^{k-x}.
Otherwise, v = 1^{x-2} 0 1 0 0 0 1 0^{k-x-4}

It can be verified that in all cases, \beta(v) is a multiple of 5. For example, if k=13, x=8, then \beta(1111111100000) = 6305.

(Notice that the legit claim 2^5 - 3^3 = 5, never enters the picture here, because n=5 isn’t less than x/2=1.5.)

We could proceed like this, ruling out claims with n=5, n=7, n=11, and so on, but every v construction would be bespoke. We want a general construction that works no matter what 2^k - 3^x = n claim we are handed.

Empirically, we observe that over all (k,x)-parity sequences, about 1/n of them have \beta(v) divisible by n. For example, 60/210 of the (k=10, x=6) sequences have \beta(v) divisible by 5, and the stats seem to approach 1/n as parity sequences get longer.

But we do math, not physics, so we need an ironclad proof, not a so-called law based on empirical observations.

A couple of quick points:

  1. Notice that n must odd, because it’s the difference between an even number and an odd number. It must also be throdd. So it’s coprime with 6.

  2. We only have to deal with claims where k = \lceil x \log_2 3 \rceil. If k is larger than that, then having shown 2^{\lceil x \log_2 3 \rceil} - 3^x > x/2 suffices to show 2^k - 3^x > x/2, so refuting claims with this special k is enough for Proposition 6.

Now create parity sequence w from the k and x in the claim:

w = (10)^{k-x} 1^{2x-k}

\beta(w) may very well be a multiple of n, in which case we’re done! It probably isn’t, though.

But it turns out some modification of w (call it v) is guaranteed to have a \beta(v) that’s a multiple of n.

Each of the 10 substrings is a “swap opportunity”, where we can flip the 10 to 01. There are at least x/2 opportunities:

k-x > \lceil x \log_2 3 - x \rceil \approx 0.58x > x/2.

Each swap changes the value of \beta(w). Removing the 1, we subtract some 2^i 3^j from \beta(w), and putting it back in (one position over), we add 2^{i+1} 3^j. The other 1s make their same contributions undisturbed. Therefore, a swap changes \beta(w) by \delta = 3^j (2^{i+1} - 2^i) = 2^i 3^j.

Our goal is to execute a subset of swaps that will render \beta(v) \equiv 0 \mod n. The sum of the \delta values of those swaps just needs to be congruent to -\beta(w).

If the x/2 swap \delta values covered the range of residues (mod n), from 0 to n-1, we’d be golden. Just pick the single swap that “kills” \beta(w).

Unfortunately, we don’t see this kind of diversity. For all we know, all the \delta values are congruent to each other, so maybe none of them individually offset \beta(w).

Enter the key point:

Each \delta is coprime to n, because n is odd and throdd (no factors of 2 or 3), while \delta only has factors of 2 and 3.

Now imagine if we could execute a swap on the first 10 … but multiple times. (We can’t, but bear with me.) Eventually, because of the co-primeness, we’d change \beta(w) by a \cdot \delta \equiv -\beta(w) mod n, so that \beta(v) \equiv 0 mod n.

Now consider swapping the first 10 multiple times, plus swapping the second 10 multiple times. We’d get b \cdot \delta_1 + c \cdot \delta_2. Even if the \delta values were the same, our b and c would each be smaller than a.

Finally, considering swapping the first n instances of 10. We now only have to swap each one at most one time (so: “swap” or “not swap”) to obtain a sum of \delta values that “kill” \beta(w).

That’s the crux of it. Imagine the nightmare case where all n \delta values are congruent to each other mod n. Then there’s still some subset of them whose \delta values offset \beta(w), because \delta is coprime to n. It would just like swapping a single 10 multiple times.

I don’t know what the name of this theorem is, but I convinced myself it must be true: Given a list of n numbers coprime to n, then for any residue r (such as 0), there’s some subset whose sum is congruent to r (mod n).

I can post a working example if useful, e.g., produce a Collatz loop for the claim 2^{65}-3^{41}=19. If the claim is true, then here’s an integer m that loops back to itself after Collatz operations given by parity vector v … a Collatz counterexample loop. A putative proof of the Collatz conjecture would say “Hey, no such loop exists,” automatically falsifying the claim (which, by the way, is false!), and all other claims like it. So as a side effect, that Collatz proof would establish Proposition A.

Note: If we want to strengthen Proposition A to something like 2^k - 3^x > 1.4^x, we might try to manufacture a w where the x/2 swaps modify \beta(w) by adding residues 1, 2, 4, 8, 16, \ldots. From those residues (a “binary basis”), it’s easy to see how any \beta(w) mod n (between 0 and n-1) can be killed by some combination of swaps. Then with x/2 swaps, we would actually cover 2^{x/2} \approx 1.4^x residues, instead of just x/2 residues. It seems hard to do exactly that, but maybe something similar to that.

Interested in any comments/corrections, either on the technical part or the philosophical part …

Here’s a working example.

Someone claims: 2^{65} - 3^{41} = 19.

We create this parity vector of length 65 with 41 ones:

w = 10101010101010101010101010101010101010101010101011111111111111111

\beta(w) = 72626082261146049464221

-\beta(w) \equiv 10 mod 19.

We want to modify w so that the \beta becomes 0 mod 19.

We have 24 possible flip opportunities (10 \rightarrow 01). If you flip them individually, the delta \beta(v) - \beta(w) values are:

12157665459056928801,
16210220612075905068,

6815573319906622439424,
9087431093208829919232

The residues of those deltas (mod 19) are:

5, 13, 4, 2, 9, 12, 16, 15, 1, 14, 6,8, 17, 10, 7, 3, 4, 18, 5, 13, 11, 2, 9, 12

We’re looking for a combination of these flip residues that sums to -\beta(w) mod 19 = 10 (ten). Given the diversity, it’s clear that this should be easy to do! For example, flip the 5th and 9th instances of 10 to 01, because 9 + 1 = 10.

v = 10101010{\color{red}01}101010{\color{red}01}10101010101010101010101010101011111111111111111

\beta(v) = 72785946018878675490973

\cfrac{\beta(v)}{2^k-3^x} = \cfrac{72785946018878675490973}{19} = 3830839264151509236367

So, if 2^{65} - 3^{41} = 19, then we have a Collatz counterexample loop of length 65 starting at 3830839264151509236367. But in fact, that start number goes to 1 … so we conclude 2^{65} - 3^{41} \neq 19 (that is, the claim is wrong).

The residue diversity above is typical, and it always looks “easy” to find a combination of flips to create \beta(v) \equiv 0 mod n. But it’s hard to theoretically ensure there’s enough diversity in all cases. That’s what I’m getting at with “if each of n deltas is coprime with n = 2^k-3^x, then any residue (mod n) … including -\beta(w) … can be hit with some combination of deltas.” We can’t give a formula for which of those delta combinations does the job, but the idea is that we know one exists.

This looks promising!

Maybe we should dig into the most famous theorems of additive combinatorics … And I did just that and found that the Cauchy–Davenport (CD) theorem might be helpfull when m is prime. According to CD, a collection A of (m+1)/2 distinct residues is always sufficient to cover all classes with only one addition of two elements of A (two swaps). However, we cannot use a single element twice unless this element corresponds to a repeated residue. So we have to be carefull when applying such a powerful theorem!

According to wikipedia, a direct consequence of the CD theorem is: Given any sequence S of p−1 or more nonzero elements, not necessarily distinct, of \mathbb {Z} /p\mathbb {Z} (where p is prime) , every element of \mathbb {Z} /p\mathbb {Z} can be written as the sum of the elements of some subsequence (possibly empty) of S.

When m is not prime, we should maybe have a look at the more general Kneser’s theorem.

1 Like

Thanks! Ah, all this unfortunate group theory …

I think I get the picture.

We want to show that if we have \delta_1, \delta_2, \ldots \delta_n, all coprime with n, then for any target residue r (mod n), there’s some subset of deltas whose sum is congruent to r.

The strategy is to incrementally add deltas, one at a time, and show that each new delta enlarges the set of “hittable” residues (unless/until that set is complete).

If we start with the empty set of deltas, we can hit residue 0 mod n. (In our context, “make no flips” and just go with parity vector w.)

When we add \delta_1, then we can hit at least one more residue, because \delta_1 \not\equiv 0 mod n (because they’re coprime).

When we further add \delta_2, then there are two cases:

  1. If \delta_2 \not\equiv \delta_1 \not\equiv 0 mod n, then we now have at least three hittable residues.

  2. If \delta_1 \equiv \delta_2, then the subset sum \delta_1 + \delta_2 provides a third hittable residue that’s neither 0 nor \delta_1.

    Showing \delta_1 + \delta_2 \not\equiv 0:
    \delta_1 + \delta_2 = 2 \delta_1 \not\equiv 0, because \delta_1 \not\equiv 0.

    Showing \delta_1 + \delta_2 \not\equiv \delta_1:
    Suppose \delta_1 + \delta_2 = 2 \delta_1 \equiv \delta_1. Then 2 \delta_1 - \delta_1 \equiv 0, so \delta_1 \equiv 0, a contradiction.

Then, when we further add \delta_3 … more of the same.

I’m fairly sure this is correct, though “more of the same” is hardly acceptable, so this reasoning has to be telescoped into a short proof.

Which I was sorta hoping already has a name, though I don’t find a basic modular-arithmetic proof out there (yet), only abstract group theory proofs. Which, if I understood those, I’d probably have already understood Tao’s original proof of Proposition 6!

Those results date back to Augustin Cauchy (around 1815), even before the very notion of a group was introduced. Hence, they should be regarded as “elementary”, and definitely not as “Baker-hard”.

By the way, it is amazing that you managed to “rediscover” the exact statement that directly follows from Cauchy–Davenport theorem.

Another observation is that Tao himself is an expert in the field of additive combinatorics and almost certainly had these kinds of approaches in mind when formulating his proposition.

Haha, hardly … I learned from a guy to deal with a pile of rocks by first imagining a pile of one rock. Turning all this into a five-minute YouTube video might qualify as amazing though :slight_smile:

For the example: x=41 and K= 65 and 3^41 has 6 leading one bits

Applying proposition A we have:

x/2 < 2^(K-Ones-1) < 2^K - 3^x

20.5 < 288_23037_61517_11744 < 420_49177_02483_16829

Considering the number of leading one bits in 3^x might be useful.

1 Like

I get you: there are reasons why x=41, k=65 is a “close call” (2^{65}-3^{41} is smaller than expected). Not as close as x/2, and not even as close as 2.56^x! Pretty sure even Baker’s celebrated bounds are way loose.

They are close because 3^41 has 6 leading one bits. The more leading ones the closer they get.

Lead   Lowest Exponent for
Ones   3^Exponent with One

1                   0
2                   3
3                  10
4                   5
5                  29
6                  41
7                 147
8                 253
9                 306
10               1_636
11               8_951
12              12_276
13              14_271
14              31_202
15              15_601
16              47_468
17             158_670
18              79_335
19           2_858_055
20           1_524_296
21             762_148
22             381_074
23             190_537
24          32_343_822
25          21_562_548
26          10_781_274
27         118_212_940
28         343_857_546
29         171_928_773

I don’t know a good way to approach this question from the zoomed-out perspective of “true in generality.” Instead, I tried to narrow it down and focus instead on a clear application of Baker, to see if we could unravel the need for Baker in that specific case.

The specific case I used: Can we come up with an elementary (non-Baker) way to put absolute bounds on the size the minimum valued member of a non-trivial cycle must be, based on the cycle’s length. That is, if a cycle is a billion (odd) terms long, what size constraints exist for the smallest term in that cycle ? What if it’s a trillion (odd) terms long?

This approach is supported by a widely accepted cycle equation and some solid published work that discusses minimum cycle lengths, etc.

For an odd‑only Collatz cycle with odd terms x_0,\dots,x_{n-1} (so n is the number of odd steps) and with

3x_i+1=2^{a_i}x_{i+1},\qquad K=\sum_{i=0}^{n-1} a_i,

one has the exact identity

K\ln 2 - n\ln 3 \;=\; \sum_{i=0}^{n-1}\ln\!\Bigl(1+\tfrac{1}{3x_i}\Bigr)\;>\;0.

If we let a_{\min}=\min_i x_i. Since x_i\ge a_{\min} and \ln(1+u)\le u for u>0,

K\ln 2 - n\ln 3 \;\le\;\sum_{i=0}^{n-1}\frac{1}{3x_i}\;\le\; \frac{n}{3a_{\min}}

This part is elementary and means that large a_{\min} forces the mismatch K\ln 2-n\ln 3 to be small (tiny, actually).

What it doesn’t give you, though, is a bound on a_{\min}. It’s only able to say “if the smallest term were really big, then the 2–3 mismatch would be really small.”

For the actual bound, you need Baker.

K\ln 2 - n\ln 3 \;\ge\; L(n),

where L(n) is an explicit (but really tiny) function of n.

For two fixed rationals 2,3, standard bounds (linear in logs) have the shape

L(n)\;\gtrsim\; \frac{1}{n^{C}}

(for some value of the constant C that is really big, but computable)

The exact constant/exponent isn’t really the point - the point is simply that there is a uniform positive lower bound which depends only on the sizes of the integer coefficients K,n (and here K\asymp n).

If we combine the two sides:

L(n)\;\le\; K\ln 2 - n\ln 3 \;\le\; \frac{n}{3a_{\min}} \quad\Longrightarrow\quad a_{\min}\;\le\; \frac{n}{3\,L(n)}

That yields an upper bound on the smallest cycle term (as a function of n).

With a typical shape L(n)\gtrsim n^{-C}, this reads

a_{\min}\;\lesssim\; \tfrac{1}{3}\,n^{C+1}

Those constants (which I don’t know) are ridiculous, so the bound is totally useless numerically - but it does give an upper bound on a_{\min} for fixed n, which the elementary part could not supply.

If we take away Baker, the elementary identity still says

K\ln 2 - n\ln 3 \;\le\; \frac{n}{3a_{\min}}

(large a_{\min}tiny mismatch)

But without a uniform positive lower bound on K\ln 2 - n\ln 3, there’s nothing to stop the mismatch from being arbitrarily small. That means you can’t turn the elementary inequality into any concrete bound on a_{\min} (upper or lower) that depends only on n.

This doesn’t say anything about Collatz being “Baker‑hard,” but it’s the only way I can come up with to couple the ideas of “mismatch” and “smallest term” with the needed lower bound on the mismatch itself.

Here’s a clue for you all.

If N is the first value of the sequence less than the Seed
and L is the number of odd steps to that point
and Ones is the number of leading one bits in 3^L then:

  Seed - Seed / 2^Ones   <=   N   <=   Seed

The walrus was Paul. It’s funny because it’s true.

1 Like

I like it, it’s a seed specific, or orbit-segment specific, version of E\ln2-L\ln3. In fact, I think you can use for any a n + 1 family if you replace (3,2) by (a,2) and “leading ones” by the leading-ones run of a^L in base 2.