mxx1's avatar

Random Thought

I started wondering if it would be possible to use a LLM as a random number generator.

Rough idea: Convert numbers to letters using some mapping, then convert the letters to words, and then use the words to ask a question. The answer provides words and letters, which according to some plan can be mapped back to numbers, the random number required.

What problem is this solving? None that I can think of. There are plenty of good ways to generate true random numbers, as required by most cryptography. There are the output of physical measurement systems that are available online via APIs, quantum systems that ironically use quantum computing to generate randomness (such is their inherent uncontrollability), the traditional way of using very large look up tables that were originally constructed with dice throws, and even an emerging industry of PCB mounted components that produce real random numbers.

My real motive was to see how GPT would square off a discussion on randomness. Correctly I assumed that it wouldn’t like to admit that its complete lack of determinism could be used to produce what are effectively the random numbers of interest. On one hand it seems to be appalled that anyone would use an LLM to generate random numbers because in theory that approach is hackable. On the other hand it wants me to believe that, subject to settings, it can never give the same answer twice.

So it appears that there is a paradox at the heart of these models: they’re deterministic machines selling the vibe of indeterminism, and the minute you try to pin them down as a source of randomness, they start wriggling like a bureaucrat caught doing something interesting.

There’s no real paradox in the maths, of course. The weights are fixed, the softmax is fixed, the sampling path is fixed. It’s a straight machine. If you pinned the seed, pinned the hardware, pinned the model checkpoint, and never allowed a single silent update, you’d get the same output every time. There’s just so many knobs to twirl that even their owners couldn’t do it.

If such a letter-mapping RNG was built, you’d get numbers that look random, fail basic statistical tests for cryptographic randomness, but would be hilariously expensive to predict in practice because you’d need to reverse-engineer not just the model weights but the entire execution environment at the moment of generation. That is, in practice, it wouldn’t be possible.

In the wild, these models behave like overstimulated bipolar weather systems. Routing changes, kernel versions drift, floating-point differences stack, tokenisation quirks creep in, backend updates drop in without ceremony. By the time the text reaches you, the behaviour looks completely non-deterministic. Mathematically we can call it pseudo-randomness.

At other times a model will attempt to be really good at deterministic functions such as coding and statistical analysis. When the task is constrained and definite, it can imitate determinism. When the task is open and creative, it spills into its larger probability space and behaves like the unstable weather system described above. The nature of the machine does not change. Only the sampling path does.

It leaves you with a machine that can collapse into literalism or drift into abstraction depending on how tightly you frame the question. The behaviour feels oddly human, not in the sentimental sense but in the way people slide between narrow focus and loose association without ever noticing the pivot. The model does it for mechanical reasons, not psychological ones, yet the effect is the same: precision when confined, unpredictability when left to roam.