How does an LLM "know" the sky is blue?

For most people, LLMs are a magic box that spews out language. Somehow, that language is very similar...