It wasn’t always the case that everyone in the world kept the same time. For centuries, it was impossible, and time could only be defined locally by the nearest clock. At one place it was midday, but down the road, it was 12:15. As recently as the 1800s, the US was operating with hundreds of different time standards, defined by cities and local railroad managers.
Part of the reason was that there was no feasible way to synchronise every clock in a country, let alone right across the Earth. For much of human history, this didn’t matter: people worked when they needed to, didn’t travel far, and if they wanted to know the time, could find out by checking a nearby sundial, town clock, or listening for church bells, or a call to prayer.