The town of Kuala Lipis evolved at the confluence of the Lipis River and Jelai River. Due to its strategic location, the town grew into a prominent trading and commercial centre in the state. So prominent was the town that the British, who came to Pahang since 1872 made Kuala Lipis the state capital in 1891, and remained so until 1955 when Kuantan, the present state capital took over the honors. Now, though no longer a bustling trading centre, it remains as the administrative centre for the district of Lipis, one of the largest districts in Pahang.
As a town that was formerly the state capital, Kuala Lipis is dotted by many historical buildings and structures of colonial architectures. Among them are the various government office buildings in the town, the Clifford School, the rest house at Residency Hill and the Pahang Club buildings.
Railway service in Kuala Lipis started in 1926, when the railway tracks from Johor Bahru in the south that have already reached Gemas in Negeri Sembilan were extended to Kuala Lipis and Cegar Perah in the north. Later, the railway tracks up north was extended to reach Tumpat in Kelantan.