Although there are datable traces of Pre and Proto-history found in Castelo Branco Castle, it was during the Middle Ages that the Templar fortress was founded.

This building would have occurred between 1214 and 1230 and closed a wall and tower enclosure. It is admitted that its limits were ordered to be extended by D. Dinis, but, to be sure, it is only known that it was D. Afonso IV (1343) who ordered the construction of the second belt of walls.

In the enclosure of this fortress, the so-called alcáçova, there was the Church of Santa Maria do Castelo and the Palace of the Alcaides.

Through the engravings of the Book of Fortities, by Duarte d'Armas, we can analyze Castelo Branco from the 19th century. XVI as a village-fortress, with narrow streets, featuring buildings with Manueline architecture ornaments.

This fortification was part of a broader defensive system, built along the Tagus River valley.