In the 17th century, Bishop Fray Agustín de Carvala authorized the installation of the convent of San José, run by the Augustinian Recollects. On April 15, 1612 the convent of San José was founded. That same year the seminary of San Agustín was founded for the preparation of priests. The first stone of the church was laid in 1612 in an orchard donated by Captain Lorenzo de Roa near Panama City. But misfortune afflicted the order; when they were about to inaugurate it, the dome of the church collapsed due to a construction defect and the whole structure was left in ruins.
The religious who occupied it moved to rented premises in the vicinity of the hermitage of Santa Ana, but were evicted by the authorities. After strong tremors in May 1621, they moved back into the same house next to the city from which they had been expelled. The position of this new monastery, although closer than the previous one to the town, but outside of it, saved it from the Panama fire of 1671 caused by the assault and capture of the city by Morgan and his pirates.