The house that was built in 1847 by Claes Werner Rydström is the second oldest building in Gislaved. C Rydström opened Gislaveds first general store. His bookkeeper, Adolf Magnus Dyberg, took over the general store and with it the house 1860. The store was located on the first floor and lived with his family on the second floor.
In the early 1900 century the house was converted into private housing to the banker David Rydin, who was married to Karin Gottfriedz a grandchild to the Dyberg spouses, and his family. Up until Mrs Johanna Dybergs death 1929, at the age of 101, The family lived only on the second floor as Johanna lived on the first. After Johannas death, David Rydin started to refurbish the house.
The original wooden facade was exchanged for a plastered one, and it was also at this time the wall and ceiling paintings on the first floor were created. The property was owned by the Dyberg family for 90 years, until 1950, before Gislaveds Missions bought the house to have as a youth and parish house. The house was given the name "Missionsborgen" and was sold 1983 when the Missions built a new church.
It was at this time the municipality bought the property and converted it to a cultural centre. They renovated the building piously with the help of the county museum's workforce. Among other things the property regained it's wooden facade and the paintings on the first floor was restored.
The Cultural centre Borgen was opened 1985. Jönköpings community group awarded the 1986 honorary prize - "Hedersenen" - to Gislaveds municipality for the historically cultural restoration of the culture centre Borgen.
2013 Gislaveds konsthall took ownership of the house, exhibitions were held on the second floor while the rooms on the first floor served as meeting room, reading room and as a bookable surface for exhibitions.
From 2018 the entire building on Köpmangatan 12 serves as exhibition space.