Residents are concerned about the impact that a redevelopment of a 0.66 acre site could have on Llandaf Road in Canton.
Suffolk House was built in 1800 by a wealthy father as a gift to his daughter.
But it has seen better days.
The council originally agreed to sell the premises due to the “poor condition and high cost of outstanding repair”.
Planning permission is being considered by the Council, to turn the listed building into student accommodation.
But it has divided opinion.
Residents say local traffic, architectural value and trees are all factors to be considered in the reconstruction.
Labour Councillor, Ramesh Patel, says that developing the property into student apartments will seriously affect congestion around the building.
“I don’t believe the transport people or the highways agency have looked at it properly.
“It’s the problems with the parking and the traffic, that it will generate. The exit from Llandaf Road will have queues of cars lining up”.
Suffolk House Campaigner, Ceri Williams, says that Cardiff Council needs to change its policy, so there is more control over what happens to unused properties.
“It does seem to be part of a wider policy that Cardiff Council have to be very pro-development, very pro-business and don’t really listening to residents about what type of city they want to live in.
“It is important to remember that it used to be a council property”.
Some of the people of Canton have held public meetings in opposition to the move about protecting the trees around the building which are potentially being cut down.
A document published by them refers to “ancient and magnificent Copper Beech, Limes, Holm Oak and Sycamore trees”.
In a statement Cardiff Council have pledged to plant more trees.
But Suffolk House Campaigner, Joy Williams, says such saplings are not as effective as the trees that have been there for hundreds of years.
“They won’t be able to take in as much Carbon Dioxide as these trees do around Suffolk House and they will take time to grow.
“Why not keep something so precious which is doing a wonderful job for us, instead of planting saplings”.
Whilst Suffolk House has a deep-rooted history in Canton, its future remains unknown until the 23rd of February, when a final decision will be made.