To destroy a building, you need to collect resources from the building first.

Please, collect the income from the building before attempting to demolish it.

Some buildings cannot be removed, such as Warehouses and buildings that were originally present on your islands and you’ve restored them.