Put on protective clothing, gloves and eye wear before starting the job of getting rid of rose bushes.
Cut back the rose bush with pruning shears, making the cut sections 2 to 3 feet long. Tie the pruned canes into manageable bundles using twine.
Remove as much of the rose bush canes as possible to better expose the shank of the rose bush. Dispose of the pruned branches before digging up the roots to clear the work area.
Dig the root ball out using a shovel. A large root system may be divided and removed in two or three sections. The root ball is made up of two types of roots--anchor roots hold the bush upright while feeder roots spread out and gather nutrients for the rose bush.
Feel around the hole where the root ball was growing to ensure no anchor roots or large feeder roots were left in the ground. Keep the protective gloves on during this process. Remove any roots left behind.
Fill the vacant hole with potting soil to avoid a depression in the yard. According to Rose Magazine, when removing an old rose bush to plant a new rose, it is necessary to use new soil for which to grow the incoming rose.