Well established roses can have large and deep root systems. Even rose plants that have long been dead or weakened maintain a grip in the soil. The solution is to remove the surface canes, save a handle you can grasp and pull on, then dig, dig, dig. In some cases the root system will come up intact while others will have to be severed and pulled from the soil in pieces. A thorough job will require some patience.
Don your garden gloves and cut back the rose bush until there are just a few large strong canes remaining. Cut each remaining cane to about 10 to 12-inches long so it is easy to grasp with both hands.
Dig down into the soil around what used to be the drip line of the rose. Use the shovel to cut a circumference line around the main root ball. Dig down in circles until you can get your shovel under the main roots.
Use your shovel as a lever to lift the rose up and out of the soil. When sufficiently loosened by levering, grasp the canes with both hands and pull up and away using your body weight as leverage. Dig up and gather the remaining ancillary roots in the soil and discard.