It's normal for bees and other insects to make homes in empty birdhouses. They are the ideal shelter during winter weather extremes. When the birdhouses are empty, bees will build their homes inside the four walls. When it's time for the birds to come home, and the bees have taken over, the birdhouses must be completely cleaned of all bee residue so birds can move in once again. There are a few methods of keeping bees out of birdhouses.
Rub a bar of soap on the inside of the birdhouse near the ceiling, away from the entrance.
Hang a bar of soap from a string on a tree branch near the entrance of the birdhouse.
Put a layer of Vaseline on the ceiling of the birdhouse to keep the bees away.
Place an open container of beer, or a wine cooler, near the birdhouse. The bees will be attracted to the scent of the alcohol and drown in the liquid.
Build a few bee houses and place them around the yard, away from the birdhouses. Place bowls of cut fruit near the bee houses to draw the bees away from the birdhouses.