Holly is famous as a Christmas decoration, but it is actually a versatile and popular shrub that exists on every continent except for Antarctica and Australia. There are more than 400 varieties of holly that range from 3-foot tall bushes to 70-foot tall trees. Hollies are very hardy, though they are not fond of severe over-pruning. Hollies will often survive attempts to kill them, but you can permanently remove them with effort and the proper technique.
Remove all of the branches with a saw or hedge clippers, if it's at bush or shrub height, so that you have good access to the trunk.
Cut the trunk down as close to the ground as possible -- the lower the better.
Back up 2 feet or more from the bush or hedge stump, depending on the size of the holly, and use a spade to cut off the roots around the trunk.
Dig up the trunk and the attached roots. The root system for a mature holly can be extensive, so you may have to remove the outlying roots if you wish to plant other plants in the same area.
Remove the trunk and roots from the ground for permanent removal of smaller holly bushes.
Drill multiple holes in the holly stump if the holly is too large to dig out and remove from the ground.
Place tree stump killer in the holes, following package directions. You can find this at any garden supply store.
Break up the stump down to the ground as it dies and rots. Eventually it will rot flush with the ground.
If you only cut the holly down to the ground and leave it, it will sprout new growth and continue to grow.