A permanent magnet contains many microscopic domains, each of them like a miniature magnet. All of these are lined up in the same orientation, so the magnet as a whole has a substantial net magnetic field. Heating the magnet to high temperatures or generating a magnetic field with an alternating current in the vicinity of the permanent magnet are two ways to demagnetise it (assuming you want to do so). The simplest way to demagnetise it, however, is with a hammer.
Put on gloves and safety glasses as it never hurts to take extra precautions.
Line the magnet up on a bench or hard surface; secure it in a vice if necessary to ensure it stays still.
Hit the magnet hard with a hammer. Make sure you really pound it with a good hard strike, not a flimsy tap. A smaller or weaker magnet may break under this kind of force, so you should only attempt this operation if the magnet is tough enough to take the blow.
Test the magnet by bringing it near the paper clip. If it has not been demagnetised to your full satisfaction, hit it again.
Make sure that you are careful when using the hammer. If the magnet shatters, it's possible that chips could fly toward you or anyone standing nearby.