If you can only use your phone on speakerphone, not only is it irritating for you, but also for all those around you. This can happen on a variety of phones, the iPhone 2G and 3G being two of them. It can also apply to some Samsungs and Motorolas.
Check you have not activated the speakerphone by mistake.
Switch off the phone, take off the back and remove the battery. Leave for a minute and replace all.
Clean the headphone insert with some alcohol and a cotton bud as the problem could have been caused by plugging in headphones and removing them. Plug the headphones in and out again a few times to see if that fixes the connection.
Do a complete restore on your phone, which take everything back to the state it was in when you bought it originally. Some phones have a reset button, others have to have the battery taken out and left for a while, and some need codes in the "Services" area of your options (BlackBerry).
Telephone the supplier, explain the problem and get the phone changed.
You can also squirt air into the headphone insert to clean it. Check your manufacturer's user guide under "Troubleshooting" to see if it has any further advice on this topic.
Do not take the battery out too often as this can upset various settings. When removing headphones, ensure you have stopped whatever it is you are playing listening to and are back to the main screen.