An executor is the person(s) appointed by the decedent to make sure his or her wishes as laid out in the will are fulfilled. In other words, it's an administrative position with statutory responsibilities. The court can remove an executor on any of three grounds: incapability, disqualification or unsuitability. Each of these legal terms has its own specific standards that must be met. For example, unsuitability is the most difficult to prove, given that its measure is subjective.
File a claim of incapability against the executor. The court will require a physical or mental handicap to rule the executor incapable of fulfilling his or her duties.
Research filing a case of disqualification against the executor. However, courts will consider disqualification only if the executor has a criminal conviction (or doesn't meet the age requirement).
File a case of unsuitability against the executor. For example, look for a conflict of interest. The decedent's awareness of the conflict of interest and appointment of the executor any way would weaken this case. Furthermore, courts likely will not remove the executor unless s/he acts on the conflict of interest to the demonstrable detriment of the estate.
Prove examples of misconduct, as another example of unsuitability. Theft from the estate is an obvious example of misconduct. Less obvious conduct includes regular drunkenness and failure to obey court orders. You may want to visit the estate regularly to monitor the whereabouts of all items, especially those not specifically listed in the will, since they are the easiest to misappropriate.
Research whether an executor is meeting statutory duties, as another example of unsuitability. The law requires executor make filings and accountings of the estate. Some may not muster the meticulousness necessary to follow through with these duties.
Ask for removal on the grounds of mismanagement of the estate, as another example of unsuitability. Mismanagement can consist of both action and inaction on the part of the executor, so be thorough in considering examples of both--especially since this charge is the hardest on which to get the executor dismissed.
Contest the appointment of the executor. If the document in which the decedent appoints the executor has been superseded by a later document, or hasn't been properly signed according to state law, then this approach may be far easier than getting the executor dismissed on the grounds of unsuitability.
File your complaint against the executor with the local probate court. Which court (county vs. city) will depend on state law, but it's usually the county court.
Heirs don't need to be unanimous in wanting an executor removed. The judge decides on removal, not the heirs.