Business letters are an important form of communication no matter what country you work in, but Japan has turned the practice of writing a business letter into an art form. Writing a letter in the Japanese business world without showing a clear understanding of cultural protocol may inadvertently complicate your relationship with the company in question. A clearly written letter that displays a sensitivity to other cultures will speak highly of your efforts, and your company.
Opening comments are very important, and go beyond the “Dear Mr./Ms.” opening we use in America. In general, the most polite form of greeting is “Haikei”, equivalent to “Dear” but in a much more formal setting. Depending on the status of the person you're writing to, it's appropriate to use either the -san or -sama suffix, though the latter should only be used if you're being highly deferential to the recipient. A slightly less formal greeting is “Zenryaku” (translated “without an opening comment”), but this is only used when the letter will be very short, and is treated like an apology for brevity.
Closing comments are, of course, just as important as the opening comment. The commonly paired word with “Haikei” is “Keigu”, which roughly translates into the English “Sincerely”. Women will sometimes use “Kashiko” instead, as it's more formal. The word paired with “Zenryaku” is “Sousou”.
The Japanese language has different endings for root words, each dependent on the situation and to whom you're speaking. Asian languages are unique from their Western counterparts in the use of keigo, or honorific speech, to express respect for the other party. In these cases the word endings will differ, such as using the -desu ending instead of the more casual -da. Business letter etiquette dictates that you try to use the highest, most polite form possible in a given situation out of respect for the recipient.
Stock phrases are generally avoided in America, but in Japan they're actually expected. Saying something like “How are you?” may sound like a bad idea on a business letter, but even if you're writing to the VP of marketing in a large company, asking about the recipient's health or family is considered a polite and thoughtful gesture. Just remember to use the polite language form at all times, even when talking about the weather.
A meshi is a Japanese business card, and something akin to a badge of honour in the world of the Japanese salaryman. An entire ritual follows it involving the act of receiving the card with both hands, and the recipient is expected to keep the card on the table at all times during the meeting. In a business letter, if you have never met the recipient before, it's courtesy to include a meshi with the letter. This isn't necessary if you've previously met the letter's recipient.