Size does matter, just not in the way you think it does. Your height is a legitimate consideration when selecting a bicycle, but it is not necessarily the most important. You must weigh the size of the frame against your height, but just because the sticker on the frame says the bike fits you doesn't make it so.
Up until Japanese manufacturers took serious aim at the western market in the early 1980s, bicycle sizing was largely unsophisticated. According to the late bicycling expert Sheldon Brown, only two dimensions of a bike's frame changed with size -- the head tube and seat tube. The seat tube -- the primary concern -- extends from the seat post to the bottom bracket, which is the portion of the frame that holds the crank arms. The crank arms accept the pedals. As Brown notes, all you needed to do was stand over a frame. If there was room between yourself and the top tube you were good to go. Larger size bikes simply had larger seat tubes. While the focus on height is still real, bike companies now produce "proportional-sized frames" that take into account factors other than how tall you are.
If you talk to ten different bike "experts," you are likely to walk away with several methods for calculating the correct frame size for your height. Bicycling Magazine's Joe Lindsey details one popular approach. Lindsey advises standing in your socks with a thin hardcover book, spine facing up, between your legs, against your groin. Measure your in-seam from the spine of the book to the floor in centimetres. Multiply the result by 0.65 to get your road bike frame size. Subtract 10 to 12 cm (4 to 5 inches) for your mountain bike equivalent. Use this number, however, as just one piece of information.
When you come to your "magic number" using Lindsey's scheme, it will likely fall into a range given by a manufacturer for one of their bikes. Realise that these numbers are based on averages. They do not take into account individual body styles and proportions. The average, however, is a good place to start. Trek's sizing system is representative of the industry. For a road bike, generally you ride a 43 or 47 cm (16.9 to 18.5 inches) if you are less than 158 cm (5 feet 2 inches) tall. At 160 cm (5 feet 3 inches), you'll ride a 47 or 50 cm (18.5 to 19.7 inches). At 168 cm (5 feet 6 inches), look between 50 and 54 cm (19.7 to 21.3 inches). At 175 cm (5 feet 9 inches), consider frame sizes ranging from 52 to 56 cm (20.5 to 22 inches). As you approach and surpass 183 cm (6 feet) tall, you will, typically, need between a 56 and 62 cm (22 to 24.4 inches) frame. See Trek's website for your mountain bike average.
As Brown notes, the length of your top tube -- the bar that extends from the handlebar to the seat post -- is equally as, if not more, important than the bike's vertical height, measured by the seat tube. An individual who is 188 cm (6 feet 2 inches) tall with unusually short legs might require a smaller frame than someone under 183 cm (6-foot) with long legs. The larger torso on the 188 cm (6 feet 2 inch) person, however, calls for a longer top tube. Since frame geometries vary from bike to bike, you might need to shop around or tweak other parts on a bike to get the right fit.
As Trek explains, height is just a starting point when sizing kids' bikes as well. Kids' bikes are not sized by the frame. Bike companies use the diameter of the wheel for this purpose. For instance, a 30.5 cm (12-inch) bike tends to be suitable for children between 89 and 102 cm (35 and 40 inches) tall, while a 61 cm (24-inch) bike, the largest kid's size, generally fits youngsters who are between 130 and 160 cm (51 and 63 inches) tall.