Testing batteries with a voltmeter is quick and easy. The test can tell you if the problem is with the battery or with the device. The important things to keep in mind are making a good connection, reading the right scal and what it is exactly you are looking for.
Consider how much power the device needs to function. AA batteries are supposed to deliver 1.5 volts. When fresh they usually deliver slightly more than this and they continuously degenerate. Every manufacturer has an online page describing the characteristics of their batteries. Typically manufacturers consider AA batteries spent when the voltage reaches 0.75 volts.
Set your meter to DC to measure batteries. Voltmeters measure both AC and DC. You need to choose the first scale that is larger than 1.5. If the scale steps are X2, set your meter to 2V--this will work out fine. If the steps are X10, set your scale to 10V--this is going to be harder to read, especially if you have an analogue meter (with a needle instead of digital readouts). Make sure the leads are plugged into the meter correctly--red to positive and black to negative.
Hold the test leads to the ends of the battery. The red leads goes to the end marked positive, and the black lead goes to the negative terminal. The battery terminals are usually marked, but if they are not, the terminal that has a protrusion is positive and the terminal that is flat is negative. Move the leads around a little without loosing contact with the battery. The highest reading you get is the correct one.
Read the meter. If it shows 0.75 volts or lower, the battery is probably worthless. If it is around 1.5 volts, the device it came out of might need repairing or replacing. If the reading is near one volt you might have a battery that will not work for this device but is not completely worthless. Try a new battery in the device and if that doesn't work the device is the problem. Save weak batteries for emergency use in other devices, Smoke detectors can usually function with weak batteries and they will tell you loudly if the battery is too weak.
Electricity likes to flow through clean, solid connections. Sometimes just taking the batteries out of a device and cleaning both terminals with a rough but clean cloth will work miracles. Check the place where the batteries fit and make sure you are getting a solid and clean connection.
Meters are not always 100=percent accurate. Humidity and temperature can change the readings. Have your meter calibrated often.