Your reflexes and your reaction times are extremely similar. Indeed, without quick reflexes, you cannot react very fast. A variety of physiological factors can change your reaction times, and an awareness of some of these factors can help you better understand your own body.
The complexity of the task you need to reflexively carry out can have an effect on how long it takes you to do it. The more complex the task, the more parts of your system need to be activated, and the longer it takes to do. The more movements your system has to do, the slower your reaction time will be.
Heat is caused by molecules moving faster. If molecules are moving faster, chemical processes (like sending signals from your brain to your extremities) are also faster. The converse is also true, though, and colder temperatures produce slower reflexes and reaction times. The temperature itself isn't a physiological factor, but its effect on your system is.
Nerves are covered in a fatty substance called myelin. This substance makes the signals that go through your nerves stronger by insulating the nerves. The more insulated the nerves are, the clearer and faster the signals that run through them are. The more myelin you have, the faster your reflexes are. Myelin naturally begins to dwindle after about age 40, and neurological diseases such as multiple sclerosis can also cause deterioration.
Certain drugs can speed up or slow down reaction time and reflex response. For example, alcohol, as a depressant, slows down reaction time, which is part of why you should not drive after drinking. Alcohol slows down all of your brain's processes, and reactions are just one of these processes.