If an installation of recessed lights wasn't performed properly, the fixtures could be a source of drafts. This most commonly occurs when the opening is cut larger than the recessed light fixture's can. The gap between the can and the drywall allows air to flow through and even the finishing trim is unable to prevent it. Luckily, draft-proofing recessed lights is an easy process and only takes a few minutes.
Turn off the power to the recessed lights and wait for the bulbs to cool down if they were recently on.
Remove the light bulbs and use the needle-nose pliers to detach the finishing trim.
Insert a properly-sized foam insulation disc to the inside of the finishing trim.
Install the finishing trim to the recessed light fixture with the foam insulation disc situated between the trim and the drywall. It should cover the gap between the light can and the drywall completely and effectively stop drafts from leaking through.
Install the light bulbs and turn the power back on to the circuit.
Even if your recessed lights are rated airtight, draughty air can still leak around the fixture if the access hole is cut wrong. Always wear eye protection when working above your head, especially when working around cut drywall as it can produce plenty of falling debris.
Don't try to draft-proof your recessed lights by placing insulation over the top of it. If it is not an IC-rated fixture, you could be creating a fire hazard.