In short, we actually don't get sick in cold weather.
Infectious illnesses, germs make you sick, not cold weather itself.
You have to come in contact with rhinoviruses to catch a cold. And you need to be infected with influenza viruses to contract the flu.
Rhinoviruses peak in spring and fall, and influenza viruses peak in the winter.