University of Cambridge researchers said about 676,000 deaths each year were down to inactivity, compared with 337,000 from carrying too much weight.
They concluded that getting everyone to do at least 20 minutes of brisk walking a day would have substantial benefits.
Experts said exercise was beneficial for people of any weight.
Obesity and inactivity often go hand in hand.
However, it is known that thin people have a higher risk of health problems if they are inactive. And obese people who exercise are in better health than those that do not.
The study, published in the American Journal of Clinical Nutrition, attempted to tease out the relative dangers of inactivity and obesity.
Obese v inactive
Researchers followed 334,161 Europeans for 12 years. They assessed exercise levels and waistlines and recorded every death.
“The greatest risk [of an early death] was in those classed inactive, and that was consistent in normal weight, overweight and obese people,” one of the researchers, Prof Ulf Ekelund told BBC News.
He said eliminating inactivity in Europe would cut mortality rates by nearly 7.5%, or 676,000 deaths, but eliminating obesity would cut rates by just 3.6%.
Prof Ekelund added: “But I don’t think it’s a case of one or the other. We should also strive to reduce obesity, but I do think physical activity needs to be recognised as a very important public health strategy.”