I was told by an ENT that I had a deviated septum and should get surgery to correct it.

When I told my primary care doctor that, he said rather firmly that many, many people had deviated septums and it was not a big deal. That surgery was not necessary and that I should take nasal inhalers that cleared my nasal passages - since I was a snorer at night. Is it a lifestyle surgery, or something that I should get done? I realize it depends on the severity of my deviation, but curious what some of the risks may be if I do undergo surgery. Do most people get the surgery done? I am a snorer at night. I am of normal weight. I don't take any medications other than nexium.