Health Secretary Jeremy Hunt said he has a "great deal of sympathy" with patients who do not want to be treated by a doctor or nurse wearing a full-face veil.
Mr Hunt said if he was a patient he would want to see his doctor or nurse's face. But he insisted the matter was one for professionals rather than politicians to address.
He was speaking after Health Minister Dan Poulter launched a review into health service guidelines on full-face veils to ensure patients always have "appropriate face-to-face contact".
Asked whether patients have the right to demand not to be treated by a doctor or nurse wearing a veil, Mr Hunt told BBC Radio 4's Today programme: "They have the right to say that and I have a great deal of sympathy for that, but I do think this should be a professional matter and not a political matter, and that's why I think the people who should be pronouncing on this are the body responsible for professional standards and not the politicians."
Later, he told BBC Breakfast: "I think it is very important that patients do get proper contact with the doctors and nurses who are looking after them. Certainly if I was a patient myself, I'd want to be able to see the face of the doctor or nurse who was treating me. But I think this is a matter for the professional standards bodies, which is why one of my ministers has written to the GMC (General Medical Council), that regulates the standards for doctors."
Dr Poulter has ordered a review of current advice and asked regulators to devise new uniform rules.
"I am proud of the rich ethnic diversity of our healthcare workforce and support appropriate religious and cultural freedoms, but a vital part of good patient care is effective verbal and non-verbal communication," Dr Poulter told The Daily Telegraph.
"Being unable to see a healthcare professional's face can be a barrier to good and empathetic communication with patients and their families. That is why I am writing to all healthcare regulators to ask them to look into this matter and to review their professional regulations, to ensure that there is always appropriate face-to-face contact between healthcare professionals and their patients."
A ban on staff wearing the full-face veil when dealing with patients has already been introduced at 17 NHS hospitals, according to the newspaper. Home Secretary Theresa May insisted it is for women to ''make a choice'' about what clothes they wear, including veils, but said there will be some circumstances when it will be necessary to ask for them to be removed.
Earlier this week a judge ruled that a Muslim woman will be allowed to stand trial while wearing a full-face veil but must remove it while giving evidence. The ruling followed calls by Home Office Minister Jeremy Browne for a national debate on whether the state should step in to prevent young women having the veil imposed upon them.