You are not a failure as a public speaker if you use an occasional “um” or “uh.” But that doesn’t mean people who urge you to avoid them as much as you can are stupid or nit-pickers. Most people with limited experience in public speaking have no idea how many times they say these things, and are horrified to hear them when they listen to a recording of their remarks. It is very grating on the audience as well.
Slate tries to say that people who don’t use “um” or “uh” come across as too scripted, which is why such people do less well in telemarketing. But public speaking is not telemarketing. I know of numerous people who do not say “um” and who keep an audience captivated. I say, strive for the most beautiful presentation you can make.