Speech Recognition - Ready for Prime Time?
by Jarred Walton on April 21, 2006 9:00 AM EST- Posted in
- Smartphones
- Mobile
Processor Utilization - Rapid Speech
Accuracy was obviously far lower with the faster, more casual delivery. What happens with processor utilization?
Dictation Processor Utilization
Transcription Processor Utilization
Processor requirements go way up with a faster speech delivery, at least with Dragon NaturallySpeaking. Perhaps it's my lack of enunciation when I'm speaking fast, but obviously you end up with not only lower accuracy, but also longer speech recognition times.
When you consider that our test system was able to keep up with our regular delivery when dictating (barely), the fact that it requires 15.7 minutes on maximum accuracy for an 8:11 length file shows how important "user training" really is. When dictating at 120 to 130 wpm, both software packages are able to keep up. At 146 wpm, maximum accuracy ends up processing only 77 words per minute. When you throw in the increased number of errors, rapid delivery is definitely not the preferred way to utilize either of these packages. If "Natural" means "Fast" to you, you might want to mentally rename DNS to Dragon "Enunciate-And-Speak-Slower" Speaking.
Transcribe mode is also slower, but it doesn't take nearly as big of a hit. Also interesting is that both the transcription mode and the dictation mode manage to max out processor usage with Dragon NaturallySpeaking but live dictation takes 2 to 5 times as much CPU time. Clearly, the full speech UI is putting a decent load on the CPU. That makes sense, as continually trying to determine whether the user has spoken a special command (i.e. trying to access the file menu) would create some overhead. Still, being two to three times slower seems a bit extreme. Of course, keep in mind that during normal writing, rarely can you speak at full speed for several minutes; normally, you'll take frequent pauses to think of exactly what you want to say next.
Accuracy was obviously far lower with the faster, more casual delivery. What happens with processor utilization?
Dictation Processor Utilization
.png) |
| DNS8 Maximum Accuracy |
.png) |
| DNS8 Medium Accuracy |
.png) |
| DNS8 Minimum Accuracy |
.png) |
| MSWord Maximum Accuracy |
.png) |
| MSWord Medium Accuracy |
.png) |
| MSWord Minimum Accuracy |
Transcription Processor Utilization
.png){kind=spacer} |
| DNS8 Maximum Accuracy |
.png){kind=spacer} |
| DNS8 Medium Accuracy |
.png){kind=spacer} |
| DNS8 Minimum Accuracy |
Processor requirements go way up with a faster speech delivery, at least with Dragon NaturallySpeaking. Perhaps it's my lack of enunciation when I'm speaking fast, but obviously you end up with not only lower accuracy, but also longer speech recognition times.
When you consider that our test system was able to keep up with our regular delivery when dictating (barely), the fact that it requires 15.7 minutes on maximum accuracy for an 8:11 length file shows how important "user training" really is. When dictating at 120 to 130 wpm, both software packages are able to keep up. At 146 wpm, maximum accuracy ends up processing only 77 words per minute. When you throw in the increased number of errors, rapid delivery is definitely not the preferred way to utilize either of these packages. If "Natural" means "Fast" to you, you might want to mentally rename DNS to Dragon "Enunciate-And-Speak-Slower" Speaking.
Transcribe mode is also slower, but it doesn't take nearly as big of a hit. Also interesting is that both the transcription mode and the dictation mode manage to max out processor usage with Dragon NaturallySpeaking but live dictation takes 2 to 5 times as much CPU time. Clearly, the full speech UI is putting a decent load on the CPU. That makes sense, as continually trying to determine whether the user has spoken a special command (i.e. trying to access the file menu) would create some overhead. Still, being two to three times slower seems a bit extreme. Of course, keep in mind that during normal writing, rarely can you speak at full speed for several minutes; normally, you'll take frequent pauses to think of exactly what you want to say next.
Facebook
Twitter
RSS
38 Comments
View All Comments
JarredWalton - Friday, April 21, 2006 - link
That's definitely true -- if you look at how accuracy scales with CPU usage, doubling and even tripling the processor time comes with only incremental increases in accuracy. I do have to say that I noticed it being a little sluggish on my single core system when I was multitasking, but obviously I push my computers a little harder than a lot of people. Depending on what you're willing to live with in terms of speed, I'm sure both Dragon and Microsoft speech recognition can work on a Pentium III level system.LanceM - Friday, April 21, 2006 - link
So is that selection typical Asimov? If so, it has convinced me to never bother reading any of his works.His ideas/plots/etc. may be interesting, but I don't think I could handle phrases like, "as if she were some dried-up, old-maid teacher." Give me Joseph Conrad or William Faulkner.
Dfere - Monday, April 24, 2006 - link
Asimov is classic Sci-Fi- pulp, which usually had a gritty detective-novel appeal. Hs works are in large part murder mystery type novels. You have to understand the nature of the literature, the history and the author. I don't think a critique is deserved until then.Most Sci Fi writers of any ability first master imaginative concepts and apply them, even Drke and Sirling.
I give Kudos to the staff for including literary comments, the poster who said this should not be a book of the month club lives a very one dimensional life.
Shoal07 - Friday, April 21, 2006 - link
What makes Asimov special is many of his ideas in sci fiction are comming true today or are atleast on the horizon. Asimov shaped the way many of us picture the future.goinginstyle - Friday, April 21, 2006 - link
Why does the Anandtech staff revert to literary quotes in their reviews now? This is a computer website, not a book club.JarredWalton - Friday, April 21, 2006 - link
I read Asimov's foundation series as a teenager, and I loved it. He gave me lots of fanciful dreams about where technology might go in the future, and even though some of the writing styles have changed over the years, I still find a lot of these old sci-fi books to be entertaining. You should try reading War of the Worlds if you think that quote was bad. LOLSorry if some of you didn't like the quote. Everyone has their own dislikes and likes, but in the end it's just an introduction. I hope to one day be able to yell at my computer and have it properly understand what I say, as well as the context (i.e., yelling means something is going wrong, and maybe it can help me out). Will we ever get there? Probably some day, but whether it happens in our lifetimes or not is anyone's guess.
NegativeEntropy - Saturday, April 22, 2006 - link
I like the use of quotes -- though it does remind me a bit of being in English/writing class ("Always do something in the introduction to get your audience's attention...").On the subject of "classic" Sci-fi writers, I also still enjoy old school Heinlein. Though his characters can get a bit repetitive across his pile of works, many of the science ideas are still valid (and I share much of his apparent personal philosophy).
On the actual article -- thanks for doing it. I have been curious where this technology was at in terms of every day usage and hardware requirements.
Regarding CPU usage, it's possible DNS attempts to use whatever resources are available based on preferences. i.e. on minimum, it attempts to impact the system minimally, regardless of the CPU resources available; say 25% on min, 50% on med and 95% on max with the percentage staying relatively consistent from a P3 1GHz to an A64 2.6GHz. This would explain its reported good scaling from system to system. If you want to test it, underclock your A64 system to half its frequency and compare utilization at the medium setting.
kristof007 - Friday, April 21, 2006 - link
Here at Anandtech you can always count on to find something else. Great article! I tried out speech recognition a few years back and I got frustrated with it over one thing or another so I just dropped it and went back to typing. I've been typing for about 8 years now. I never learned the "proper" way to type where every finger has a spot. Anyway I hope Vista will make speech recognition WAAY better so that it could be used around the OS AND for speech recognition.Thanks for the article!