In the relatively short time that Artificial Intelligence has been in the public consciousness, it has transformed many industries, there is no doubt about it.
While AI can be used to do some pretty incredible things, it does have some rather sizeable limitations that the vast majority of people probably wouldn’t even realise.
Obviously, these limitations vary depending on the situation in which the AI is being used.
One such situation is what we will be talking about today, the use of AI in the production of voice overs and challenge a belief held by many, by seeing how suitable it truly is in replacing a human voice over artist.
To keep things laser focused in what can be quite a controversial topic, this blog post is going to specifically discuss the use of AI in the production of voice overs from a purely practical standpoint, i.e. we will not be discussing the ethics of the matter.
To start off let’s talk about what people expect from an AI voice over.
The vast majority, especially those not in the industry assume that an AI voice over is quick, easy, inexpensive and professional.
AI tends to promise a lot, but not quite live up to its own hype on the whole.
In reality, creating a voice over using Artificial Intelligence often involves far more work and related costs than expected and try as you might, it probably still will not quite hit that ‘professional’ descriptor.
One of the biggest challenges that comes with AI voice generation is that it needs an incredible amount of hand holding.
This is because it doesn’t truly understand language or context.
It can’t realise tricky words ahead of time and ask questions like a human would or research pronunciations.
It attempts to predict how words should sound based on patterns in its training data, which is fine when they are common words, but can stumble at other simple things such as symbols, abbreviations, technical terminology, names, locations and so on.
In a recent project we were using an AI voice over as a dummy track in an edit we were cutting.
We use dummy voice overs as a temporary placeholder track, so we have something to build an edit around before replacing it with the final voice over near the end of a project.
Hilariously, we wanted ‘3 / 4’ to be read ‘3 to 4’, and every time the AI would read it as ‘3 quarters’ – a human would easily pick this up when they read the context in which it is written, not AI however.
To get around this you end up re-writing your script to suit the voice over, and when there can be plenty more similar examples of this in the same script it can make the whole thing sound off, as you are being forced to write in a way that is not natural to you, and sometimes not even natural for a human being.
Not to mention how hard it can be to get final script sign off, the last thing you want to do is get that, go to create the voice over and realise the script will need another re-write.
Bottom line, you’re forced to think about what the software can and cannot understand, which means spending time altering a perfectly good script just to accommodate the limitations of the technology.
This feels like the antithesis of a good workflow, especially in a creative field.
Along similar lines Artificial Intelligence struggles with what we’d call a natural delivery.
Its delivery has come a long way, but it still is nowhere near what it would need to be to fool a person into thinking they were listening to a human artist.
A good example of this is that sometimes, no matter how many times you try, or how many different prompts you use, the generated voice over will struggle with the pronunciation of words it should be able to easily read, elongating random vowels or getting a sound completely wrong.
Though natural delivery is more than just pronunciation – a human performs the script, not just reads it – so, a human knows when to pause for effect, which words need emphasis and where to inject humour, emotion, urgency or subtlety.
AI simply doesn’t and can come across overly polished or emotionally flat.
Another element that impacts natural delivery is pacing and timing.
A professional voice over artist can masterfully use pacing and timing as a tool to give a certain feel to a video or an emotional resonance by very subtly speeding up/slowing down their speaking or shortening/lengthening the gaps between certain words.
They can also use it for more practical reasons like when a voice over needs to be timed to a premade video.
AI voice overs on the other hand, really struggle with it.
For obvious reasons, understanding the emotional drive to differ pacing or timing is just something AI can’t do and therefore is pretty terrible at it.
Though worse still, sometimes the AI voice will start speeding up or slowing down seemingly at random, and you can forget about automatically timing the speed of the voice over to any visuals.
What you end up having to do is creating the pacing and timing yourself by editing around the delivery the AI gives you, which 1 – takes a lot longer than if you just had a human read the script in the first place, and 2 – although will improve the generated voice, still won’t sound perfectly natural.
If Artificial Intelligence could feel emotion I’m sure it would be disappointed by its efforts.
One of our clients recently discovered the issues with using an AI voice over firsthand.
We had used one for a dummy track in their edit, planning on recording the final voice over with a human artist when we were finalising the project.
When we got to the end of the project the client asked if we could keep the AI voice over as the final version, although it did have similar issues described above, she had asked if we could clean it up as best as we could.
We spent quite a large amount of time trying to get it into a useable shape, good enough to be viewed by the public.
In the end the voice over still had certain words mispronounced, some passages that felt unnatural and other areas that didn’t flow correctly, but no matter how many times you’d run the script through the software, or even which AI software you’d use, it wasn’t sounding any better.
The client had then just assumed as the AI voice was female that they could simply record the problematic sections themselves and drop them into the finished audio, but obviously you’d have to be a master of impersonations to do this to a level where it sounded even remotely useable.
The purpose of this blog post is not merely to bash AI voice overs, but more to explain some of the lesser known difficulties that come with using them, especially when they get painted as an effortless solution.
This is why we think human voice over artists are still important and will be for a long time to come!
And that’s it, we hope you found this blog helpful and informative!
If you wanted to have a chat about anything in it or any other elements of video production, contact us today. We love chatting through ideas and we also will happily provide you with free, no obligation quotes.
