Ok, today i've been working with Mutagen. It's definitely a feature-full package but to my dismay i've notice it's very, very flakey.
The main thing i've noticed, when coming from tagpy, is that Mutagen is extremely picky. I've come across so many instances where mp3/ogg/flac files worked fine in tagpy but not in Mutagen. I feel it's because Mutagen likes to have pristine files with no encoding errors, which frankly is a bit much.
This would be OK if there was some crude method to get around this which requires some hacking about but there's no way. It either works or it doesn't.
I am starting to have my doubts about the switch but the support of aac files, which tagpy lacks, is driving me to figure this issue out.