Personally, I think the drawbacks weigh more than benefits.
For starters, enemies seldom use spells, except for Presa and Muzet. And spells in this game is pretty weak overall; Rowen is awesome not because of his high damaging spells, but his rapid casting and side effects of the spells.
Secondly, absorbing spell damage is far better than negating it. By piling up equipment with elemental resistance and elemental guard skill, you can absorb a lot of (or all?) elements with ease. Rune Coat + Tactician's Hat + Elemental Cape, with all of Elemental Guard skill equipped (iirc, by linking with another character also with all Elemental Guard skill equipped will double the effect) makes you absorb the four main elements, and heavily resist the other two.
If you know your next enemy's element, you can swap the equipment out with another equipment with corresponding elemental resistance for better defense stat against physical attacks. I usually only swap them out when battling bosses, regardless of how crappy Rune Coat might be.
Furthermore, absorbing element also lets you absorb elemental artes and mystic artes, which is definitely better because I'm pretty sure Anti-Magic doesn't negate Mystic Artes.
Thirdly, the lack of healing is somewhat crippling. I think Resurrection will also fail, so there's that too. If you're maining as Rowen, then it might not be such a downside because you can still manually heal with items.
If you're going to use Anti Magic, then I suggest having him equip Krona Symbol.
Also, if you haven't beaten Maxwell yet, be sure to steal Maxwell's Elemental Cape for both fight. Sure, you can buy them eventually, but getting 2 free copies of (arguably) the best accessory after Elemental is always good.