Loyalty is not necessarily love. Jack White describes the frustrations of being with, or wanting to be with, someone who blindly and silently sticks in a relationship. It isn't that the other doesn't say "i love you," it's that she rarely says anything else. The lack of criticism or anger comes off as a lack of interest or passion.
In the video the Malloys show the couple walking on the beach, but though they begin holding hands they end up separately framed. Jack makes his way over to her side of the duplex, but she remains coy and blank. Yet Meg's reasons for staying away are likely more complicated than he understands, and in one scene she clearly points at him while the chorus of the song plays - implying that perhaps neither of them really know what love is.