Yes and no. Think of attack and defense as units that cancel each other out. A defended team will not be harmed, but a half-defended team will be half-harmed.

Though I may be persuaded to let you defend yourself more than once over, if it's a concern.