I tacked to get out of starboard's way and we still touched — who takes the penalty?

You do. On port you already had to keep clear under Rule 10, and from the moment you pass head to wind until you are close-hauled, Rule 13 makes a tacking boat keep clear of everyone. There is no gap between the two rules to hide in — and even a completed tack wouldn't save you, because Rule 15 requires a boat acquiring right of way to give the other room to keep clear.

⚡ 30-sec version

<iframe width="560" height="315" src="https://www.youtube.com/embed/-HYYm3V0bWs" title="Sailing Kiwi" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<p><a href="https://sailing.kiwi.nz/v/-HYYm3V0bWs/">Source: Sailing Kiwi</a></p>