Could you not do something like:
<Button.Content>
<StackPanel Orientation="Horizontal">
<TextBlock FontFamily="#Gilroy" FontSize="19" Text="☞">
<TextBlock.RenderTransform>
<RotateTransform Angle="90"/>
</TextBlock.RenderTransform>
</TextBlock>
<TextBlock FontFamily="#Arial" FontSize="17" Text="Login"/>
</StackPanel>
<Button.Content>
It's been a while since I've used WPF but that would be how I would have accomplished it perhaps?