If you want to read more than 1 Byte (this answer is additional to the solution from OG) then you could/should wait until "SerialPort1.BytesToRead" has that value you want to have - perhaps with a loop or "Do ... until" ... or in the method "serialPort1_DataReceived" you check first if BytesToRead has the right value - in other case you leave the method without doing anything ...
for example :
Private Sub serialPort1_DataReceived(ByVal sender As Object, ByVal e As SerialDataReceivedEventArgs) Handles SerialPort1.DataReceived
Dim rx As Integer
rx = SerialPort1.BytesToRead
if rx >= 2 then
Dim comBuff As Byte() = New Byte(rx - 1) {}
SerialPort1.Read(comBuff, 0, rx)
tbRx.Invoke(New DisplayDelegate(AddressOf DisplayCharacter), New Object() {BytetoHex(comBuff)})
end if
End Sub