One way to do this is to pass a secure token or key which is shared across the client or server.
The algorithm that generates the key is shared across both so that the server understands the key sent by the client.
This could help you get started -
Client/Server Encryption plus extras[
^].
Another way to do this could be to use an SSL certification that would be installed on the client - however, that is probably not the best approach for a Silverlight application running in a sandbox.