I had it working now. I posted my question a little to early. :-)
But anyway here it the solution :
It seemed that it is necessary to send a linefeed after the binary data that needs to be send. So the SPCI command would be in it's complete form :
[SPCI COMMAND][Space][BINARY HEADER][BINARY DATA][Linefeed]
An example would be :
:SYST:SET #800083042<setup prod="InfiniiVision" ver="02.35.2013061800" srver="2.0"> ...mode data...
The defined header size must be the length of the binary data excluded the Linefeed at the end.
(header info see :
http://cp.literature.agilent.com/litweb/pdf/ads2001/vee6at/appxA10.html)
Convert the whole string then to a 8bit bytes array and send it.
I don't know if this will work for all Lxi devices but it did work for a DSO 3034A.
So what I learned is that basically you enclose the whole binary data block with header into a SPCI "string" command, that means including the linefeed at the end that indicates the end of the command.