From the official BUGS (http://geshi.svn.sourceforge.net/viewvc/geshi/branches/RELEASE_1_0_7_STABLE/geshi-1.0.X/src/docs/BUGS?view=markup):
Only opening a < in XML will cause non-compliant output. This is a problem with the way output for this is being processed. There's no easy way to fix this (at least in the 1.0.X series), but version 1.2 will handle this correct.
BTW, while you cannot actually *see* the error in Firefox or IE, you can "see" it in Opera: The normal text after the incorrect code is erroneously blue ...
The "GeSHi update to 220.127.116.11" from 20 Jun 18:06 seems to have fixed this issue.