Actually, Ogg and Vorbis are two different things.
Ogg is a container format, Vorbis is an audio spec (or codec).
As for the names (thanks to wikipedia):

"Vorbis" is named after a Discworld character, Exquisitor Vorbis in Small Gods by Terry Pratchett. The Ogg format, however, is not named after Nanny Ogg, another Discworld character; the name is in fact derived from ogging, jargon that arose in the computer game Netrek.