Olver was born in 1975 in Bristol, where he grew up in the Brislington area. His father John was a policeman and his mother Philomena worked at a cinema in Bristol. After graduating from university in 1996, he worked as a careers adviser in prisons and young offenders' institutes.