Exception is something which will cause your processor to skip the normal operation (i.e. it'll stop in its tracks) and jump to a pre fixed location in memory. For instance resetting some of the ARM cores will cause a 'jump' to 0x0 or to 0xFFFF0000 (depending on a setting).
So there you see, reset is just another exception albiet an important one as it is used to start from a known state (although there is not much that is available at this state and basically a lot of set up like things have to be done here).
The reason it is called a vector is because it has a certain direction. From your high school physics you may be remember that a vector is always directional and since these exceptions (including the reset) takes the program counter (in effect the processor) to a certain location in memory (hence in a certain direction) they are called vectors.
