We investigate the rotational properties of a Bose-Einstein condensate (BEC) in an effective magnetic field. The corresponding gauge potential is optically generated, and based on the adiabatic motion of the atoms. We demonstrate that the nucleation of vortices is seeded by instabilities in surface excitations and show that this picture also holds when the applied effective magnetic field is not homogeneous. The eventual configuration of vortices in the cloud depends on the geometry of the applied field. © 2009 The American Physical Society.