Segregation for two granular species is studied numerically in a vertically vibrated quasi-two-dimensional (quasi-2D) box. The height of the box is smaller than two particle diameters so that particles are limited to a submonolayer. Two cases are considered: grains that differ in their density but have equal size, and grains that have equal density but different diameters, while keeping the quasi-2D condition. It is observed that in both cases, for vibration frequencies beyond a certain threshold—which depends on the density or diameter ratios—segregation takes place in the lateral directions. In the quasi-2D geometry, gravity does not play a direct role in the in-plane dynamics and gravity does not point to the segregation directions; hence, several known segregation mechanisms that rely on gravity are discarded. The segregation we observe is dominated by a lack of equipartition between the two species; the light particles exert a larger pressure than the heavier ones, inducing the latter to form clusters. This energy difference in the horizontal direction is due to the existence of a fixed point characterized by vertical motion and hence vanishing horizontal energy. Heavier and bigger grains are more rapidly attracted to the fixed point and the perturbations are less efficient in taking them off the fixed point when compared to the lighter grains. As a consequence, heavier and bigger grains have less horizontal agitation than lighter ones. Although limited by finite size effects, the simulations suggest that the two cases we consider differ in the transition character: one is continuous and the other is discontinuous. In the cases where grains differ in mass on varying the control parameter, partial segregation is first observed, presenting many clusters of heavier particles. Eventually, a global cluster is formed with impurities; namely lighter particles are present inside. The transition looks continuous when characterized by several segregation order parameters. On the other hand, when grains differ in size, there is no partial segregation and the global cluster has a much smaller concentration of impurities. The segregation order parameters change discontinuously and metastability is observed.