The multi-directional mechanical stimulation experienced by articular cartilage during motion is transferred to the chondrocytes through a thin layer of pericellular matrix around each cell; chondrocytes in turn respond by releasing matrix proteins and/or matrix-degrading enzymes. In the present study we investigated how different types of mechanical stimulation can affect a chondrocyte's phenotype and extracellular matrix (ECM) production. To this end, we employed a cartilage-on-chip system which allows exerting well-defined compressive and multi-directional mechanical stimulation on a 3D chondrocyte-laden agarose hydrogel using a thin deformable membrane and three individually addressed actuation chambers. First, the 3D chondrocyte culture in agarose responded to exposure to mechanical stimulation by an initial increase in IL-6 production and little-to-no change in IL-1β and TNF-α secretion after one day of on-chip culture. Exposure to mechanical stimulation enhanced COL2A1 (hyaline cartilage marker) and decreased COL1A1 (fibrotic cartilage) expression, this being more marked for the multi-directional stimulation. Remarkably, the production of glycosaminoglycans (GAGs), one of the main components of native cartilage ECM, was significantly increased after 15 days of on-chip culture and 14 days of mechanical stimulation. Specifically, a thin pericellular matrix shell (1–5 μm) surrounding the chondrocytes as well as an interstitial matrix, both reminiscent of the in vivo situation, were deposited. Matrix deposition was highest in chips exposed to multi-directional mechanical stimulation. Finally, exposure to mechanical cues enhanced the production of essential cartilage ECM markers, such as aggrecan, collagen II and collagen VI, a marker for the pericellular matrix. Altogether our results highlight the importance of mechanical cues, and using the right type of stimulation, to emulate in vitro, the chondrocyte microenvironment.