The microporosity of calcium phosphate (CaP) ceramics has been shown to have an essential role in osteoinduction by CaP ceramics after ectopic implantation. Here we show that it is not the microporosity but the size of surface microstructural features that is the most likely osteogenic factor. Two tricalcium phosphate (TCP) ceramics, namely TCP-S and TCP-B, were fabricated with equivalent chemistry and similar microporosity but different sizes of surface microstructural features. TCP-S has a grain size of 0.99 ± 0.20 μm and a micropore size of 0.65 ± 0.25 μm, while TCP-B displays a grain size of 3.08 ± 0.52 μm and a micropore size of 1.58 ± 0.65 μm. In vitro, both cell proliferation and osteogenic differentiation were significantly enhanced when human bone marrow stromal cells were cultured on TCP-S without any osteogenic growth factors, compared to TCP-B ceramic granules. The possible involvement of direct contact between cells and the TCP ceramic surface in osteogenic differentiation is also shown with a trans-well culture model. When the ceramic granules were implanted in paraspinal muscle of dogs for 12 weeks, abundant bone was formed in TCP-S (21 ± 10% bone in the available space), whereas no bone was formed in any of the TCP-B implants. The current in vitro and in vivo data reveal that the readily controllable cue, i.e. the size of the surface microstructure, could be sufficient to induce osteogenic differentiation of mesenchymal stem cells, ultimately leading to ectopic bone formation in calcium phosphate ceramics.