Vertex elimination is a graph operation that turns the neighborhood of a vertex into a clique and removes the vertex itself. It has widely known applications within sparse matrix computations. We define the ELIMINATION problem as follows: given two graphs G and H, decide whether H can be obtained from G by |V(G)|−|V(H)| vertex eliminations. We show that ELIMINATION is W-hard when parameterized by |V(H)|, even if both input graphs are split graphs, and W-hard when parameterized by |V(G)|−|V(H)|, even if H is a complete graph. On the positive side, we show that ELIMINATION admits a kernel with at most 5|V(H)| vertices in the case when G is connected and H is a complete graph, which is in sharp contrast to the W-hardness of the related CLIQUE problem. We also study the case when either G or H is tree. The computational complexity of the problem depends on which graph is assumed to be a tree: we show that ELIMINATION can be solved in polynomial time when H is a tree, whereas it remains NP-complete when G is a tree.