A method is given to estimate the geometry and motion of a moving body surface from image sequences. To this aim a parametric model of the surface is used, in order to reformulate the problem to one of parameter estimation. After linearization of the model standard linear estimation methods can be used to estimate the parameters. The main contribution of this paper is that a method is provided to perform the linearization without specifying the model. Therefore structure from motion estimation and nonrigid body motion estimation can be performed regardless of the model.