We use a diffuse interface method for solving Poisson's equation with a Dirichlet condition on an embedded curved interface. The resulting diffuse interface problem is identified as a standard Dirichlet problem on approximating regular domains. We estimate the errors introduced by these domain perturbations, and prove convergence and convergence rates in the -norm, the -norm and the -norm in terms of the width of the diffuse layer. For an efficient numerical solution we consider the finite element method for which another domain perturbation is introduced. These perturbed domains are polygonal and non-convex in general. We prove convergence and convergences rates in the -norm and the -norm in terms of the layer width and the mesh size. In particular, for the -norm estimates we present a problem adapted duality technique, which crucially makes use of the error estimates derived for the regularly perturbed domains. Our results are illustrated by numerical experiments, which also show that the derived estimates are sharp.