Assistive technologies for elderly often use ambient sensor systems to infer activities of daily living (ADL). In general such systems assume that only a single person (the resident) is present in the home. However, in real world environments, it is common to have visits and it is crucial to know when the resident is alone or not. We deal with this challenge by presenting a novel method that models regular activity patterns and detects visits. Our method is based on the Markov modulated Poisson process (MMPP), but is extended to allow the incorporation of multiple feature streams. The results from the experiments on nine months of sensor data collected in two apartments show that our model significantly outperforms the standard MMPP. We validate the generalisation of the model using two new data sets collected from an other sensor network.