Separation of timescales in a two-layered network

We investigate a network consisting of two layers occurring in, for example, application servers, and model the first layer as a many-server Jackson network. Active servers acts as customers at the second layer, where they are served by a common CPU. This system provides a benchmark example of a layered system. Our main result shows a separation of time scales in heavy traffic: the main source of randomness occurs at the (aggregate) CPU level; the interactions between different types of nodes at the other level is shown to converge to a fixed point at a faster time scale; this also yields a state-space collapse property. Apart from these fundamental insights, we also obtain an explicit approximation for the joint law of the system which is provably accurate for heavily loaded systems, and performs numerically well for moderately loaded systems. The obtained results for the model under consideration can be applied to thread-pool dimensioning in application servers.