Quantum confinement effects in lower-dimensional derivatives of the ABX3 (A = monocation, X = halide) single perovskites afford striking optical and electronic changes, enabling applications ranging from solar absorbers to phosphors and light-emitting diodes. Halide double perovskites form a larger materials family, known since the late 1800s, but lower-dimensional derivatives remain rare and prior work has revealed a minimal effect of quantum confinement on their optical properties. Here, we synthesize three new lower-dimensional derivatives of the 3D double perovskite Cs2AgTlBr6: 2D derivatives with mono-(1-Tl) and bi-layer thick (2-Tl) inorganic sheets and a quasi-1D derivative (1'-Tl). Single-crystal ellipsometry studies of these materials show the first clear demonstration that dimensional reduction can significantly alter the optical properties of 2D halide double perovskites. This large quantum confinement effect is attributed to the substantial electronic delocalization of the parent 3D Ag-Tl perovskite. Calculations track the evolution of the electronic bands with dimensional reduction and the accompanying structural distortions and show a direct-to-indirect bandgap transition as the 3D perovskite lattice is thinned to a monolayer in 1-Tl. This bandgap transition at the monolayer limit is also evident in the calculations for 1-In, an isostructural, isoelectronic analogue to 1-Tl in which In3+ replaces Tl3+, underscoring the orbital basis for the direct/indirect nature of the bandgap. Thus, in complement to the massive compositional diversity of halide double perovskites, dimensional reduction may be used as a systematic route for harnessing electronic confinement effects and obtaining new electronic structures.