Deep learning approaches face challenges due to their resource-intensive nature and the demand for explainability in high-risk applications. This talk highlights the untapped potential of tree-based ensembles like gradient boost and random forest, which offer state-of-the-art performance and high explainability. Despite their advantages, these methods have received less attention from the computer systems research community in terms of optimizing their resource efficiency. Drawing on our experience in inference optimization since 2018, this talk will share practical strategies to enhance the efficiency of tree-based ensembles. A few potential research topics to advance the field will also be provided.