The field of Artificial Intelligence has been consistently advancing, with Major Language Models (LLMs) at the peak of this progress. However, scaling these models presents significant challenges in more info terms of {computeresources, storage, and infrastructure. To address these hurdles, a robust framework for efficiently managing LLM utilizatio