Authors: Pawan Kalyan Jonnalagadda
Abstract: The development of the large language models (LLMs) has brought many new opportunities to automate the workflow involving complex computations. Nevertheless, current pipeline systems are still mostly fixed, needing to be configured manually and cannot adapt to dynamic settings. The paper suggests a new framework that is based on prompt-driven pipeline synthesis via context-aware auto-configuration of LLMs, which allows automatic synthesis and optimization of task-specific pipelines based on natural language input. The proposed approach involves a combination of timely engineering, contextually-based learning and sequence improvement to construct and adjust pipelines dynamically based on contextual information. An artificial performance model is used to determine the performance of the system relative to the current models that demonstrate that the accuracy, scalability and adaptability are improved without affecting the competitive latency. Such results demonstrate the utility of deploying LLMs as intelligent agents to optimally design pipelines as a scalable and adaptable solution to problems in the real world.
DOI: https://doi.org/10.5281/zenodo.19628685
