Cracking the Code: A Deep Dive into Open Source Language Model Deployment and Inference

This talk will include material on how we can utilize and perform inference on Open-Source Large Language Models (LLM) in different environments such as GPU instances and local machines effectively. It will also minimally include tradeoffs between open-source deployment on premises and closed-source services. We will go through deployment and creating REST APIs that users can interact with. It will include the comparison of tradeoffs between several frameworks as well.