{"id":74436,"title":"OpenPipe: Convert expensive LLM prompts into fast, cheap fine-tuned models","tagline":"OpenPipe automatically captures your existing prompts and completions, then uses them to fine-tune a drop-in replacement. The replacement is faster, cheaper and often more accurate than your original prompt.","body":"Hi there! We’re Kyle and David, and we’re building [OpenPipe](https://openpipe.ai/). OpenPipe lets you capture your existing prompts and completions, and then use them to fine-tune a model specific to your use-case. Here’s how the process works:\n\n![uploaded image](/media/?type=post\u0026id=74436\u0026key=user_uploads/203501/750c83fe-b66d-40b8-bec3-63049b029360)\n\n## Problem: General-Purpose LLMs are slow and expensive\n\nBefore working on OpenPipe, we each ran into limitations of GPT-3.5 and GPT-4:\n\n* **David** built an app that required searching Reddit and classifying the results on user-specific dimensions. Because the set of possible results that GPT-3.5 had to classify was so large, each search cost multiple dollars!\n* **Kyle** built (and sold) a startup that involved translating official documents. It was difficult to express the specific requirements that official translations must comply with in a way that GPT-3.5 would reliably follow, but GPT-4 was too slow to provide a good user experience.\n\nWe’ve spoken with many other companies and these issues are common. Cost and latency are two major factors blocking production deployment of LLM-backed functionality.\n\n## Solution: Replace prompts with auto-deployed fine-tuned models\n\nSmall models fine-tuned on a specific prompt are highly performant and can excel at many tasks. They’re particularly good at data extraction and classification, even on tasks that need significant world knowledge.\n\nFor example, in [one project](https://github.com/OpenPipe/OpenPipe/tree/main/examples/classify-recipes) we built to classify recipes, our model was able to determine that a recipe that calls for sautéed mushrooms needs a stovetop, despite not being explicitly trained on that connection. It outperformed GPT-3.5 in classification accuracy and reached 95% of GPT-4’s performance.\n\nAnd not only does our fine-tuned model outperform GPT-3.5, it costs 50X less to run!\n\n![uploaded image](/media/?type=post\u0026id=74436\u0026key=user_uploads/203501/c4655ba3-5023-4010-9fae-ea71cc0e2ccb)\n\nWe’ve built infrastructure to make fine-tuning your own model extremely easy. The process works like this:\n\n* Develop a prototype of your LLM-powered feature using GPT-3.5 or 4 like normal.\n* Collect prompts and completions over time using our reporting SDK.\n* Once you have a few hundred to a few thousand completions recorded, kick off a training job in our UI.\n* A few hours later, your model will be ready. You can either download it and self-host, or host it with us.\n* Update your SDK’s URL to point to the new model. Keep your prompts and application code exactly the same — everything will just work!\n\n## Our Ask\n\n* If you’re prototyping or rolling out a product that is hurt by the slow responses or high costs of your existing LLM provider, talk to us.\n* If you know someone who is prototyping or rolling out AI-based functionality within a large company, link them to this post or just put them in contact with us directly.\n\nYou can reach us at [founders@openpipe.ai](mailto:founders@openpipe.ai). We’d love to help out if we can!","slug":"JMa-openpipe-convert-expensive-llm-prompts-into-fast-cheap-fine-tuned-models","created_at":"2023-08-28T14:59:50.515Z","updated_at":"2026-06-20T06:14:05.247Z","total_vote_count":99,"url":"https://www.ycombinator.com/launches/JMa-openpipe-convert-expensive-llm-prompts-into-fast-cheap-fine-tuned-models","share_image_url":"//bookface-static.ycombinator.com/assets/ycdc/yc-og-image-c440a0ad1dacfb86eeeb343717479cc54d256614449b4ef719977a0a451f8bc8.png","company":{"id":29187,"name":"OpenPipe","slug":"openpipe","url":"https://openpipe.ai/","logo":"https://bookface-images.s3.amazonaws.com/small_logos/78827e9e8ab6d62b5e66b11dffea9ae771b7d97a.png","batch":"Summer 2023","industry":"B2B","tags":["AIOps","Artificial Intelligence","Open Source"],"search_path":"https://bookface.ycombinator.com/company/29187"}}