{"id":73893,"title":"🧪 Chatter - Dead simple LLM iteration and testing","tagline":"The Chatter platform helps companies and developers test their LLM models. ","body":"**tl;dr**  Chatter is Postman for LLMs. Our platform helps companies and developers test their LLM models Iterate on prompts, run them across model families and evaluate them against test cases – all in one place. With collaboration features, engineers can design LLM chains while QA can write test cases.\\\n—\n\nHi everyone, we’re Anish and Kasyap aka the team behind [Chatter.](https://www.trychatter.ai/) We met at UPenn and dived deep into LLMs. After working on a couple projects, we realized LLM testing was a growing issue and built Chatter.\n\n![uploaded image](/media/?type=post\u0026id=73893\u0026key=user_uploads/1383268/18ea07e1-dd28-4625-b51c-44473efcd189)\n\n**🙋‍♂️ Who is it for?**\n\nWe built Chatter for developers and companies wanting observability while building with LLMs. If you’re building LLM-powered applications, refining your prompts, and care about their accuracy/performance (incorrect responses, hallucinations, etc), then Chatter is built for you.\n\n**⚙️ How does it work?**\n\n**🏗️ Build / Iterate Fast**, **with all the features you need**\n\nMultiple foundation models, LLM chaining, formatted responses, Jinja2 prompt templating, parameter tuning, and more – all without a single line of code. When you’re done, export the call(s) to your favorite programming language.\n\n![uploaded image](/media/?type=post\u0026id=73893\u0026key=user_uploads/1383268/fe7c2027-7741-4b86-bd00-3dc970d1faf2)\n\n🧪 **Test on hundreds of inputs, with automatic evaluation**\n\nEasily build a test suite as large as you need, and run it all with a single click. Add assertions for each LLM call, with automatic evaluation using anything from regex to LLM-powered methods. Everything is versioned in one place for easy comparison.\n\n![uploaded image](/media/?type=post\u0026id=73893\u0026key=user_uploads/1383268/8e29831b-d195-4105-9c7f-0cd1f4d3fe05)\n\n🚀 **Collaborate**\n\nNeed domain experts to review results, or your customer success team to bring in user feedback? Invite anyone into shared workspaces and make sure your users benefit from your combined expertise. \n\n![uploaded image](/media/?type=post\u0026id=73893\u0026key=user_uploads/1383268/d5bee1a6-612e-40e2-af2c-0463d7af8424)\n\n**🙏 Our ask**\n\nIf you know a company in your network that’s looking to improve their process for building with LLMs, we’d love a warm intro! My email is [**anish@trychatter.ai**](mailto:anish@trychatter.ai)\n\n🎡 Try the [playground](https://platform.trychatter.ai/playground) (bonus: share your feedback with us [founders@trychatter.ai](mailto:founders@trychatter.ai))\n\nShare this post!","slug":"JDp-chatter-dead-simple-llm-iteration-and-testing","created_at":"2023-08-08T00:53:47.195Z","updated_at":"2026-06-19T20:46:13.753Z","total_vote_count":27,"url":"https://www.ycombinator.com/launches/JDp-chatter-dead-simple-llm-iteration-and-testing","share_image_url":"//bookface-static.ycombinator.com/assets/ycdc/yc-og-image-c440a0ad1dacfb86eeeb343717479cc54d256614449b4ef719977a0a451f8bc8.png","company":{"id":29004,"name":"Chatter","slug":"chatter","url":"https://chatter.anish.xyz","logo":"https://bookface-images.s3.amazonaws.com/small_logos/bf0019c6ad5bec3a367e3a88e2bc4f39705513aa.png","batch":"Summer 2023","industry":"B2B","tags":["AIOps","Developer Tools","Generative AI","SaaS"],"search_path":"https://bookface.ycombinator.com/company/29004"}}