RubyLLM: A delightful Ruby way to work with AI

There’s a whole world of async IO in Ruby that doesn’t get enough attention.

Checkout the async gem, including async-http, async-websockets, and the Falcon web server.

https://github.com/socketry/falcon

Thank you for your kind words!

Valid point. I'm actually already working on testing better streaming using async-http-faraday, which configures the default adapter to use async_http with falcon and async-job instead of thread-based approaches like puma and SolidQueue. This should significantly improve resource efficiency for AI workloads in Ruby - something I'm not aware is implemented by other major Ruby LLM libraries. The current approach with blocks is idiomatic Ruby, but the upcoming async support will make the library even better for production use cases. Stay tuned!

From https://rubyllm.com/#have-great-conversations

    # Stream responses in real-time
    chat.ask "Tell me a story about a Ruby programmer" do |chunk|
      print chunk.content
    end

I’ve found the Ruby community really cares about DUX. Not sure why it’s not in other language communities

I was an early contributor to Langchain and it was great at first - keep in mind, that's before chat models even existed, not to mention tools, JSON mode, etc.

Langchain really, I think, pushed the LLM makers forward toward adding those features but unfortunately it got left in the dust and became somewhat of a zombie. Simultaneously, the foundational LLM providers kept adding things to turn them more into a walled garden, where you no longer needed to connect multiple things (like scraping websites with one tool, feeding that into the LLM, then storing in a vector datastore - now that's all built in).

I think Langchain has tried to pivot (more than once perhaps) but had they not taken investor $$ early on (and good for them) I suspect that it would have just dried up and the core team would have gone on to work at OpenAI, Anthropic, etc.

langchain and llamaindex are such garbage libraries: not only they never document half of the features they have, but they keep breaking their APIs from one version to the next.

Thank you! This is what the Ruby community has always prioritized - developer experience. Making complex things simple and joyful to use isn't just aesthetic preference, it's practical engineering. When your interface matches how developers think about the problem domain, you get fewer bugs and more productivity.

Would anyone happen to know of a similar library with as good DX but for JavaScript or TypeScript?

IMO, the samples look great because they're ridiculously simple.

It doesn't deal with any of the hard problems you'll routine face with implementation.

What about it is a breath of fresh air? What do the other libraries do that this doesn't?

Thanks for flagging this. The eval was only in the docs and meant only as an example, but we definitely don't want to promote dangerous patterns in the docs. I updated them.

bobby drop table, still a thing

I think it's the very nice-looking and clean high-level API that should be a pleasure to use (when it fits the job, of course).

I'm pretty sure this API semantics (instance builder to configure, and then it's ask/paint/embed with language-native way to handle streaming and declarative tools) would look beautiful and easy to use in many other languages, e.g. I can imagine a similar API - save, of course, for the Rails stuff - in Python, C# or Erlang. While this level of API may be not perfectly sufficient for all possible LLM use cases, it should certainly speed up development time when this level of API is all that's possible needed.

Is it really Ruby or they just made a nice interface? I don't see why a hypothetical TypeScript example would be all that different.

    // Just ask questions
    const chat: Chat = LLM.chat;
    chat.ask("What's the best way to learn TypeScript?");
    
    // Analyze images
    chat.ask("What's in this image?", { image: "ts_conf.jpg" });
    
    // Generate images
    LLM.paint("a sunset over mountains in watercolor style");
    
    // Create vector embeddings
    LLM.embed("TypeScript is powerful and scalable");
    
    // Let AI use your code
    class Calculator {
      description = "Performs calculations";
      params = {
        expression: { type: "string", desc: "Math expression to evaluate" },
      };
    
      execute(args: { expression: string }): string {
        return eval(args.expression).toString();
      }
    }
    
    chat.withTool(new Calculator()).ask("What's 123 * 456?");

been using https://github.com/alexrudall/ruby-openai for years with no issues which is a fine gem and works great.

Ruby has a bunch of tools to interact with LLMs already. Also has had bindings to stuff like Torch and Tensorflow for years.

What about it is so thoughtful?

That's my takeaway from Ruby - syntax matters, and good syntax makes programmers happier

I support software in many languages and ruby has to be the most syntactically stimulating.

You're confusing beautiful with simple. There's a lot of complexity and magic that's hidden behind the curtains of that "beautiful" syntax. Great for scripts and small programs, and an absolute nightmare on large projects. It's too simple.