(At Doctor Droid, we use Portkey) Portkey originally got popular for their Open Source LLM Gateway that was helping abstract 100+ LLM endpoints via one API. Post that, they started working on the LLM Observability tool, which is their current focus area.

Pricing

Its free-tier offers 10K monthly requests.

Things To Consider

Only does logging of requests and responses. It does not trace requests.

Benefits

Portkey is a proxy where you can maintain a prompt library and just pass variables in the template to call your LLM.
The tool manages all basic configurations for your integration like temperature. It gives options to cache response, create load balancing between models and setup fallbacks.

‍