Elastic Endpoint Service (EES) — is a unified entrypoint that helps to manage integration between AI providers and your application. Using this service its possible to empower your solution with the following features:
Integration Type: Sync/Async You can define how do you prefer receive responses: using webhooks, stream or sync HTTP request.
Throttling and Load Control: Define RPM (Requests Per Minute) or RPMO (Requests Per Month) to align solution with your expectations.
Retry Strategy: You can define retry behaviors, and corner scenarios.
Fallback Model: it is possible to define default fallback model to prevent unsuccessful generations.
