A Python tutorial for using Ubicloud’s managed AI Inference Endpoints.
%pip install openai --upgrade --quiet
This command installs the latest version of the OpenAI SDK, which is fully compatible with Ubicloud’s endpoints.
If you are using a terminal or a vanilla Python environment, you can run:
pip install openai --upgrade
INFERENCE_API_KEY
is used to authenticate your requests.
The MODEL
and BASE_URL
are specific to your chosen model and Ubicloud deployment.
completion.choices[0].message.content
.
stream=True
tells the API to return partial results as they become available.
The loop iterates over each chunk and prints the delta (the latest addition) of the message.
json.loads()
function converts the JSON-formatted string into a Python dictionary for further manipulation.