-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore: roles management update (#119)
* Update roles-management.md * Update sidebars.js * fix user-guide.md * Update sidebars.js * Update sidebars.js * Update architecture.md * Update README.md
- Loading branch information
Showing
5 changed files
with
68 additions
and
217 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,240 +1,78 @@ | ||
| # Roles Management Guide | ||
|
|
||
| AI DIAL enables assignment of roles to Models, Applications, Addons, and Assistants to restrict the number of tokens that can be transmitted in a specific time frame. These roles and their limitations can be created in external systems and then assigned in AI DIAL's configuration. | ||
| # Roles Management for API Keys | ||
|
|
||
| This tutorial provides a guide on how to create these roles in both KeyCloak and MS Azure, and how to integrate them into AI DIAL's configuration. | ||
| > Refer to IDP Configuration to learn how to set and configure supported identity providers. | ||
| > Refer to [AI DIAL Configuration](/docs/Deployment/configuration.md#some-of-the-dynamic-parameters) to learn more abour `roles` and other parameters. | ||
| ## Introduction | ||
|
|
||
| ## KeyCloak | ||
| External systems can communicate with AI DIAL via its APIs using API keys. To authorize and authenticate API keys, you can set and configure roles and limits applicable to Models, Applications, Addons, and Assistants to restrict the number of tokens that can be transmitted in a specific time frame. | ||
|
|
||
| In this instruction, you will learn how to create roles in KeyCloak and add them to AI DIAL Models, Applications, Assistants, and Addons. | ||
| **To authenticate and authorize an API key:** | ||
|
|
||
| In KeyCloak **target realm**: | ||
| 1. AI DIAL Core config is checked to see if a particular API Key is included. | ||
| 2. If the first requirement is met, the system checks its roles and limits. In case the limit is exceeded or access to a particular requested AI DIAL resource is not specified, API Key's access is rejected. | ||
|
|
||
| 1. Create a group(s) | ||
| 2. Create role(s) | ||
| 3. Map roles to groups | ||
| 4. Assign groups to users | ||
| ## AI DIAL Core Configuration | ||
|
|
||
| In AI DIAL config: | ||
| In [AI DIAL Core dynamic settings](https://github.com/epam/ai-dial-core?tab=readme-ov-file#dynamic-settings), you can define API keys, their roles and limits. | ||
|
|
||
| 5. Assign roles to AI DIAL Models, Applications, Assistants, and Addons. | ||
| ### Define API Keys | ||
|
|
||
| AI DIAL Core receives the roles assigned to it through user access tokens (JWTs). These roles can be accessed via the path: `resource_access.<client-id>.roles` where `client-id` is a client registered in KeyCloak `realm`, e.g. `chatbot-ui`. | ||
| In the `keys` section in the `aidial.config.json` file, you can add API keys ("proxyKey1" in the example below) and assign them `project` and `role`. Note, that you can assign only roles that are defined in the `roles` section - see [further in text](#define-roles-and-limits). | ||
|
|
||
| ### Step 1: Create groups | ||
| > API Key should be a secure random key of at least 128 bit size. | ||
| > Refer to [KeyCloak Documentation](https://www.keycloak.org/docs/latest/server_admin/index.html#proc-managing-groups_server_administration_guide) to learn how to create groups. | ||
| > Refer to [AI DIAL Core](https://github.com/epam/ai-dial-core/blob/development/sample/aidial.config.json) to view the full example. | ||
| In **Groups**, you can create a hierarchy of groups through admin console, similar to this example: | ||
|
|
||
| ``` | ||
| Models | ||
| -> AWS.Bedrock | ||
| -> Stable.Diffusion | ||
| Addons | ||
| -> Wolfram | ||
| ``` | ||
|
|
||
| ### Step 2: Create roles | ||
|
|
||
| > Refer to [KeyCloak Documentation](https://www.keycloak.org/docs/latest/server_admin/index.html#con-client-roles_server_administration_guide) to learn how to create roles. | ||
| 1. In **Clients**, choose the client `chatbot-ui`. | ||
| 2. In the table **Roles**, create required roles, for example: | ||
|
|
||
| - AWS.Bedrock | ||
| - Stable.Diffusion | ||
| - Wolfram | ||
|
|
||
| ### Step 3: Map roles to groups | ||
|
|
||
| In **Groups**, do the roles mapping for each group: | ||
|
|
||
| 1. Open the tab **Role Mapping** | ||
| 2. Press the button **Assign role** and choose a role you want to assign to a group | ||
|
|
||
| ### Step 4: Assign groups to users | ||
|
|
||
| 1. In **Groups**, choose a group you want to work with | ||
| 2. In **Members**, press the button **Add member** | ||
| 3. Select a member you want to assign to a group | ||
|
|
||
| > Refer to [KeyCloak Documentation](https://www.keycloak.org/docs/latest/server_admin/index.html#proc-managing-groups_server_administration_guide) for the detailed info about group assignments. | ||
| ### Step 5: Assign KeyCloak roles to AI DIAL Core deployments | ||
|
|
||
| The last step is to assign KeyCloak roles to AI DIAL core deployments: Applications, Addons, Assistants and Models. | ||
|
|
||
| > Refer to [AI DIAL Configuration](/docs/Deployment/configuration.md#some-of-the-dynamic-parameters) to learn more. | ||
| ```yaml | ||
| "addons": { | ||
| "search": { | ||
| "endpoint": "http://localhost:7010/search", | ||
| "displayName": "Search", | ||
| "userRoles": ["keycloak-role1"] | ||
| } | ||
| } | ||
|
|
||
| ... | ||
|
|
||
| "applications": { | ||
| "app": { | ||
| "endpoint": "http://localhost:7001/openai/deployments/app/chat/completions", | ||
| "userRoles": ["keycloak-role2"] | ||
| } | ||
| } | ||
|
|
||
| ... | ||
|
|
||
| "assistant": { | ||
| "endpoint": "http://assistant.dial-development/openai/deployments/assistant/chat/completions", | ||
| "assistants": { | ||
| "assistant-1": { | ||
| "prompt": "Your system prompt.", | ||
| "addons": [ | ||
| "addon-1", | ||
| "addon-2" | ||
| ], | ||
| "userRoles": ["keycloak-role1"] | ||
| } | ||
| } | ||
| } | ||
|
|
||
| ... | ||
|
|
||
| "models": { | ||
| "chat-gpt-35-turbo": { | ||
| "type": "chat", | ||
| "endpoint" : "http://localhost:7001/v1/openai/deployments/gpt-35-turbo/chat/completions", | ||
| "upstreams": [ | ||
| {"endpoint": "http://localhost:7001", "key": "modelKey1"}, | ||
| {"endpoint": "http://localhost:7002", "key": "modelKey2"}, | ||
| {"endpoint": "http://localhost:7003", "key": "modelKey3"} | ||
| ], | ||
| "userRoles": ["keycloak-role2"] | ||
| ```json | ||
| //Example extract from aidial.config.json | ||
| "keys": { | ||
| "proxyKey1": { | ||
| "project": "Project1", | ||
| "role": "default" | ||
| }, | ||
| "proxyKey2": { | ||
| "project": "Project2", | ||
| "role": "role1" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Microsoft Azure Active Directory | ||
|
|
||
| > Refer to [Azure AD Configuration](./azure-ad-configuration.md) to learn how to enable authentication with Azure AD for AI DIAL. | ||
| ### Define Roles and Limits | ||
|
|
||
| In this instruction, you will learn how to create roles in MS Azure Active Directory and add them to AI DIAL Models, Applications, Assistants, and Addons. | ||
| In the `roles` section in the `aidial.config.json` file, you can add roles ("default" and "role1" in the example below), which you can assign to API keys. Each role can have `limits` - a number of tokens (unlimited by default) that can be transmitted in a specific time frame (`minute` or `day`) to AI DIAL Models, Applications, Addons, and Assistants. | ||
|
|
||
| Group management process is consisted of three steps: | ||
| > Refer to [AI DIAL Core documentation](https://github.com/epam/ai-dial-core?tab=readme-ov-file#dynamic-settings) to view the description of parameters. | ||
| 1. Create groups in Microsoft Azure Active Directory | ||
| 2. Include groups into JWT custom `claim` | ||
| 3. Assign roles to AI DIAL Models, Applications, Assistants, and Addons | ||
| > Note, that unless a Model, Application, Addon or Assistants is explicitly specified in the config ("chat-gpt-35-turbo" model is specified in the example below), access to it is prohibited for the API key that requested it. | ||
| The roles are provided to AI DIAL Core via user access token(JWT) by MS Azure AD and are available via the path: `groups`. | ||
| > Refer to [AI DIAL Core](https://github.com/epam/ai-dial-core/blob/development/sample/aidial.config.json) to view the full example. | ||
| ### Step 1: Create groups in Microsoft Azure Active Directory | ||
|
|
||
| 1. In **Groups**, you can create groups that you need | ||
| 2. Add members for each group | ||
|
|
||
| > Refer to [MS Documentation](https://learn.microsoft.com/en-us/entra/fundamentals/how-to-manage-groups) to view the detailed info about group management. | ||
| ### Step 2: Include groups into JWT custom claim | ||
|
|
||
| 1. In Microsoft Entra ID, open the item **App Registrations** | ||
| 2. Open the application you want to configure groups for | ||
| 3. Go to the item **Token Configuration** | ||
| 4. Press the button **Add Groups claim** and customize which groups you want to include and where (access, ID token) | ||
| 5. When customizing the claim **groups**, you can choose the option **sAMAccountName** instead of **Group ID** to include group names instead of group UUIDs in the token | ||
|
|
||
| > Refer to [MS Documentation](https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-fed-group-claims#important-caveats-for-this-functionality) to view the detailed info about configuration of group claims for applications. | ||
| > **Note**: by default, the claim **groups** is not included into the access token. You should add an application **scope** to the authorization request to obtain the claim from an access token. | ||
| > The **claim** is available in the ID token only. | ||
| > Refer to [stackoverflow](https://stackoverflow.com/questions/70542675/azure-ad-access-token-does-not-contain-groups-claim) to read more about this case. | ||
| > **Note***. The application **scope** is required to validate signature of the access token. | ||
| > Refer to [Microsoft Portal](https://learn.microsoft.com/en-us/answers/questions/318741/graphapi-cannot-validate-access-token-signature) to read more about this case. | ||
| How to create a custom **scope**: | ||
|
|
||
| 1. Go to **Expose an API** | ||
| 2. Set **Application ID URI** to something meaningful, e.g `api://chatbot-ui` | ||
| 3. Press the button **Add a scope** | ||
| 4. Fill the required fields in the form to create a new scope. Make sure **Who can consent?** is set to **Admin and users**. | ||
|
|
||
| Add the **scope** to the authorization request in `params`: | ||
|
|
||
| ``` | ||
| AzureProvider({ | ||
| clientId: process.env.AUTH_AZURE_AD_CLIENT_ID, | ||
| clientSecret: process.env.AUTH_AZURE_AD_SECRET, | ||
| tenantId: process.env.AUTH_AZURE_AD_TENANT_ID, | ||
| name: process.env.AUTH_AZURE_AD_NAME ?? DEFAULT_NAME, | ||
| authorization: { | ||
| params: { scope: 'api://chatbot-ui/Client.Consumer openid profile user.Read email offline_access' }, | ||
| }, | ||
| token: tokenConfig, | ||
| }), | ||
| ``` | ||
|
|
||
| > **Note**: if the custom `scope` is provided, the rest of `scopes` in the authorization request is ignored by Microsoft Azure AD. | ||
| > As a result, clients who have requested the access token do not have access to MS Azure Graph API for reading user info such as user picture and job title. | ||
| ### Step 3: Assign MS Azure AD groups to AI DIAL Core deployments | ||
|
|
||
| The last step is to assign Microsoft Azure AD groups to AI DIAL Core deployments: Applications, Addons, Assistants and Models. | ||
|
|
||
| > Refer to [AI DIAL Configuration](/docs/Deployment/configuration.md#some-of-the-dynamic-parameters) to learn more. | ||
| > **Note**: in the example below, MS Azure groups are mapped to AI DIAL Core user roles one to one. | ||
| ```yaml | ||
| "addons": { | ||
| "search": { | ||
| "endpoint": "http://localhost:7010/search", | ||
| "displayName": "Search", | ||
| "userRoles": ["azure-group1"] | ||
| } | ||
| } | ||
|
|
||
| ... | ||
|
|
||
| "applications": { | ||
| "app": { | ||
| "endpoint": "http://localhost:7001/openai/deployments/app/chat/completions", | ||
| "userRoles": ["azure-group2"] | ||
| ```Json | ||
| //Example extract from aidial.config.json | ||
| "keys": { | ||
| "proxyKey1": { | ||
| "project": "Project1", | ||
| "role": "default" | ||
| }, | ||
| "proxyKey2": { | ||
| "project": "Project2", | ||
| "role": "role1" | ||
| } | ||
| } | ||
|
|
||
| ... | ||
|
|
||
| "assistant": { | ||
| "endpoint": "http://assistant.dial-development/openai/deployments/assistant/chat/completions", | ||
| "assistants": { | ||
| "assistant-1": { | ||
| "prompt": "Your system prompt.", | ||
| "addons": [ | ||
| "addon-1", | ||
| "addon-2" | ||
| ], | ||
| "userRoles": ["azure-group1"] | ||
| }, | ||
| "roles": { | ||
| "default": { //role name | ||
| "limits": { | ||
| "chat-gpt-35-turbo": { //model name | ||
| "minute": "100000", //overrides the default value | ||
| "day": "10000000" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| ... | ||
|
|
||
| "models": { | ||
| "chat-gpt-35-turbo": { | ||
| "type": "chat", | ||
| "endpoint" : "http://localhost:7001/v1/openai/deployments/gpt-35-turbo/chat/completions", | ||
| "upstreams": [ | ||
| {"endpoint": "http://localhost:7001", "key": "modelKey1"}, | ||
| {"endpoint": "http://localhost:7002", "key": "modelKey2"}, | ||
| {"endpoint": "http://localhost:7003", "key": "modelKey3"} | ||
| ], | ||
| "userRoles": ["azure-group2"] | ||
| } | ||
| }, | ||
| "role1": { | ||
| "limits": { | ||
| "chat-gpt-35-turbo": {} // the default value applies - unlimited | ||
| } | ||
| }, | ||
| } | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters