Access control
Braintrust has a robust and flexible access control system. You can grant permissions to users or service accounts either at the organization level or on specific Braintrust objects (projects, experiments, logs, datasets, prompts, and playgrounds).
Permission groups
The core concept of Braintrust's access control system is the permission group. Permission groups are collections of users that can be granted specific permissions. Braintrust has three pre-configured Permission Groups that are scoped to the organization.
- Owners - Unrestricted access to the organization, its data, and its settings. Can add, modify, and delete projects and all other resources. Can invite and remove members and can manage group membership.
- Engineers - Can access, create, update, and delete projects and all resources within projects. Cannot invite or remove members or manage access to resources.
- Viewers - Can access projects and all resources within projects. Cannot create, update, or delete any resources. Cannot invite or remove members or manage access to resources.
If your access control needs are simple and you do not need to restrict access to individual projects, these ready-made permission groups may be all that you need.
A new user can be added to one of these three groups when you invite them to your organization.
Creating custom permission groups
In addition to the built-in permission groups, it's possible to create your own groups as well. To do so, go to the 'Permission groups' page of Settings and click on the 'Create permission group' button. Give your group a name and a description and then click 'Create'.
To set organization-level permissions for your new group, find the group in the groups list and click on the Permissions button.
The 'Manage Access' permission should be granted judiciously as it is a super-user permission.
It gives the user the ability to add and remove permissions, thus any user with 'Manage Access' gains the ability to grant all other permissions to themselves.
The 'Manage Settings' permission grants users the ability to change organization-level settings like the API URL.
To set group-level permissions for your new group, i.e. who can read, delete, and add members to this group, find the group in the groups list and click on the Group access button.
Project scoped permissions
To limit access to a specific project, create a new permission group from the Settings page.
Navigate to the Configuration page of that project, and click on the Permissions link in the context menu.
Search for your group by typing in the text input at the top of the page, and then click the pencil icon next to the group to set permissions.
Set the project-level permissions for your group and click Save.
Object scoped permissions
To limit access to a particular object (experiment, dataset, or playground) within a project, first create a permission group for those users on the 'Permission groups' section of Settings.
Next, navigate to the Configuration page of the project that holds that object and grant the group 'Read' permission at the project level.
This will allow users in that group to navigate to the project in the Braintrust UI.
Finally, navigate to your object and select Permissions from the context menu in the top-right of that object's page.
Find the permission group via the search input, and click the pencil icon to set permissions for the group.
Set the desired permissions for the group scoped to this specific object.
Service accounts and service tokens
Service accounts are designed for system integrations and automation. Unlike regular user accounts, service accounts are not tied to individual people and can be assigned granular permissions for specific use cases. Service accounts can inherit permissions from groups or be granted permissions like users.
Service tokens are the authentication mechanism for service accounts. They use the bt-st- prefix to distinguish them from regular API keys (sk- prefix).
Service tokens can be used anywhere API keys can be used in the SDK, AI proxy, and API requests.
You must be in the Owner group of your organization to manage service accounts and service tokens.
API support
To automate the creation of permission groups and their access control rules, you can use the Braintrust API. For more information on using the API to manage permission groups, check out the API reference for groups and for permissions.