Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copliot: Add an extension point for credentials resolution #1260

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions workspaces/copilot/.changeset/real-mice-rhyme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@backstage-community/plugin-copilot-backend': minor
---

Added plugin extension point for configuring the credentials provider.
44 changes: 44 additions & 0 deletions workspaces/copilot/plugins/copilot-backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,50 @@ These variables are used to configure the plugin and ensure it communicates with

Ensure that your GitHub integration in the Backstage configuration includes the necessary token for the `GithubCredentialsProvider` to work correctly.

If you need to have more control how credentials are provided you can use the plugin extension point to provide your own implementation of the `CopilotCredentialsProvider`.

```typescript
/* istanbul ignore file */
import { createBackendModule } from '@backstage/backend-plugin-api';
import {
copilotExtensionPoint,
CopilotCredentialsProvider,
} from '@backstage-community/plugin-copilot-backend';
import { GithubInfo } from '@backstage-community/plugin-copilot-backend';

class CredentialsProvider implements CopilotCredentialsProvider {
async getCredentials(): Promise<GithubInfo> {
/// your implementation here
}
}

export const copilotCredentialsProviderModule = createBackendModule({
pluginId: 'copilot',
moduleId: 'credentials',
register(env) {
env.registerInit({
deps: {
copilot: copilotExtensionPoint,
},
async init({ copilot }) {
copilot.useCredentialsProvider(new CredentialsProvider());
},
});
},
});
```

You then register this in your backend index.ts

```typescript
import { copilotCredentialsProviderModule } from './extension_points/copilot';

...

backend.add(copilotCredentialsProviderModule);

```

### YAML Configuration Example

```yaml
Expand Down
24 changes: 24 additions & 0 deletions workspaces/copilot/plugins/copilot-backend/api-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,26 @@ import { BackendFeatureCompat } from '@backstage/backend-plugin-api';
import { Config } from '@backstage/config';
import { DatabaseService } from '@backstage/backend-plugin-api';
import express from 'express';
import { ExtensionPoint } from '@backstage/backend-plugin-api';
import { GithubCredentials } from '@backstage/integration';
import { LoggerService } from '@backstage/backend-plugin-api';
import { SchedulerService } from '@backstage/backend-plugin-api';
import { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api';

// @public
export interface CopilotCredentialsProvider {
getCredentials(): Promise<GithubInfo>;
}

// @public
export interface CopilotExtensionPoint {
// (undocumented)
useCredentialsProvider(provider: CopilotCredentialsProvider): void;
}

// @public
export const copilotExtensionPoint: ExtensionPoint<CopilotExtensionPoint>;

// @public
const copilotPlugin: BackendFeatureCompat;
export default copilotPlugin;
Expand All @@ -20,6 +36,13 @@ export function createRouterFromConfig(
routerOptions: RouterOptions,
): Promise<express.Router>;

// @public
export type GithubInfo = {
credentials: GithubCredentials;
apiBaseUrl: string;
enterprise: string;
};

// @public
export interface PluginOptions {
schedule?: SchedulerServiceTaskScheduleDefinition;
Expand All @@ -28,6 +51,7 @@ export interface PluginOptions {
// @public
export interface RouterOptions {
config: Config;
credentialsProvider: CopilotCredentialsProvider;
database: DatabaseService;
logger: LoggerService;
scheduler: SchedulerService;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@
*/

import { ResponseError } from '@backstage/errors';
import { Config } from '@backstage/config';
import { Metric } from '@backstage-community/plugin-copilot-common';
import fetch from 'node-fetch';
import { getGithubInfo, GithubInfo } from '../utils/GithubUtils';
import {
CopilotCredentialsProvider,
GithubInfo,
} from '../utils/CopilotCredentialsProvider';

interface GithubApi {
getCopilotUsageDataForEnterprise: () => Promise<Metric[]>;
Expand All @@ -27,8 +29,8 @@ interface GithubApi {
export class GithubClient implements GithubApi {
constructor(private readonly props: GithubInfo) {}

static async fromConfig(config: Config) {
const info = await getGithubInfo(config);
static async fromConfig(credentialsProvider: CopilotCredentialsProvider) {
const info = await credentialsProvider.getCredentials();
return new GithubClient(info);
}

Expand Down
10 changes: 9 additions & 1 deletion workspaces/copilot/plugins/copilot-backend/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,12 @@
*/

export * from './service/router';
export { copilotPlugin as default } from './plugin';
export {
copilotPlugin as default,
copilotExtensionPoint,
type CopilotExtensionPoint,
} from './plugin';
export {
type CopilotCredentialsProvider,
type GithubInfo,
} from './utils/CopilotCredentialsProvider';
34 changes: 34 additions & 0 deletions workspaces/copilot/plugins/copilot-backend/src/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,32 @@
import {
coreServices,
createBackendPlugin,
createExtensionPoint,
} from '@backstage/backend-plugin-api';
import { createRouterFromConfig } from './service/router';
import {
CopilotCredentialsProvider,
DefaultCopilotCredentialsProvider,
} from './utils/CopilotCredentialsProvider';

/**
* Interface for providing credentials for accessing the Copilot API.
*
* @public
*/
export interface CopilotExtensionPoint {
useCredentialsProvider(provider: CopilotCredentialsProvider): void;
}

/**
* Extension point for providing credentials for accessing the Copilot API.
*
* @public
*/
export const copilotExtensionPoint =
createExtensionPoint<CopilotExtensionPoint>({
id: 'copliot.credentials',
});

/**
* Backend plugin for Copilot.
Expand All @@ -27,6 +51,13 @@ import { createRouterFromConfig } from './service/router';
export const copilotPlugin = createBackendPlugin({
pluginId: 'copilot',
register(env) {
let credentialsProvider: CopilotCredentialsProvider;
env.registerExtensionPoint(copilotExtensionPoint, {
useCredentialsProvider(provider: CopilotCredentialsProvider) {
credentialsProvider = provider;
},
});

env.registerInit({
deps: {
httpRouter: coreServices.httpRouter,
Expand All @@ -42,6 +73,9 @@ export const copilotPlugin = createBackendPlugin({
database,
scheduler,
config,
credentialsProvider:
credentialsProvider ??
new DefaultCopilotCredentialsProvider({ config }),
}),
);
httpRouter.addAuthPolicy({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import { DatabaseHandler } from '../db/DatabaseHandler';
import Scheduler from '../task/Scheduler';
import { GithubClient } from '../client/GithubClient';
import { DateTime } from 'luxon';
import { CopilotCredentialsProvider } from '../utils/CopilotCredentialsProvider';

/**
* Options for configuring the Copilot plugin.
Expand Down Expand Up @@ -67,6 +68,11 @@ export interface RouterOptions {
* Configuration for the router.
*/
config: Config;

/**
* Credentials provider for the router.
*/
credentialsProvider: CopilotCredentialsProvider;
}

const defaultSchedule: SchedulerServiceTaskScheduleDefinition = {
Expand Down Expand Up @@ -106,11 +112,12 @@ async function createRouter(
routerOptions: RouterOptions,
pluginOptions: PluginOptions,
): Promise<express.Router> {
const { logger, database, scheduler, config } = routerOptions;
const { logger, database, scheduler, config, credentialsProvider } =
routerOptions;
const { schedule } = pluginOptions;

const db = await DatabaseHandler.create({ database });
const api = await GithubClient.fromConfig(config);
const api = await GithubClient.fromConfig(credentialsProvider);

await scheduler.scheduleTask({
id: 'copilot-metrics',
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
/*
* Copyright 2024 The Backstage Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { Config } from '@backstage/config';
import {
DefaultGithubCredentialsProvider,
GithubCredentials,
ScmIntegrations,
} from '@backstage/integration';

/**
* Information required to access the GitHub API
*
* @public
*/
export type GithubInfo = {
credentials: GithubCredentials;
apiBaseUrl: string;
enterprise: string;
};

/**
* Interface for providing credentials for accessing the copilot API
*
* @public
*/
export interface CopilotCredentialsProvider {
/**
* Retrieve the credentials required to access the copilot API
*
* @public
*/
getCredentials(): Promise<GithubInfo>;
}

export class DefaultCopilotCredentialsProvider
implements CopilotCredentialsProvider
{
private readonly host: string;
private readonly enterprise: string;
private readonly integrations: ScmIntegrations;
private readonly credentialsProvider: DefaultGithubCredentialsProvider;

constructor(options: { config: Config }) {
const { config } = options;

this.integrations = ScmIntegrations.fromConfig(config);
this.credentialsProvider =
DefaultGithubCredentialsProvider.fromIntegrations(this.integrations);

this.host = config.getString('copilot.host');
this.enterprise = config.getString('copilot.enterprise');

if (!this.host) {
throw new Error('The host configuration is missing from the config.');
}

if (!this.enterprise) {
throw new Error(
'The enterprise configuration is missing from the config.',
);
}
}

async getCredentials(): Promise<GithubInfo> {
const githubConfig = this.integrations.github.byHost(this.host)?.config;

if (!githubConfig) {
throw new Error(
`GitHub configuration for host "${this.host}" is missing or incomplete.`,
);
}

const apiBaseUrl = githubConfig.apiBaseUrl ?? 'https://api.github.com';

const credentials = await this.credentialsProvider.getCredentials({
url: apiBaseUrl,
});
Comment on lines +89 to +91
Copy link
Author

@ScottGuymer ScottGuymer Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to change to work for most people.

For most users of github they have github.com as the host in their integrations config. The code here means that even if they configure github.com in the copilot section it will try to resolve api.github.com and fail (most people would not have this configured in their config)

So i thin kthis needs to be url: host to make it work for most people.

There is also the issue that the integrations config can be done with a github app which would not have access to the enterprise API at all. This is how we have it configured and probably most users of GH will use an app (I'm guessing).

I think that the default here could be to have a specific token in the copilot section as this token is specific to accessing the enterprise API.

@awanlin i know you suggested this change here but im not sure it works in this instance as this enterprise API is a bit specific and i dont think enterprise APIs are used anywhere else that i have seen in backstage (yet).


if (!credentials.headers) {
throw new Error('Failed to retrieve credentials headers.');
}

return {
apiBaseUrl,
credentials,
enterprise: this.enterprise,
};
}
}
Loading
Loading