Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REST: Standardize vended credentials used in loadTable / loadView responses #11118

Open
1 of 6 tasks
nastra opened this issue Sep 12, 2024 · 0 comments
Open
1 of 6 tasks
Assignees
Labels
proposal Iceberg Improvement Proposal (spec/major changes/etc)

Comments

@nastra
Copy link
Contributor

nastra commented Sep 12, 2024

Proposed Change

Motivation

When a table is loaded, the OpenAPI REST spec uses a generic config field in LoadTableResult to pass a table-specific configuration that includes vended credentials, which are used to configure the respective FileIO implementation.
At this moment, only the S3-related credentials are properly documented in the OpenAPI spec as can be seen here and there have been attempts to document ADLS and GCS-related configuration options by #10576. The same config field is used in LoadViewResult but lacks proper documentation.

This proposal aims at standardizing the credential-specific configurations for S3 / GCS / ADLS in the REST spec to be used for credential vending.
This proposal is also a required predecessor for providing a mechanism to refresh vended credentials, which will be handled in a follow-up proposal.

Goals

  • standardize credentials in a well-defined structure for S3 / GCS / ADLS
  • allow adding credentials for other FileIO implementations in the future

Non-Goals

  • defining credentials for other FileIO implementations
  • standardizing other configuration options for S3 / GCS / ADLS
  • refreshing vended credentials (this will be a separate proposal that depends on this proposal)

Proposal document

https://docs.google.com/document/d/1lySd_5hMZNtISLKsOvAq7xiNzdXU6TAoHF_yrOXWQvM/edit?usp=sharing

Specifications

  • Table
  • View
  • REST
  • Puffin
  • Encryption
  • Other

Spec changes: #10722

@nastra nastra added the proposal Iceberg Improvement Proposal (spec/major changes/etc) label Sep 12, 2024
@nastra nastra self-assigned this Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proposal Iceberg Improvement Proposal (spec/major changes/etc)
Projects
None yet
Development

No branches or pull requests

1 participant