Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added conservative tokenizer for Claude 3 #146

Open
wants to merge 20 commits into
base: development
Choose a base branch
from

Commits on Sep 6, 2024

  1. Configuration menu
    Copy the full SHA
    476ae14 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b5eb9d1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ee40a84 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Configuration menu
    Copy the full SHA
    13d3dc9 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5b22aaa View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    43b4613 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    04234cb View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. chore: updated README

    adubovik committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    b7dfdb8 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f6bce34 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3a6514d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0523508 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8f9ea11 View commit details
    Browse the repository at this point in the history
  6. chore: fixed README

    adubovik committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    4cbbacf View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    c680102 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    ca99288 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. Configuration menu
    Copy the full SHA
    5eb1ff0 View commit details
    Browse the repository at this point in the history
  2. chore: updated comment

    adubovik committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    2049084 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Configuration menu
    Copy the full SHA
    d4ca3e3 View commit details
    Browse the repository at this point in the history
  2. fix: renamed truncate_prompt to compute_discarded_messages; introduce…

    …s DiscardedMessages type alias; got rid of Set[int] for the discarded messages
    adubovik committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    da2fcda View commit details
    Browse the repository at this point in the history
  3. chore: renamings

    adubovik committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    86638a2 View commit details
    Browse the repository at this point in the history