Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add wikitext2 ppl benchmark with fixed context length in eval_llm #2547

Closed
wants to merge 12 commits into from
Closed

Add wikitext2 ppl benchmark with fixed context length in eval_llm #2547

wants to merge 12 commits into from

Commits on Dec 29, 2023

  1. added scoring method for InferenceEnginePY

    @l-k-11235
    l-k-11235 committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    dd5815f View commit details
    Browse the repository at this point in the history

  2. wip

    @l-k-11235
    l-k-11235 committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    b97253e View commit details
    Browse the repository at this point in the history

  3. copied the fix for scaled-dot masked attention

    @l-k-11235
    l-k-11235 committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    e83ffea View commit details
    Browse the repository at this point in the history

Commits on Jan 2, 2024

  1. Merge branch 'master' into enable-scoring-in-InferenceEnginePY

    @l-k-11235
    l-k-11235 committed Jan 2, 2024
    Configuration menu
    Copy the full SHA
    aa6543e View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2024

  1. moved the _score method to translator and added score_file_parallel a… 

    …nd score_lit_parallel methods to InferenceEnginePY
    @l-k-11235
    l-k-11235 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    6800c30 View commit details
    Browse the repository at this point in the history

  2. fixed file scoring

    @l-k-11235
    l-k-11235 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    edf9e06 View commit details
    Browse the repository at this point in the history

  3. added wikitext-2 benchmark

    @l-k-11235
    l-k-11235 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    32aa17f View commit details
    Browse the repository at this point in the history

  4. reformatted run_wikitext-2_benchmark.py

    @l-k-11235
    l-k-11235 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    9bb5909 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2024

  1. use a fixed window size of 512 tokens in the wikitext-2 ppl benchmark

    @l-k-11235
    l-k-11235 committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    2a53db4 View commit details
    Browse the repository at this point in the history

  2. remove config_opts in run_wikitext-2_benchmark.py

    @l-k-11235
    l-k-11235 committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    037de91 View commit details
    Browse the repository at this point in the history

  3. tokenize without dataset iterator in eval_llm/WIKITEXT2/run_wikitext-… 

    …2_benchmark.py
    @l-k-11235
    l-k-11235 committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    68f7ad7 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into enable-scoring-in-InferenceEnginePY

    @l-k-11235
    l-k-11235 committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    5b7e2bf View commit details
    Browse the repository at this point in the history