Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add simple download progress option #12586

Merged
merged 9 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/html/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -856,6 +856,12 @@ We are using `freeze`_ here which outputs installed packages in requirements for

reqs = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze'])

To programmatically monitor download progress use the ``--progress-bar=raw`` option.
This will print lines to stdout in the format ``Progress CURRENT of TOTAL``, where
``CURRENT`` and ``TOTAL`` are integers and the unit is bytes.
If the real total is unknown then ``TOTAL`` is set to ``0``. Be aware that the
specific formatting of pip's outputs are *not* guaranteed to be the same in future versions.

If you don't want to use pip's command line functionality, but are rather
trying to implement code that works with Python packages, their metadata, or
PyPI, then you should consider other, supported, packages that offer this type
Expand Down
1 change: 1 addition & 0 deletions news/11508.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add a 'raw' progress_bar type for simple and parsable download progress reports
4 changes: 2 additions & 2 deletions src/pip/_internal/cli/cmdoptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -226,9 +226,9 @@ class PipOption(Option):
"--progress-bar",
dest="progress_bar",
type="choice",
choices=["on", "off"],
choices=["on", "off", "raw"],
default="on",
help="Specify whether the progress bar should be used [on, off] (default: on)",
help="Specify whether the progress bar should be used [on, off, raw] (default: on)",
)

log: Callable[..., Option] = partial(
Expand Down
26 changes: 26 additions & 0 deletions src/pip/_internal/cli/progress_bars.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import functools
import sys
from typing import Callable, Generator, Iterable, Iterator, Optional, Tuple

from pip._vendor.rich.progress import (
Expand All @@ -14,6 +15,7 @@
TransferSpeedColumn,
)

from pip._internal.cli.spinners import RateLimiter
from pip._internal.utils.logging import get_indentation

DownloadProgressRenderer = Callable[[Iterable[bytes]], Iterator[bytes]]
Expand Down Expand Up @@ -55,6 +57,28 @@ def _rich_progress_bar(
progress.update(task_id, advance=len(chunk))


def _raw_progress_bar(
iterable: Iterable[bytes],
*,
size: Optional[int],
) -> Generator[bytes, None, None]:
def write_progress(current: int, total: int) -> None:
sys.stdout.write("Progress %d of %d\n" % (current, total))
sys.stdout.flush()

current = 0
total = size or 0
rate_limiter = RateLimiter(0.25)

write_progress(current, total)
for chunk in iterable:
current += len(chunk)
if rate_limiter.ready() or current == total:
write_progress(current, total)
rate_limiter.reset()
yield chunk


def get_download_progress_renderer(
*, bar_type: str, size: Optional[int] = None
) -> DownloadProgressRenderer:
Expand All @@ -64,5 +88,7 @@ def get_download_progress_renderer(
"""
if bar_type == "on":
return functools.partial(_rich_progress_bar, bar_type=bar_type, size=size)
elif bar_type == "raw":
return functools.partial(_raw_progress_bar, size=size)
else:
return iter # no-op, when passed an iterator
Loading