Be quick or be dredd

Yet another release candidate.
While discussing with @markft in issue #87 about GUI performance which has been adressed in latest RC5, we also discussed memory usage.
I've switched the standard json library to msgspec, and use msgspec.struct internal data representation, for a whopping 40% memory usage improvement in restore window.

Since msgspec is only available for Python 3.8+, there are now legacy builds using Python 3.7, and fast Python 3.9+ builds that leverage msgspec. These builds also highly improve GUI responsiveness on Linux, since they use a newer Tcl/tk implementation.

On linux, the legacy builds require glibc >= 2.17, and the newer builds require glibc >= 2.34.
On windows, the legacy x86 builds work on vista+, and the newer x64 builds work on anything windows 7+.

Features

• Added exec time to prometheus metrics
• Added cancel button for GUI execution window
• Added heartbeat logging every hour
• Symlinks and irregular files are now shown in restore GUI

Misc

• Minor GUI improvements and more clear log messages
• EV Signature is now part of compilation process

This is it (second edition)

Yet another release candidate !

But hey, restic 0.17.1 came out, with some interesting fixes, so here we are.
Also implemented two new commands (that's fairly easy now and shouldn't break anything).
Metrics and auto upgrade have been reworked as part of what we missed before going final.
Also, the Grafana Dashboard has been updated for new metrics.

Features

• Improved restore GUI UX
• Implemented recover command
• Implemented repair packs command
• Implemented config file identification via CRC32
• Metrics
  • Metrics are now sent for every action, including upgrade
  • Merged oper and state metrics into one single metric
• Updated Grafana Dashboard for new metrics

Fixes

• Fix upgrade client, since we don't ship single binaries anymore
• Don't report auto upgrades as errors when self invoked (prevents errors when upgrade server is not reachable, prefer prometheus metrics for reporting)
• Hide repository uri in logs
• Fixed snapshot date parsing in GUI which is not platform agnostic
• Fixed exclude file search expansion implemented in last rc

Misc

• Added VSS mountpoints windows_excludes
• Reworded some logs

Interim build

This is a test build to validate various enhancements and restic 0.17.1 support

The GUI, the bad and the ugly

So this is hopefully the last release candidate before RTM.

Usually, at this stage, we wouldn't add new features, but restic 0.17 made a killer feature that was quite easy to implement.
The feature allows to run commands and backup their stdout, while getting the error codes from said command (I look at you mysqldump !)
It has been implemented as backup_opts.source_type: stdin_from_command and backup_opts.stdin_from_command: your command parameters in the config file.

Another minor feature is --housekeeping CLI parameter which is just a quick shorthand for existing code.
Finally, and since we have housekeeping, a new task creation entry was necessary, in order to fully appreciate it ;)

As usual for the last RCs, CLI didn't really need a lot of love (it's used for more than 100 workloads), but GUI did.
Getting the GUI right si really a painfull experience, but the payout is enough when it comes to quickly write config files that later can be used on headless servers.

Anyway, here's the full list of changes since RC3

• Features
  • Implement --stdin-from-command restic function
  • Implement --housekeeping shorthand for --check, --forget & --prune
  • Improve task creation
    • Implement housekeeping task creation
    • Fix double task creation on linux
• GUI fixes
  • Fix slow GUI when lots of snapshots exists, thanks to @markft
  • Fix loading alternative config file, also try to load default config files with absolute paths
  • Fix prometheus labels not saving
  • Fix reloading config UX
  • Improve group inheritance by removing identical group/repo values from repo config
  • Allow the GUI to act as CLI with --run-as-cli argument
  • Improve UX by allowing non existent repos to not drop errors on first load
• Default logs to current dir for Windows
• Reintroduce --debug parameter on top of _DEBUG=true
• Check source type before launching backup

Interim build

This is a quick and dirty interim build for internal testing purposes.

The release I wanted

This third release candidate of NPBackup v3 is mostly done because we shifted from PySimpleGUI to FreeSimpleGUI framework.

Other minor issues resolved:

• Added special __all__ argument to --repo-name for group operations
• Fixed group operations when bogus repo names / group names were given
• Better error messages when missing subject for restic commands

This is it (for a better release candidate)

So TBH, last RC1 was a trainwreck when talking about the GUI experience, and should have been a beta instead of an RC.
To be fair, making the GUI has been a real pain experience to deal with, where compiled builds couldn't work because of esoteric framework code, and the framework has been forked because it changed it's licensing... Lots of fun situations.

ARM builds will be uploaded later this day.

Anyway, this RC2 fixes the following since RC1:

• Fixed linux executables built on RHEL 7 didn't work on Ubuntu and alikes, thanks to @guss77 for reporting
• Fixed environment variables weren't parsed, thanks to @BJSchmidt for reporting
• Fixed GUI inheritance didn't show up properly, thanks to @GuitarBilly for reporting
• Fixed GUI manager password didn't save on first try, thanks to @GuitarBilly for reporting
• Fixed GUI repo/group creation didn't update window
• Fixed GUI full repo URI was shown in operations view, thanks to @GuitarBilly for reporting
• Improved master password strength test
• Improved bytes/bits values reading from configuration
• Various cosmetic GUI improvements
• Updated to latest restic version 0.17
• Fixed linux task creation missed username
• Added --no-cache option
• Added manual --init command (even if backup already does init)
• exclude_files path are now tested against current path if not found

Pre-compiled binaries can be found in the assets below, or install via python -m pip install npbackup==3.0.0rc2

This is it (at least for a release candidate)

v3.0.0 first release candidate

This is a full rewrite of NPBackup v2.
The application has been splitted into GUI, a CLI and Viewer applications.
Viewer application doesn't need any configuration file, and just can connect to virtually any restic repo to view and restore files.

The GUI has been a real pain to make right, since it must run on all platforms without being a web app, the delay has been long. It took more than 500 commits and alot of sleepless nights / caffeinated days to get here ^^
Configuration format is brand new, and allows to set multiple repo settings that inherit repo group settings, making it easy to create an orchestrator instance that handles retention policies, checks and cleaning on multiple repositories.
Prometheus support has been extended for other operations than backup.
CLI has full json support via --json allowing to be used via APIs.

There are much too many new features & fixes to state here, see CHANGELOG for more.

I know the documentation isn't existing yet, but I'd love to get some feedback in order to make things right. Issue #59 is open for blockers that should make it in v3. Please create separate issues for any trouble you might encounter.

Pre-compiled binaries can be found in the assets below.
Of course you can install via python -m pip install npbackup==3.0.0rc1.post4

Cheers

Getting the GUI done

The second beta release of NPBackup 3.

• Adds missing NTP parameter for retention policies
• GUI: Moves retention policies to separate tab
• GUI: Multiple configuration fixes
• GUI: Reduces backend output overhead
• CLI: Fixes bogus return on --forget
• CLI/GUI: Adds --repo-name parameter
• CLI: Makes --dry-run acceptable for other commands than backup

Features, features, features

This is our first beta release of v3.

Beta testers are highly welcome at this point.

What is working ?

• Almost everything
  What isn't fully working but should in v3 ?
• Loading a new config, cancel loading a config
• Retention policiy (yes, I do understand the irony for the PR comming with the specifc branch name)
• pre/post script status for prometheus

What's probably not going to make it into v3 ?

• Task creation (will probably postpone this to v3.1 or so)
• Windows installer (needs full overhaul. Perhaps even a simple SFX script is enough)