Core24 Python init_fs in configure hook

[mcw-work]

Bug Description

Trying to update the Landscape client snap to a Core 24 base, the snap will build but when installing we hit issues with the configure hook (on a 24.04 device). This issue is similar to canonical/craft-parts#786 and was thought resolved in 8.3.2 but it appears to be still loitering around.

To Reproduce

Build Landscape client snap using the amended snapcraft.yaml below.

Environment

Build is Ubuntu 24.04/AMD64

snapcraft.yaml

name: landscape-client
base: core24
version: '24.08'
icon: snap/gui/landscape-logo-256.png
website: https://ubuntu.com/landscape
summary: Client for the Canonical systems management product Landscape
description: |
  Landcape is a web-based tool for managing Ubuntu systems. This snap, or the
  equivalent debian package is necessary if you want your machine to be managed
  in a Landscape account. It provides the Landscape client and requires a
  Landscape account.

grade: stable # must be 'stable' to release into candidate/stable channels
platforms:
  amd64:
  arm64:
  armhf:
  ppc64el:
  s390x:
confinement: strict

environment:
  LANDSCAPE_CLIENT_SNAP: 1  
  PYTHONHOME: $SNAP
  PYTHONPATH: $SNAP

plugs:
  var-lib-ubuntu-advantage-status:
    interface: system-files
    read:
      - /var/lib/snapd/hostfs/var/lib/ubuntu-advantage/status.json

apps:
  landscape-client:
    daemon: simple
    install-mode: disable
    command: usr/bin/landscape-client
    plugs:
      - hardware-observe
      - mount-observe
      - network
      - network-bind
      - network-observe
      - scsi-generic
      - shutdown
      - snapd-control
      - system-observe
      - account-control
      - process-control
      - network-control
      - network-manager
      - log-observe
      - var-lib-ubuntu-advantage-status
  config:
    command: usr/bin/landscape-config
    plugs: [network]
layout:
  /etc/landscape-client.conf:
    bind-file: $SNAP_COMMON/etc/landscape-client.conf
  /var/lib/landscape/client:
    bind: $SNAP_DATA/var/lib/landscape/client
  /var/log/landscape:
    bind: $SNAP_DATA/var/log/landscape

parts:
  landscape-client:
    plugin: python
    source: .
    stage:
      - -pyvenv.cfg
    build-packages:
      - gawk
      - libdistro-info-perl
      - lsb-release
      - net-tools
      - po-debconf
      - python3-apt
      - python3-configobj
      - python3-dev
      - python3-distutils-extra
      - python3-twisted
      - python3-pycurl
      - python3-netifaces
      - python3-yaml
      - ubuntu-advantage-tools
      - locales-all
      - python3-dbus
    override-build: |
      make build
      python3 setup.py install --prefix ${SNAPCRAFT_PRIME}/usr
    stage-packages:
      - adduser
      - bc
      - ca-certificates
      - debconf
      - libpam-modules
      - lsb-base
      - lsb-release
      - lshw
      - python3
      - python3-apt
      - python3-configobj
      - python3-gdbm
      - python3-netifaces
      - python3-pycurl
      - python3-twisted
      - python3-dbus
      - ubuntu-advantage-tools
    override-stage: |
      craftctl default
      # Copy the scripts over
      mkdir -p ${SNAPCRAFT_PRIME}/usr/bin/
      cp -r ${SNAPCRAFT_PART_SRC}/scripts/. ${SNAPCRAFT_PRIME}/usr/bin/

Relevant log output

mikec-w@mcwlaptop:~/source/landscape-client$ snap install ./landscape-client_24.08_amd64.snap --dangerous
error: cannot perform the following tasks:
- Run configure hook of "landscape-client" snap if present (run hook "configure": 
-----
Python path configuration:
  PYTHONHOME = '/snap/landscape-client/x1'
  PYTHONPATH = '/snap/landscape-client/x1'
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/snap/landscape-client/x1/lib/python3.12'
  sys._base_executable = '/usr/bin/python3'
  sys.base_prefix = '/snap/landscape-client/x1'
  sys.base_exec_prefix = '/snap/landscape-client/x1'
  sys.platlibdir = 'lib'
  sys.executable = '/usr/bin/python3'
  sys.prefix = '/snap/landscape-client/x1'
  sys.exec_prefix = '/snap/landscape-client/x1'
  sys.path = [
    '/snap/landscape-client/x1',
    '/snap/landscape-client/x1/lib/python312.zip',
    '/snap/landscape-client/x1/lib/python3.12',
    '/snap/landscape-client/x1/lib/python3.12/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0000723bac4fb540 (most recent call first):
  <no Python frame>
-----)

Additional context

No response

[tigarmo]

This probably isn't a problem with the python plugin, since you override the build completely. That PYTHONPATH and PYTHONHOME variables look wrong, is there a reason to set them to $SNAP?

[mcw-work]

Sorry - that's my fault and a mistake in copying and pasting... it should be:

  LANDSCAPE_CLIENT_SNAP: 1
  PYTHONPATH: $SNAP/usr/lib/python3/dist-packages:$SNAP/usr/lib/python3.12/site-packages:$PYTHONPATH
  PYTHONHOME: $SNAP

The pythonhome setting was a suggestion from the previous bug as without running the configure hook can't even find the internal code... i.e.

error: cannot perform the following tasks:
- Run configure hook of "landscape-client" snap if present (run hook "configure": 
-----
Traceback (most recent call last):
  File "/snap/landscape-client/x1/meta/hooks/configure", line 5, in <module>
    from landscape.client.deployment import Configuration
ModuleNotFoundError: No module named 'landscape'
-----)

For the working Core22 build, the PYTHONPATH is set as above (but 3.10 not 3.12) and PYTHONHOME is not set.

[tigarmo]

I don't think PYTHONHOME needs to be set at all

[mcw-work]

If it is not set though - you get the issue detailed above where the configure hook can't even find the landscape python modules.

Setting Pythonhome resolves this but then results in the init_fs issue.