Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue: trivialBuild should retire in favor of melpaBuild #278925

Open
AndersonTorres opened this issue Jan 5, 2024 · 28 comments
Open

Tracking issue: trivialBuild should retire in favor of melpaBuild #278925

AndersonTorres opened this issue Jan 5, 2024 · 28 comments
Labels
0.kind: bug 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: emacs

Comments

@AndersonTorres
Copy link
Member

AndersonTorres commented Jan 5, 2024

Describe the bug

According to @jian-lin the trivialBuild function has some problems that make it a bad fit for Emacs ecosystem in Nixpkgs.

In packages made via trivialBuild:

  • The builtin package manager package.el will not activate it at the Emacs' startup
    • autoloaded commands does not work out-of-the-box, requiring bolierplate code like (require 'foo-autoloads)
    • info manual becomes inaccessible viaM-x info
  • its package dir is not searched by Emacs
  • it does not contain a foo-pkg.el file

References:


Add a 👍 reaction to issues you find important.

@jian-lin
Copy link
Contributor

jian-lin commented Jan 5, 2024

I think we should:

  • make reviewers and contributors aware that trivialBuild has some issues and its usage is discouraged
  • replace existing trivialBuild in manual-packages with melpaBuild or elpaBuild

@AndersonTorres
Copy link
Member Author

AndersonTorres commented Jan 5, 2024

We should do this in reverse: first by looking at current code that uses trivialBuild, and only then to warn new devs.

I say this because I remember some packages fail with melpaBuild. I didn't know how to use it with sunrise-commander

@AndersonTorres

This comment was marked as outdated.

@AndersonTorres
Copy link
Member Author

Example: I have tried to update sunrise-commander:

{ lib
, elpaBuild
, fetchFromGitHub
, emacs
}:

elpaBuild {
  pname = "sunrise-commander";
  ename = "sunrise-commander";
  version = "unstable-2021-09-27";

  src = fetchFromGitHub {
    owner = "sunrise-commander";
    repo = "sunrise-commander";
    rev = "16e6df7e86c7a383fb4400fae94af32baf9cb24e";
    hash = "sha256-D36qiRi5OTZrBtJ/bD/javAWizZ8NLlC/YP4rdLCSsw=";
  };

  packageRequires = [
    emacs
  ];

  meta = {
    homepage = "https://github.com/sunrise-commander/sunrise-commander/";
    description = "Orthodox (two-pane) file manager for Emacs";
    license = lib.licenses.gpl3Plus;
    maintainers = [ lib.maintainers.AndersonTorres ];
    platforms = lib.platforms.all;
  };
}

The error log:

emacs-sunrise-commander-unstable> Running phase: installPhase
emacs-sunrise-commander-unstable> Error: file-error ("Read error" "Is a directory" "/nix/store/da7c3cm3m6m0in0c6d28m25r2qc585b5-source")
emacs-sunrise-commander-unstable>   mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0xcc6be0a6510e41>))
emacs-sunrise-commander-unstable>   debug-early-backtrace()
emacs-sunrise-commander-unstable>   debug-early(error (file-error "Read error" "Is a directory" "/nix/store/da7c3cm3m6m0in0c6d28m25r2qc585b5-source"))
emacs-sunrise-commander-unstable>   insert-file-contents("/nix/store/da7c3cm3m6m0in0c6d28m25r2qc585b5-source")
emacs-sunrise-commander-unstable>   (if is-tar (insert-file-contents-literally file) (insert-file-contents file))
emacs-sunrise-commander-unstable>   (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer))
emacs-sunrise-commander-unstable>   (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
emacs-sunrise-commander-unstable>   (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
emacs-sunrise-commander-unstable>   (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
emacs-sunrise-commander-unstable>   (let ((is-tar (string-match "\\.tar\\'" file))) (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))
emacs-sunrise-commander-unstable>   elpa2nix-install-file("/nix/store/da7c3cm3m6m0in0c6d28m25r2qc585b5-source")
emacs-sunrise-commander-unstable>   (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))
emacs-sunrise-commander-unstable>   (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive)))
emacs-sunrise-commander-unstable>   (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))))
emacs-sunrise-commander-unstable>   (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive)))))
emacs-sunrise-commander-unstable>   (if (consp x1) (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))))))
emacs-sunrise-commander-unstable>   (let* ((x0 (car-safe command-line-args-left)) (x1 (cdr-safe command-line-args-left))) (if (consp x1) (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive)))))))
emacs-sunrise-commander-unstable>   (if (consp command-line-args-left) (let* ((x0 (car-safe command-line-args-left)) (x1 (cdr-safe command-line-args-left))) (if (consp x1) (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))))))))
emacs-sunrise-commander-unstable>   elpa2nix-install-package()
emacs-sunrise-commander-unstable>   command-line-1(("-l" "/nix/store/4jj63z4v1xp13rh2md053dccq920hd45-elpa2nix.el" "-f" "elpa2nix-install-package" "/nix/store/da7c3cm3m6m0in0c6d28m25r2qc585b5-source" "/nix/store/mv5i7kvw8m1siga6g3n6avbh0qzk71nz-emacs-sunrise-commander-unstable-2021-09-27/share/emacs/site-lisp/elpa"))
emacs-sunrise-commander-unstable>   command-line()
emacs-sunrise-commander-unstable>   normal-top-level()
emacs-sunrise-commander-unstable> Read error: Is a directory, /nix/store/da7c3cm3m6m0in0c6d28m25r2qc585b5-source
error: builder for '/nix/store/qzdqqh8dfy63jdnhm4n06x0pv97ap53i-emacs-sunrise-commander-unstable-2021-09-27.drv' failed with exit code 255;

@jian-lin
Copy link
Contributor

jian-lin commented Jan 5, 2024

elpaBuild expects src to be an elisp file or a tarball meeting some requirements. The requirement of src of melpaBuild is less strict than the one of elpaBuild.

Have you tried melpaBuild?

@AndersonTorres
Copy link
Member Author

I tried it a long time ago. But let's do it again:

{ lib
, emacs
, fetchFromGitHub
, melpaBuild
, writeText
}:

melpaBuild {
  pname = "sunrise-commander";
  ename = "sunrise-commander";
  version = "unstable-2021-09-27";

  src = fetchFromGitHub {
    owner = "sunrise-commander";
    repo = "sunrise-commander";
    rev = "16e6df7e86c7a383fb4400fae94af32baf9cb24e";
    hash = "sha256-D36qiRi5OTZrBtJ/bD/javAWizZ8NLlC/YP4rdLCSsw=";
  };

  packageRequires = [
    emacs
  ];

  recipe = writeText "recipe" ''
    (sunrise-commander :repo "sunrise-commander/sunrise-commander" :fetcher github :files ("*.el"))
  '';

  fileSpecs = [ "*.el" ];

  meta = {
    homepage = "https://github.com/sunrise-commander/sunrise-commander/";
    description = "Orthodox (two-pane) file manager for Emacs";
    license = lib.licenses.gpl3Plus;
    maintainers = [ lib.maintainers.AndersonTorres ];
    platforms = lib.platforms.all;
  };
}

Log:

emacs-sunrise-commander-unstable> Running phase: installPhase
emacs-sunrise-commander-unstable> Error: file-missing ("Opening input file" "No such file or directory" "/build/packages/sunrise-commander-unstable-2021-09-27.tar")
emacs-sunrise-commander-unstable>   mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0xcc6be0a6510e41>))
emacs-sunrise-commander-unstable>   debug-early-backtrace()
emacs-sunrise-commander-unstable>   debug-early(error (file-missing "Opening input file" "No such file or directory" "/build/packages/sunrise-commander-unstable-2021-09-27.tar"))
emacs-sunrise-commander-unstable>   insert-file-contents-literally("/build/packages/sunrise-commander-unstable-2021-09-27.tar")
emacs-sunrise-commander-unstable>   (if is-tar (insert-file-contents-literally file) (insert-file-contents file))
emacs-sunrise-commander-unstable>   (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer))
emacs-sunrise-commander-unstable>   (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
emacs-sunrise-commander-unstable>   (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
emacs-sunrise-commander-unstable>   (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
emacs-sunrise-commander-unstable>   (let ((is-tar (string-match "\\.tar\\'" file))) (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if is-tar (insert-file-contents-literally file) (insert-file-contents file)) (if is-tar (progn (tar-mode))) (elpa2nix-install-from-buffer)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))
emacs-sunrise-commander-unstable>   elpa2nix-install-file("/build/packages/sunrise-commander-unstable-2021-09-27.tar")
emacs-sunrise-commander-unstable>   (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))
emacs-sunrise-commander-unstable>   (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive)))
emacs-sunrise-commander-unstable>   (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))))
emacs-sunrise-commander-unstable>   (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive)))))
emacs-sunrise-commander-unstable>   (if (consp x1) (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))))))
emacs-sunrise-commander-unstable>   (let* ((x0 (car-safe command-line-args-left)) (x1 (cdr-safe command-line-args-left))) (if (consp x1) (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive)))))))
emacs-sunrise-commander-unstable>   (if (consp command-line-args-left) (let* ((x0 (car-safe command-line-args-left)) (x1 (cdr-safe command-line-args-left))) (if (consp x1) (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq package-user-dir elpa) (elpa2nix-install-file archive))))))))
emacs-sunrise-commander-unstable>   elpa2nix-install-package()
emacs-sunrise-commander-unstable>   command-line-1(("-l" "/nix/store/4jj63z4v1xp13rh2md053dccq920hd45-elpa2nix.el" "-f" "elpa2nix-install-package" "/build/packages/sunrise-commander-unstable-2021-09-27.tar" "/nix/store/6gf924kvnbpjgf7mrwmm93ih5pcjpkrr-emacs-sunrise-commander-unstable-2021-09-27/share/emacs/site-lisp/elpa"))
emacs-sunrise-commander-unstable>   command-line()
emacs-sunrise-commander-unstable>   normal-top-level()
emacs-sunrise-commander-unstable> Opening input file: No such file or directory, /build/packages/sunrise-commander-unstable-2021-09-27.tar

@jian-lin
Copy link
Contributor

jian-lin commented Jan 6, 2024

Here is a working example:

  • ename and recipe need to be change to sunrise
  • version needs to be changed
  • commit needs to have a value for now
{ lib
, emacs
, fetchFromGitHub
, melpaBuild
, writeText
}:

melpaBuild {
  pname = "sunrise-commander";
  ename = "sunrise";
  version = "20210924.1620";

  src = fetchFromGitHub {
    owner = "sunrise-commander";
    repo = "sunrise-commander";
    rev = "16e6df7e86c7a383fb4400fae94af32baf9cb24e";
    hash = "sha256-D36qiRi5OTZrBtJ/bD/javAWizZ8NLlC/YP4rdLCSsw=";
  };

  # not sure whether it is still needed when a newer[1] package-build is used
  # [1]: https://github.com/NixOS/nixpkgs/pull/276943 is in the staging branch for now
  commit = "foo";

  # probably redundant
  # packageRequires = [
  #   emacs
  # ];

  recipe = writeText "recipe" ''
    (sunrise :repo "sunrise-commander/sunrise-commander" :fetcher github)
  '';

  meta = {
    homepage = "https://github.com/sunrise-commander/sunrise-commander/";
    description = "Orthodox (two-pane) file manager for Emacs";
    license = lib.licenses.gpl3Plus;
    maintainers = [ lib.maintainers.AndersonTorres ];
    platforms = lib.platforms.all;
  };
}

@AndersonTorres
Copy link
Member Author

Nice. Now we need to wait the Staging Truckload.


  1. Where did the attribute version = "20210924.1620"; come from?
    I have grepped the installed files and it comes from nowhere except an autogenerated sunrise-pkg.el...
  2. What does fileSpecs do? It is not documented anywhere...

@jian-lin
Copy link
Contributor

jian-lin commented Jan 6, 2024

#190746 is related

@jian-lin
Copy link
Contributor

jian-lin commented Jan 6, 2024

Where did the attribute version = "20210924.1620"; come from?

This is the version format MELPA unstable uses.

(let ((time 1703448134))
  (concat (format-time-string "%Y%m%d." time t)
          (format "%d" (string-to-number
                        (format-time-string "%H%M" time t)))))

time is produced by a git command git log -n1 --first-parent --pretty='format:%H %cd' --date=unix.

I think fileSpecs has no effect since d3cea48#diff-2a37290b0927d8abb03148a01a10a84dd3a18be9be80d27c7aac6c02d8ea6841L14

@AndersonTorres
Copy link
Member Author

Some obvious findings:

@hraban
Copy link
Member

hraban commented Feb 24, 2024

Thanks for your work on this, y'all! 🙌

@jian-lin
Copy link
Contributor

I created #316107 to improve the UX of melpaBuild.

@jian-lin
Copy link
Contributor

jian-lin commented Jun 2, 2024

With #316726, the UX of melpaBuild should be on a par with elpaBuild and trivialBuild.

Given that it has fewer issues than trivialBuild and that its requirement for src is looser than elpaBuild, it should be recommended as the default builder for manual packages.

@AndersonTorres
Copy link
Member Author

Now that it reached Master, we can restart!

@hraban
Copy link
Member

hraban commented Jul 15, 2024

How do you use melpaBuild to create very simple in-line packages?

          (e.trivialBuild {
            pname = "foo";
            version = "2024-07-15";
            src = builtins.toFile "foo.el" ''(defun foo-three () 3) (provide 'foo)'';
          })

I use this to create light-weight emacs plugins from my Nix config, it's a neat way to integrate the broader system config into Emacs.

@jian-lin
Copy link
Contributor

EmacsWiki repacks (they should be dealt with separately):

@AndersonTorres What do you mean by "they should be dealt with separately"? Why and how?

@AndersonTorres
Copy link
Member Author

It is better explained with an example:

Let's pick control-lock.

Webpage: https://www.emacswiki.org/emacs/ControlLock
Download page: https://www.emacswiki.org/emacs/download/control-lock.el

There is virtually no version control.

However, there is a backup at GitHub:

https://github.com/emacsmirror/emacswiki.org/blob/master/control-lock.el

My intention is to create a "package" for this EmacsWiki mirror and do something like files = ''("control-lock.el")''.

@AndersonTorres
Copy link
Member Author

Another thing to be seen is the auto-update scripts.

@jian-lin
Copy link
Contributor

jian-lin commented Jul 21, 2024

Another thing to be seen is the auto-update scripts.

Since trivialBuild works well in that use case (autolaoding is not used by update-melpa.el) and users is unlikely to look at that file, it's fine to not change it to melpaBuild.


I am working on the last piece of this issue, i.e., writing doc for these builders.

UPDATE: I block my doc work on #330589. This is just a personal choice. If you want to write doc for the current builders, please feel free to do so.

@AndersonTorres
Copy link
Member Author

I wasn't clear. I was talking about passthru.updateScript.

@jian-lin
Copy link
Contributor

@AndersonTorres Could you switch mkDerivation to melpaBuild for cask?

@AndersonTorres
Copy link
Member Author

Yes I can. However I will need to study cask a little bit, since Cask is a commandline app.

@AndersonTorres
Copy link
Member Author

For now the only "non-conforming" packages are xapian-lite and cedille.
However Cedille is broken:

Broken due to Agda update. See

#129606 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: emacs
Projects
None yet
Development

No branches or pull requests

4 participants