"Failed to download `gnu' archive" error in silex/emacs:25-dev #56

gonewest818 · 2020-02-17T03:30:19Z

I've recently updated the elisp-lint package and introduced a new dependency on package-lint. I'm running this linter in CircleCI using Docker images defined here.

However, tonight I'm seeing my CI tests running in the Emacs 25.3 container based silex/emacs:25-dev) are failing early with a missing dependency let-alist (which is needed for package-lint). However exactly the same tests run successfully in silex/emacs:24-dev, silex/emacs:26-dev, silex/emacs:27-dev and silex/emacs:master-dev. So I think there's something specific in the 25.3 image.

let-alist is distributed from elpa.gnu.org. I remember there was an update to the gpg key for that archive a while back. Is it possible the new key isn't installed properly for this one specific image?

$ emacs --batch -q -l .emacs/init.el
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: elpa.gnu.org:80
Failed to download `gnu' archive.
Contacting host: stable.melpa.org:443
Contacting host: melpa.org:443
Contacting host: melpa.org:443
Package refresh done
<... etc ...>

$ emacs --batch -q -l .emacs/init.el --version
GNU Emacs 25.3.1
Copyright (C) 2017 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

$ emacs --batch -q -l .emacs/init.el -l elisp-lint.el -f elisp-lint-files-batch elisp-lint.el
Loading /root/project/.emacs/.emacs-custom.el (source)...
Loading /root/project/.emacs/dependencies.el (source)...
Error (use-package): Failed to install package-lint: Package `let-alist-1.0.6' is unavailable
Error (use-package): Cannot load package-lint
Cannot open load file: No such file or directory, package-lint
Makefile:19: recipe for target 'lint' failed
make: *** [lint] Error 255

The text was updated successfully, but these errors were encountered:

Silex · 2020-02-17T20:47:26Z

Thanks for reporting! That's very weird, the dockerfiles between 25.3 and 26.3 are identical.

My guess is more that you found some bug in 25.3. Can you confirm that it also happens with silex/emacs:25.1-dev and silex/emacs:25.2-dev ?

gonewest818 · 2020-02-17T21:05:40Z

Sure, I will try the previous images and let you know.

Right now I'm trying to track down the issue by running silex/emacs:25-dev here on my laptop. At first I thought it was some kind of reoccurrence of this problem: https://lists.gnu.org/archive/html/emacs-devel/2018-02/msg00142.html . But then I noticed the log output. Emacs is actually contacting http://elpa.gnu.org on port 80

Silex · 2020-02-17T21:11:38Z

From a quick look it look at https://travis-ci.org/Silex/docker-emacs/jobs/638014344 (https://api.travis-ci.org/v3/job/638014344/log.txt) we see this:

�[0m�[91mContacting host: elpa.gnu.org:443
�[0m�[91mFailed to download `gnu' archive.
�[0m�[91mPackage refresh done
�[0m�[91mFailed to download `gnu' archive.
�[0m�[32mSuccessfully installed Cask!  Now, add the cask binary to your $PATH:
  export PATH="/root/.cask/bin:$PATH"�[0m
Removing intermediate container 6876b2805586
 ---> 93ef24cbb6d4
Step 9/10 : RUN mkdir -p /root/.emacs.d/elpa/gnupg &&     chmod 700 /root/.emacs.d/elpa/gnupg &&     gpg -q --homedir /root/.emacs.d/elpa/gnupg -k | grep 81E42C40 ||     gpg --keyserver hkp://ipv4.pool.sks-keyservers.net --homedir /root/.emacs.d/elpa/gnupg --recv-keys 066DAFCB81E42C40
 ---> Running in ff9343ff7ac3
�[91mgpg: key 066DAFCB81E42C40: 1 signature not checked due to a missing key
gpg: key 066DAFCB81E42C40: public key "GNU ELPA Signing Agent (2019) <elpasign@elpa.gnu.org>" imported
�[0m�[91mgpg: no ultimately trusted keys found
gpg: Total number processed: 1
�[0m�[91mgpg:               imported: 1
�[0mRemoving intermediate container ff9343ff7ac3

For 24.5 here's the output:

�[0m�[91mImporting package-keyring.gpg...
�[0m�[91mImporting package-keyring.gpg...done
�[0m�[91mContacting host: melpa.org:443
�[0m�[91mContacting host: elpa.gnu.org:80
�[0m�[32mSuccessfully installed Cask!  Now, add the cask binary to your $PATH:
  export PATH="/root/.cask/bin:$PATH"�[0m
Removing intermediate container d90d75fa3cf9
 ---> a4300bfbde6f
Step 9/10 : RUN mkdir -p /root/.emacs.d/elpa/gnupg &&     chmod 700 /root/.emacs.d/elpa/gnupg &&     gpg -q --homedir /root/.emacs.d/elpa/gnupg -k | grep 81E42C40 ||     gpg --keyserver hkp://ipv4.pool.sks-keyservers.net --homedir /root/.emacs.d/elpa/gnupg --recv-keys 066DAFCB81E42C40
 ---> Running in 5f943599541a
�[91mgpg: key 066DAFCB81E42C40: 1 signature not checked due to a missing key
gpg: key 066DAFCB81E42C40: public key "GNU ELPA Signing Agent (2019) <elpasign@elpa.gnu.org>" imported
�[0m�[91mgpg: no ultimately trusted keys found
�[0m�[91mgpg: Total number processed: 1
gpg:               imported: 1�[0m�[91m
�[0mRemoving intermediate container 5f943599541a

This is weird. I guess Emacs 25 does things differently.

gonewest818 · 2020-02-17T21:35:21Z

Well, one thing I'm doing differently is invoking Emacs with

 emacs --batch -q -l /root/my-project/.emacs.d/init.el

so I'm going to have to copy the new gpg public key used to sign packages because my "elpa/gnupg" directory is someplace else.

Silex · 2020-02-17T21:37:24Z

With the default value of package-archives, the image 24-dev also fails package-refresh-contents. Do you use the default value of package-archives? Maybe I should patch the images with sane defaults.

Also, a bit of history in #52

Silex · 2020-02-17T21:43:36Z

Ok, so far my tests reveal that if I use the https URL of elpa 25.3 works, but 24.5 does not work?! this is getting weirder :-)

gonewest818 · 2020-02-17T22:03:04Z

Yes, I'm using the default package-archives and I'm adding melpa and melpa stable to that.

There must be something about my init.el which is different from what you've got in /root/.emacs.d. Because if I run emacs "normally" as root I can definitely use the default http URL for elpa and install let-alist 1.0.6.

Whereas if I run

emacs --batch -q -l /root/my-project/.emacs.d/init.el

as mentioned above, and without touching gnu's URL in the package-archives, I get the "failed to download gnu archive" issue, and I can't even select let-alist from package-list-packages.

gonewest818 · 2020-02-17T22:51:31Z

I've just about got this narrowed down, and I think my issue is just because the way I'm launching emacs I don't see the updated gpg keys for ELPA in /root/.emacs.d/elpa/gnupg. Hold on, I'll post again when I confirm this.

gonewest818 · 2020-02-18T01:02:40Z

Yep, that was it.

In my CircleCI config I am setting user-emacs-directory to a different path than /root/.emacs.d and as a result, my builds didn't have access to the gnupg keys you have downloaded to /root/.emacs.d/elpa/gnupg. I tried fixing this by modifying my build to include a gpg --recv-keys with arguments similar to those found in this project's Dockerfile. However, what was causing me to doubt this approach was that downloading the keys was failing inconsistently.

What finally seems to fix this for me, is to perform the download like so:

	mkdir -p .emacs/elpa/gnupg && \
	chmod 700 .emacs/elpa/gnupg && \
	echo "disable-ipv6" > .emacs/elpa/gnupg/dirmngr.conf && \
	gpg --keyserver pool.sks-keyservers.net \
	    --homedir .emacs/elpa/gnupg \
	    --recv-keys 066DAFCB81E42C40

Using a tip from usbarmory/usbarmory-debian-base_image#9 to disable ipv6 for better reliability.

[Edit: correction, the key downloads are still laughably inconsistent. I guess I need to pick a specific keyserver rather than the round-robin DNS that seems to be happening here]

gonewest818 · 2020-02-18T01:50:52Z

Ok, done. For posterity the change I made was:

	mkdir -p .emacs/elpa/gnupg && \
	chmod 700 .emacs/elpa/gnupg && \
	echo "disable-ipv6" > .emacs/elpa/gnupg/dirmngr.conf && \
	for i in {1..3}; do \
	gpg --keyserver keyserver.ubuntu.com \
	    --homedir .emacs/elpa/gnupg \
	    --recv-keys 066DAFCB81E42C40 \
	    && break || sleep 15; \
        done

Again, I need to do this because I'm running emacs -q and overriding the user-emacs-directory to point somewhere other than /root/.emacs.d.

Silex · 2020-02-18T07:38:55Z

Thanks for debugging.

I still have weird issues. For example, by me docker run -it --rm silex/emacs:VERSION followed by M-x package-refresh-contents:

24.5: tries elpa.gnu.org:80 forever.
25.3: tries elpa.gnu.org:80 but fails.
26.3: tries elpa.gnu.org:443 but fails.

philippe@pv-desktop:~$ for i in 24.5 25.3 26.3; do echo $i; docker run -it --rm silex/emacs:$i emacs --batch --eval '(package-refresh-contents)'; done
24.5
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: elpa.gnu.org:80
25.3
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: elpa.gnu.org:80
Failed to download `gnu' archive.
26.3
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: elpa.gnu.org:443
Contacting host: elpa.gnu.org:443
Failed to download `gnu' archive.

I need to figure out what is happening, maybe the key is not installed properly.

gonewest818 · 2020-02-18T07:50:52Z

Might be because you haven't called package-initialize? Can you try this:

docker run -it --rm silex/emacs:$i emacs --batch --eval '(package-initialize)' --eval '(package-refresh-contents)'

Works for me with silex/emacs:25-dev but I don't have all the other versions available at the moment.

Silex · 2020-02-18T08:36:58Z

Ha, it never occured to me that Emacs was broken to the point that you had to call package-initialize before package-refresh-contents.

Everything works as expected then 🎉

philippe@pv-desktop:~$ for i in 24.5 25.3 26.3; do echo $i; docker run -it --rm silex/emacs:$i-dev emacs --batch --eval '(progn (package-initialize)(package-refresh-contents))'; done
24.5
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: elpa.gnu.org:80
Contacting host: elpa.gnu.org:80
25.3
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: elpa.gnu.org:80
Package refresh done
26.3
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: elpa.gnu.org:443
Contacting host: elpa.gnu.org:443
Package refresh done

The only thing I noticed is that I install Cask before fixing the ELPA key, and probably that Cask would be happier if it was the other way around.

Also, I wonder if it'd be possible to import the ELPA key system-wide so you wouldn't have to import the key yourself?

gonewest818 · 2020-02-18T16:34:17Z

Interesting. Per the manual, what you're seeing is defined behavior for --batch.

40.1.1 Summary: Sequence of Actions at Startup
"15. It calls the function package-initialize to activate any optional Emacs Lisp package that has been installed. See Packaging Basics. However, Emacs doesn't initialize packages when package-enable-at-startup is nil or when it's started with one of the options ‘-q’, ‘-Q’, or ‘--batch’. To initialize packages in the latter case, package-initialize should be called explicitly (e.g., via the ‘--funcall’ option)".

Scanning through the rest of the document you can see there are various site configuration files and hooks that run during the startup sequence, you might be able to use them arrange to initialize those gpg keys.

Silex · 2020-02-18T21:36:16Z

Thanks for the information. Emacs just ignore system wide gpg keys and looks in .emacs.d/elpa/gnupg so there's not much I can do. I just refactored a bit so it's cleaner.

dpsutton · 2020-04-25T16:52:00Z

I'm running into this issue testing inf-clojure and I'm not sure how to take the lessons above and apply them.

When running on silex/emacs:25-dev I have the following output:

cask install
 
Loading package information... done
 
Package operations: 4 installs, 0 removals
 
  - Installing [ 1/4] clojure-mode (5.11)... downloading
 
  - Installing [ 1/4] clojure-mode (5.11)... done
 
  - Installing [ 2/4] emacs (24.4)... already present
 
  - Installing [ 3/4] assess (latest)... downloading
 
Dependency assess failed to install: Package `seq-2.14' is unavailable
 
Output:
 
nil
 
Makefile:8: recipe for target 'elpa' failed
 
make: *** [elpa] Error 255

But the exact same build steps work on silex/emacs:26-dev and silex/emacs:master-dev.
If i run silex/emacs:25-dev locally i can require seq no problem. If i run circleci local execute -c process.yml --job test-emacs-25 locally I can reproduce but not sure how to debug further.

If interesting or helpful, the circle-ci config file is https://github.com/clojure-emacs/inf-clojure/blob/master/.circleci/config.yml and the makefile is a simple one at https://github.com/clojure-emacs/inf-clojure/blob/master/Makefile.

Also confusing why emacs has (24.4) in parens but not sure if related. That is also present in the successful runs on silex/emacs:26-dev and master-dev.

Silex · 2020-04-25T19:38:11Z

@dpsutton: can you run cask --debug install and report?

Anyway, if it's indeed the same issue I guess cask creates its own .emacs.d and thus you need to fiddle with GPG keys there.

dpsutton · 2020-04-25T20:18:36Z

@Silex I tried with cask --debug install and it actually errors earlier with no information:

with --debug:

  #!/bin/bash -eo pipefail
make elpa
cask --debug install
Loading package information... Makefile:8: recipe for target 'elpa' failed
make: *** [elpa] Error 255
Error:
Exited with code exit status 2

Step failed
Error: runner failed (exited with 101)

vs without debug

 #!/bin/bash -eo pipefail
make elpa
cask install
Loading package information... done
Package operations: 4 installs, 0 removals
  - Installing [ 1/4] clojure-mode (5.11)... done
  - Installing [ 2/4] emacs (24.4)... already present
Dependency assess failed to install: Package `seq-2.14' is unavailable
Output:
nil
Makefile:8: recipe for target 'elpa' failed
make: *** [elpa] Error 255
Error:
Exited with code exit status 2

tried with --verbose

  #!/bin/bash -eo pipefail
make elpa
cask --verbose install

Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Loading package information... done
Package operations: 4 installs, 0 removals
  - Installing [ 1/4] clojure-mode (5.11)... done
  - Installing [ 2/4] emacs (24.4)... already present
Dependency assess failed to install: Package `seq-2.14' is unavailable
Output:
nil
Makefile:8: recipe for target 'elpa' failed
make: *** [elpa] Error 255

Silex · 2020-04-26T09:38:13Z

Ok I ran some checks:

Emacs 25 contains seq version 2.3
Emacs 26 contains seq version 2.20

You require version 2.14.

You need to either lower the required version or install a newer version. Maybe @NicolasPetton or @DamienCassou can help, I don't use Cask so I don't know.

dpsutton · 2020-04-26T09:48:54Z

That's incredibly awesome of you. Thank you so much. The only dependency i see is just a single (require 'seq) in inf-clojure mode and no Package-Requires anywhere else. No idea why its asking for that version in particular but I'll pursue it elsewhere. Again thank you so very much for your help.

DamienCassou · 2020-04-26T13:40:47Z

You might need to add GNU ELPA to your list of package archives. I'm not using Cask so I can't be of much more help.

dpsutton · 2020-04-26T16:38:57Z

I think my issue turns out to be unrelated. I figured it was the pkg key stuff but thanks to @Silex 's help I see it was inf-clojure requires assess requires m-buffer which requires seq 2.14. Thank you all for your help and sorry to have polluted this issue with something non-related

gonewest818 closed this as completed Feb 18, 2020

Silex reopened this Feb 18, 2020

Silex closed this as completed Feb 18, 2020

jfdm mentioned this issue Dec 22, 2021

Update semantic highlighting code in line with the update IDE protocol idris-hackers/idris-mode#547

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"Failed to download `gnu' archive" error in silex/emacs:25-dev #56

"Failed to download `gnu' archive" error in silex/emacs:25-dev #56

gonewest818 commented Feb 17, 2020

Silex commented Feb 17, 2020

gonewest818 commented Feb 17, 2020

Silex commented Feb 17, 2020

gonewest818 commented Feb 17, 2020

Silex commented Feb 17, 2020 •

edited

Loading

Silex commented Feb 17, 2020

gonewest818 commented Feb 17, 2020 •

edited

Loading

gonewest818 commented Feb 17, 2020

gonewest818 commented Feb 18, 2020 •

edited

Loading

gonewest818 commented Feb 18, 2020

Silex commented Feb 18, 2020 •

edited

Loading

gonewest818 commented Feb 18, 2020

Silex commented Feb 18, 2020

gonewest818 commented Feb 18, 2020

Silex commented Feb 18, 2020

dpsutton commented Apr 25, 2020

Silex commented Apr 25, 2020 •

edited

Loading

dpsutton commented Apr 25, 2020

Silex commented Apr 26, 2020

dpsutton commented Apr 26, 2020

DamienCassou commented Apr 26, 2020

dpsutton commented Apr 26, 2020

"Failed to download `gnu' archive" error in silex/emacs:25-dev #56

"Failed to download `gnu' archive" error in silex/emacs:25-dev #56

Comments

gonewest818 commented Feb 17, 2020

Silex commented Feb 17, 2020

gonewest818 commented Feb 17, 2020

Silex commented Feb 17, 2020

gonewest818 commented Feb 17, 2020

Silex commented Feb 17, 2020 • edited Loading

Silex commented Feb 17, 2020

gonewest818 commented Feb 17, 2020 • edited Loading

gonewest818 commented Feb 17, 2020

gonewest818 commented Feb 18, 2020 • edited Loading

gonewest818 commented Feb 18, 2020

Silex commented Feb 18, 2020 • edited Loading

gonewest818 commented Feb 18, 2020

Silex commented Feb 18, 2020

gonewest818 commented Feb 18, 2020

Silex commented Feb 18, 2020

dpsutton commented Apr 25, 2020

Silex commented Apr 25, 2020 • edited Loading

dpsutton commented Apr 25, 2020

Silex commented Apr 26, 2020

dpsutton commented Apr 26, 2020

DamienCassou commented Apr 26, 2020

dpsutton commented Apr 26, 2020

Silex commented Feb 17, 2020 •

edited

Loading

gonewest818 commented Feb 17, 2020 •

edited

Loading

gonewest818 commented Feb 18, 2020 •

edited

Loading

Silex commented Feb 18, 2020 •

edited

Loading

Silex commented Apr 25, 2020 •

edited

Loading