X509 can't load " TRUSTED CERTIFICATE "s

[fabricemarie]

Environment:

• MacOS Catalina 10.15.3 (19D76)
• Python 3.6.10
• cryptography 2.9.2
• cffi 1.14.0
• pip 20.0.2
• pycparser 2.20
• setuptools 46.1.3
• six 1.14.0
• wheel 0.34.2
• OpenSSL compatibility version 1.0.0, current version 1281.0.0

I have an issue loading certificates wrapped in -----BEGIN TRUSTED CERTIFICATE----- and -----END TRUSTED CERTIFICATE----- with x509.load_pem_x509_certificate().

When I do so it returns the following error:

Traceback (most recent call last):
  File "./toto.py", line 39, in <module>
    load_pem_x509_certificate(bytes(cert, 'utf-8'), default_backend())
  File "/Users/fabricemarie/.local/share/virtualenvs/toto/lib/python3.6/site-packages/cryptography/x509/base.py", line 52, in load_pem_x509_certificate
    return backend.load_pem_x509_certificate(data)
  File "/Users/fabricemarie/.local/share/virtualenvs/toto/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1223, in load_pem_x509_certificate
    "Unable to load certificate. See https://cryptography.io/en/la"
ValueError: Unable to load certificate. See https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file for more details.

It works as expected after replacing:

• -----BEGIN TRUSTED CERTIFICATE----- with -----BEGIN CERTIFICATE----- and
• -----END TRUSTED CERTIFICATE----- with -----END CERTIFICATE-----

Example of a random CA that I was trying to load (found in the Linux trusted rootCAs bundle):

-----BEGIN TRUSTED CERTIFICATE-----
MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7MCEwFAYIKwYB
BQUHAwQGCCsGAQUFBwMBDAlBQ0NWUkFJWjE=
-----END TRUSTED CERTIFICATE-----

OpenSSL indeed opens them without modification:

$ x509 -in /tmp/trusted_debug.crt -noout -fingerprint
SHA1 Fingerprint=93:05:7A:88:15:C6:4F:CE:88:2F:FA:91:16:52:28:78:BC:53:64:17

It's ok not to support it I guess, but I suggest that the FAQ over at https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file be modified accordingly to give a hint to the user.

[reaperhulk]

The OpenSSL "Trusted Certificate" encoding is a non-standard feature OpenSSL added that we don't have plans to support. We'd be happy to take a PR that explains that we don't support it and how to convert it into a normal X509 certificate (I believe openssl x509 -in trusted.pem -out normal.pem will actually suffice) for parsing with cryptography.

[fabricemarie]

Actually the certificate above loads just fine in cryptography after changing the line from -----BEGIN TRUSTED CERTIFICATE----- to -----BEGIN CERTIFICATE----- (remove the word TRUSTED and it works). I was just suggesting we should give that trick in the documentation in the link above.

[luzgabriel]

A TRUSTED CERTIFICATE is a OpenSSL non standard format with some data appended to the end of the BASE64 code with some trust rules. OpenSSL will read it as a standard X.509 but if it uses the string -----BEGIN TRUSTED CERTIFICATE----- it tells openssl that it must be loaded as a X509_AUX so it raises an error when you try to load it in a X509 format thus losing the trust rules data.

Taken from openssl:

    /*
     * In most cases, we can try to interpret the serialized data as a trusted
     * cert (X509 + X509_AUX) and fall back to reading it as a normal cert
     * (just X509), but if the PEM name specifically declares it as a trusted
     * cert, then no fallback should be engaged.  |ignore_trusted| tells if
     * the fallback can be used (1) or not (0).
     */
    int ignore_trusted = 1;

    if (pem_name != NULL) {
        if (strcmp(pem_name, PEM_STRING_X509_TRUSTED) == 0)
            ignore_trusted = 0;

Just removing the string might cause unintended behaviour on other software using this same certificate, you might want to run the openssl x509 -in trusted.pem -out normal.pem to strip it.

[reaperhulk]

Closing for now, but we'll happily take a doc PR to improve explanation of this if anyone is interested.