document how to verify all RPMs are signed

[ktdreyer]

From discussions with @sosiouxme : We should document a way for users to verify that all RPMs have been signed. (Like a code snippet)

@yazug @lhh you guys have some code for this right?

[sosiouxme]

code or pointers would be much appreciated... don't see anything in the erratatool api docs!

[yazug]

yes we do, in shale where this came from at a minimum.

[sosiouxme]

What is shale?

[sosiouxme]

@yazug can you share any details you have handy? I just don't even know where to start.

[yazug]

Build::signed_rpms() function returns if build is signed or not

the other part is Erratum::_get_build_list(check_signatures=True) will do additional requests to see if builds are signed or not but will bail out early on checking all of them if one of them is unsigned.

and the flag is set depending on the state of the advisory ... see the Erratum::fetch() function

it sets the flag 'needs_sigs' which can be seen via Erratum::current_flags

[sosiouxme]

On Fri, Apr 26, 2019 at 12:38 AM Jon Schlueter ***@***.***> wrote: Build::signed_rpms() function returns if build is signed or not the other part is Erratum::_get_build_list(check_signatures=True) will do additional requests to see if builds are signed or not but will bail out early on checking all of them if one of them is unsigned.

I'm using 40770 as my guinea pig. _get_build_list seems to return null whether given check_signatures=True or not. Not sure what it's doing.

and the flag is set depending on the state of the advisory ... see the Erratum::fetch() function it sets the flag 'needs_sigs' which can be seen via Erratum::current_flags

On that erratum all of the builds are signed, and there's no flag. So does the QE state and lack of flag generally indicate that everything has been signed? If I had just added some builds and flipped it to QE state, would the lack of that flag indicate that there were no builds needing signing, or that the flag just hasn't been set yet? Because that's what I'm trying to automate... add builds and get them signed once RPMdiff passes. It would be nice if there's a quicker way to tell rather than creating all the Builds and calling signed_rpms() on them!

…

[sosiouxme]

It helps if I look at the source code re _get_build_list() :)

It seems evident from

errata-tool/errata_tool/erratum.py

Lines 489 to 497 in 62303af

	if have_all_sigs and check_signatures:
	if not self._check_signature_for_build(nvr):
	self.addFlags('needs_sigs')
	have_all_sigs = False
	self.errata_builds[product_version] = builds
	if have_all_sigs:
	self.removeFlags(['request_sigs', 'needs_sigs'])

that needs_sigs in current_flags is only set if the erratum is in a state where signing would normally have happened and there are builds without signatures. Actually it's hard to see how needs_sigs and request_sigs flags differ; seems like both or neither would be present. Regardless, would be nice to have documentation of what the flags indicate.

Side note: most errata only sign in QE state. However we (OCP) are in discussions to see if we can get builds signed in NEW_FILES state, which seems to at least be possible.

[lhh]

The only problem with the code whence this came is that the code in Shale utilizes local caching so that it does not fetch signatures for n-v-rs that it has already fetched, speeding things up on repeated requests. A given build is never signed twice.

That code isn't in python-errata-tool because odd side-effects (such as editing local files when it ordinarily wouldn't) is generally poor library design, and quietly affects parallel runs of p-e-t on the same erratum. (Shale has locking and so forth so that a user won't step on themselves)

As for the flags, request_sigs is an action that used to be a button that users of the Errata Tool had to press manually. These days, things get signatures automatically in most cases. Needs_sigs was between the two: someone pressed the button, but the bot (or an ET user with signature capability) had not done their end yet. If a user was in a rush, they could then go talk to appropriate people to speed things up.

Conceivably, one could remove the request_sigs flag as of about a year (ish) ago.

[sosiouxme]

In elliott we're now looking at errata_tool.build.Build(build).signed_rpms to determine if RPMs have been signed. This works, but it seems to take up to several minutes per build. Any idea what it's doing and how to short-circuit it? errata-tool API is a bit slow, sure, but this is ridiculous even for that.

[ktdreyer]

If I were you, I'd probably skip using the Errata Tool and interact with Koji (Brew) directly. Koji's PathInfo will give you the URLs to the signed RPM files, and you can run HTTP HEAD requests on those.
Here's some example code that does this:

import requests
import posixpath
import koji
from koji_cli.lib import activate_session

KEY = 'fd431d51'  # Red Hat release key
PROFILE = 'brew'

def get_session():
    """
    Return an anonymous koji session for PROFILE.
    """
    conf = koji.read_config(PROFILE)
    conf['authtype'] = 'noauth'
    hub = conf['server']
    session = koji.ClientSession(hub, {})
    activate_session(session, conf)
    return session


def get_koji_pathinfo():
    """
    Return a koji.PathInfo object for our PROFILE.
    """
    conf = koji.read_config(PROFILE)
    top = conf['topurl']  # or 'topdir' here for NFS access
    pathinfo = koji.PathInfo(topdir=top)
    return pathinfo


# Find Koji's buildinfo for our latest PACKAGE (using ceph as an example):
TAG = 'ceph-3.3-rhel-7'
PACKAGE = 'ceph'
session = get_session()
latest_builds = session.getLatestBuilds(TAG, package=PACKAGE)
# This is our latest ceph buildinfo:
build = latest_builds[0]

# Look up our build's path.
pathinfo = get_koji_pathinfo()
# This example simply checks the "src" RPM for simplicity. To make this code
# more robust, you should verify every RPM file in the build is a
# fully-signed, non-zero size file.
buildinfo = {
    'arch': 'src',
    'name': build['name'],
    'version': build['version'],
    'release': build['release'],
}
builddir = pathinfo.build(build)
rpmpath = pathinfo.signed(buildinfo, KEY)
url = posixpath.join(builddir, rpmpath)
print(url)

# Here's an example of verifying that this is a non-zero size file using HTTP.
response = requests.head(url, allow_redirects=True)
response.raise_for_status()
headers = response.headers
if headers['Content-Length'] <= 0:
    print('%s is zero-length' % url)
# (If you switched to NFS instead (see get_koji_pathinfo()), you could check
# the files directly with python's os module.)

This sort of matches what I do when I'm waiting for robosignatory to complete a long signing operation.

I've occasionally thought that we should generalize this into a standalone koji-verify-signed CLI utility and submit it to https://pagure.io/koji-tools . It could take a build NVR and a GPG key and verify that all the artifacts are signed with that key.

[ktdreyer]

Separately, we need a tool to verify that all the RPMs in a container image are signed. I have an idea to submit a change to koji-containerbuild for that.

[ktdreyer]

Separately, we need a tool to verify that all the RPMs in a container image are signed. I have an idea to submit a change to koji-containerbuild for that.

For now I have pushed this tool in my koji-tools fork, "koji-container-signatures"