class: center, middle # Bread and butter of RPM package maintainers ## Lumír Balhar ### Senior software engineer, Red Hat, Python maintenance team --- # Agenda 1. Where are we? 1. Terms used 1. Responsibilities of package maintainers 1. Simple scenario - a leaf package 2. Fedora loves Python - all of them 2. The main Python, the main problem 3. But what if you cannot upgrade? 3. Security vulnerabilities and how we fix them 3. Conclusion --- class: middle, center # Bell curve of software users and developers  --- class: middle, center # Bell curve of software users and developers  --- class: middle, center # Code flow <img src="flow.png" alt="" width="100%"> --- # Terms used * Upstream and downstream development * CVE (Common Vulnerabilities and Exposures) * EOL (End of Life) --- # Responsibilities of package maintainers * Keep your package functioning. It should: * build from sources, * be installable, * and just work. * Keep it up to date, fix issues and CVEs. * Limit the impact on other packages. * Limit the impact on users' apps and libs. -- ### Stay invisible <img src="homer.gif" alt="" height="150px"> --- class: middle, center # Simple scenario - a leaf package  --- # Simple scenario - what could go wrong? * Somebody finds a bug or security vulnerability in your package. * An upstream issue (bug in code, CVE), or * a downstream issue (missing files, packaging problem, conflicts, FTI, FTBFS). * Somebody update or change something and it might break your package. * You update or change something and it might break users' expectations. --- # What to do? -- ### Upstream issues * Prepare a fix, test it in Fedora, and propose it upstream. 😃 * Report the problem and wait. 😕 * Do nothing. 😞 -- ### Broken dependencies * Port the code to the new version. * Look for compatible packages you can depend on. -- ### Downstream issue in your package * Fix it! --- class: center, middle # Let's move to the other side of the problem <img src="dark_side.png" alt="" width="90%"> --- # Fedora loves Python - all of them * CPython 2.7 (well, this one not so much 😉) * CPython 3.6 * CPython 3.7 * CPython 3.8 * CPython 3.9 * CPython 3.10 * CPython 3.11 * CPython 3.12 * PyPy * PyPy 3.9 --- # Fedora loves Python - all of them We're trying make Fedora the best distro for Python lovers: * long support of old Pythons, and * fast delivery of new ones: * 7 days for 3.11 alpha 1 * 3 days for 3.11 beta 1 * 3 days for 3.12 alpha 1 * much faster than CI providers (GH actions, Travis, …) -- ### Not a Fedora user? No problem! -- ### Fedora & all Pythons & tox * Container image (fedora-python-tox) * Github action (tox-github-action) --- # Fedora loves Python - all of them * CPython 2.7 * CPython 3.6 * CPython 3.7 * CPython 3.8 * CPython 3.9 * CPython 3.10 * ** CPython 3.11 <-- the main one** * CPython 3.12 * PyPy * PyPy 3.9 --- class: center, middle # Let's increase complexity (a little bit) --- class: center, middle # The main Python - the main problem  --- # We cannot just update it -- ### Incompatible changes (for example deprecated modules removed). -- ### Pre-compiled `.pyc` files in locations where only the root can write. --- # The update process of the main Python ### Write Fedora change proposal. -- ### Package a new Python version as a new package. -- ### Rebuild all thousands of packages in COPR, and * #### file bugs, open PRs, investigate issues, and * #### do that both upstream and downstream. -- ### Rebuild all the packages in side tag. -- ### Merge side tag to rawhide. --- # Some numbers from the update process to 3.12 ### 3361 packages in COPR ### 47987 builds ### 460 bugs just in Bugzilla --- # Fedora -- ## Updates are complex and you have to be careful #### One update can break many other packages or the whole system. -- ## Updates usually fix all the issues and security flaws #### Or we fix them in Fedora first and propose the fix upstream 😉 #### Or we backport fixes from newer upstream to EOL components. --- class: center, middle # You can do it in a similar way! -- # Wanna know more? ### Make sure to stay here (E112). Karolina Surma will tell you more in the next session. --- class: middle, center # But what if you cannot update components? ## Meet the other part of the bell curve - RHEL ### Red Hat Enterprise Linux --- class: center, middle # The promise (re-phrased and simplified) ### We'll keep your systems secure without breaking backward compatibility of provided components. -- ### It means that updates are usually out of the question. --- # Web cache poisoning in urllib.parse_qs (CVE-2021-23336) ## The problem Possible web cache poisoning if a proxy and an app written in Python follow different rules for parsing query strings. > `https://redhat.com/index.php?key=value&key2=value2;key3=value3` * In Python, `;` and `&` are both allowed by default. * World Wide Web Consortium recommends using only `&`. --- # Web cache poisoning in urllib.parse_qs (CVE-2021-23336) ## Upstream solution Backward incompatible change. `&` is the new default. You can switch to `;` but not to the old behavior. --- # Web cache poisoning in urllib.parse_qs (CVE-2021-23336) ## Fedora solution * Update most of the interpreters to their latest versions. * After upstream backported the patch and released the fixed versions. * Backport the patch to EOL interpreters. --- # Web cache poisoning in urllib.parse_qs (CVE-2021-23336) ## RHEL solution * Adjust the patch to keep the old behavior as the default. * When the old default is used Python raises a warning with a link to the documentation. * You can set the default: * in Python code itself, or * in the config file in `/etc`, or * via an environment variable. --- # tarfile module directory traversal (CVE-2007-4559) ## The problem TAR is a very powerful archive format designed to backup whole systems with symlinks, hardlinks, all kinds of metadata, etc. Untrusted archives should not be blindly unpacked. --- # tarfile module directory traversal (CVE-2007-4559) ## Upstream solution Nothing. The documentation warns about unpacking untrusted archives :/ --- # tarfile module directory traversal (CVE-2007-4559) ## Petr's solution * PEP 706 – Filter for tarfile.extractall * Filters: `fully_trusted`, `tar`, and `data` * Deprecation period for Python 3.12 and 3.13 * The same default (`fully_trusted`) raising DeprecationWarning. * Python 3.14 will have the safest `data` filter as the default. --- # tarfile module directory traversal (CVE-2007-4559) ## Fedora solution * Updated as well as EOL interpreters follow the solution implemented upstream. --- # tarfile module directory traversal (CVE-2007-4559) ## RHEL solution * Even more strict than upstream – the patch is the same but the default is the safest immediately. --- class: center, middle # Conclusion ## Being a package maintainer might mean a lot of different things! ### But it's centrainly not a boring job! --- # Fedora * It's good to understand your packages to be able to: * help upstream with development and porting to new dependencies, and * report issues found in Fedora, and * adapt the package for Fedora, if upstream is not interested. --- # RHEL * You have to deeply understand your packages to be able to: * support old versions much longer than upstream, and * adapt upstream patches to older versions, and * keep backward compatibility and fix security flaws. * You also have to be prepared to do tought decisions. --- class: center, middle # Thank you! ## Questions? ### Lumír Balhar, Red Hat