[gentoo-portage-dev] [PATCH] Disarm FEATURES=distcc-pump

[Michał Górny]

The pump mode of distcc has been causing issues for years now,
and upstream does even attempt to fix it. Disarm the FEATURES so that
people do not have to do that themselves after discovering all the bugs.
---
bin/phase-functions.sh | 17 -----------------
man/make.conf.5 | 5 ++++-
pym/_emerge/EbuildPhase.py | 2 +-
3 files changed, 5 insertions(+), 19 deletions(-)

diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh
index 1f9faaa41..9c6f9cbd5 100644
--- a/bin/phase-functions.sh
+++ b/bin/phase-functions.sh
@@ -402,19 +402,6 @@ __dyn_prepare() {
trap - SIGINT SIGQUIT
}

-# @FUNCTION: __start_distcc
-# @DESCRIPTION:
-# Start distcc-pump if necessary.
-__start_distcc() {
- if has distcc $FEATURES && has distcc-pump $FEATURES ; then
- if [[ -z $INCLUDE_SERVER_PORT ]] || [[ ! -w $INCLUDE_SERVER_PORT ]] ; then
- # adding distcc to PATH repeatedly results in fatal distcc recursion :)
- eval $(pump --startup | grep -v PATH)
- trap "pump --shutdown >/dev/null" EXIT
- fi
- fi
-}
-
__dyn_configure() {

if [[ -e $PORTAGE_BUILDDIR/.configured ]] ; then
@@ -434,7 +421,6 @@ __dyn_configure() {
fi

trap __abort_configure SIGINT SIGQUIT
- __start_distcc

__ebuild_phase pre_src_configure

@@ -468,7 +454,6 @@ __dyn_compile() {
fi

trap __abort_compile SIGINT SIGQUIT
- __start_distcc

__ebuild_phase pre_src_compile

@@ -492,7 +477,6 @@ __dyn_test() {
fi

trap "__abort_test" SIGINT SIGQUIT
- __start_distcc

if [ -d "${S}" ]; then
cd "${S}"
@@ -536,7 +520,6 @@ __dyn_install() {
return 0
fi
trap "__abort_install" SIGINT SIGQUIT
- __start_distcc

# Handle setting QA_* based on QA_PREBUILT
# Those variables shouldn't be needed before src_install()
diff --git a/man/make.conf.5 b/man/make.conf.5
index cb0f00237..9487991d9 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -385,7 +385,10 @@ will be reused whenever they are available.
Enable portage support for the distcc package.
.TP
.B distcc\-pump
-Enable portage support for the distcc package with pump mode.
+Ignored. In the past it used to enable distcc pump mode support
+but it has been removed due to causing numerous issues (including build
+failures, incorrect configure results and even silent miscompilations),
+and being unmaintained upstream.
.TP
.B distlocks
Portage uses lockfiles to ensure competing instances don't clobber
diff --git a/pym/_emerge/EbuildPhase.py b/pym/_emerge/EbuildPhase.py
index 4104cefa7..50e3dd1f4 100644
--- a/pym/_emerge/EbuildPhase.py
+++ b/pym/_emerge/EbuildPhase.py
@@ -49,7 +49,7 @@ class EbuildPhase(CompositeTask):

# FEATURES displayed prior to setup phase
_features_display = (
- "ccache", "compressdebug", "distcc", "distcc-pump", "fakeroot",
+ "ccache", "compressdebug", "distcc", "fakeroot",
"installsources", "keeptemp", "keepwork", "network-sandbox",
"network-sandbox-proxy", "nostrip", "preserve-libs", "sandbox",
"selinux", "sesandbox", "splitdebug", "suidctl", "test",

[Zac Medico]

Post by Michał Górny
The pump mode of distcc has been causing issues for years now,
and upstream does even attempt to fix it. Disarm the FEATURES so that
people do not have to do that themselves after discovering all the bugs.
---
bin/phase-functions.sh | 17 -----------------
man/make.conf.5 | 5 ++++-
pym/_emerge/EbuildPhase.py | 2 +-
3 files changed, 5 insertions(+), 19 deletions(-)

Maybe we should simply support RESTRICT="distcc-pump" so that
incompatible ebuilds can disable it? I don't see that many bug reports
about it, and a forums search turned up this recent thread which
suggests that some people may be using it:

https://forums.gentoo.org/viewtopic-t-1082176-highlight-distccpump.html

[Michał Górny]

W dniu pon, 16.07.2018 o godzinie 13∶16 -0700, uÅŒytkownik Zac Medico

Post by Zac Medico

Post by Michał Górny
The pump mode of distcc has been causing issues for years now,
and upstream does even attempt to fix it. Disarm the FEATURES so that
people do not have to do that themselves after discovering all the bugs.
---
bin/phase-functions.sh | 17 -----------------
man/make.conf.5 | 5 ++++-
pym/_emerge/EbuildPhase.py | 2 +-
3 files changed, 5 insertions(+), 19 deletions(-)

Maybe we should simply support RESTRICT="distcc-pump" so that
incompatible ebuilds can disable it? I don't see that many bug reports
about it, and a forums search turned up this recent thread which

I did. There are only two reasons you don't see them often:

1. because not that many people use distcc,

2. because when they do, they quickly learn how broken it is and disable
it.

RESTRICT won't be helpful because distcc-pump is also capable of silent
miscompilations and indirect breakage. If you used it at least once, my
only advice is to rebuild your entire system.

Post by Zac Medico
https://forums.gentoo.org/viewtopic-t-1082176-highlight-distccpump.html

[konsolebox]

W dniu pon, 16.07.2018 o godzinie 13∶16 -0700, użytkownik Zac Medico

Post by Zac Medico

Post by Michał Górny
The pump mode of distcc has been causing issues for years now,
and upstream does even attempt to fix it. Disarm the FEATURES so that
people do not have to do that themselves after discovering all the bugs.
---
bin/phase-functions.sh | 17 -----------------
man/make.conf.5 | 5 ++++-
pym/_emerge/EbuildPhase.py | 2 +-
3 files changed, 5 insertions(+), 19 deletions(-)

Maybe we should simply support RESTRICT="distcc-pump" so that
incompatible ebuilds can disable it? I don't see that many bug reports
about it, and a forums search turned up this recent thread which

1. because not that many people use distcc,
2. because when they do, they quickly learn how broken it is and disable
it.

It's been just a month and a half since I rebuilt a Gentoo system with
distcc-pump, and that system ended up running well.

I don't disable distcc-pump quickly. At least not globally. I only
disable it in packages that don't compile with it.

RESTRICT won't be helpful because distcc-pump is also capable of silent
miscompilations and indirect breakage. If you used it at least once, my
only advice is to rebuild your entire system.

I believe giving a general warning whenever distcc-pump is used is
enough. Users should be allowed to decide whether they'll use it or
not. There are users who know when to use it, and are capable of
managing build-time inconsistencies.

Saying that distcc-pump is capable of silent miscompilations and
indirect breakage I think is also an aggressive and ungrounded
presumption.

Also, if upstream suddenly decides to fix whatever needs to be fixed
on this, it would need another request to put the feature back, which
I find would be very hard to be granted.

I also agree that making specific packages use RESTRICT is more appropriate.

[Michał Górny]

W dniu wto, 17.07.2018 o godzinie 23∶11 +0800, uÅŒytkownik konsolebox

Post by konsolebox

Post by Michał Górny
W dniu pon, 16.07.2018 o godzinie 13∶16 -0700, uÅŒytkownik Zac Medico

Post by Zac Medico

Post by Michał Górny
The pump mode of distcc has been causing issues for years now,
and upstream does even attempt to fix it. Disarm the FEATURES so that
people do not have to do that themselves after discovering all the bugs.
---
bin/phase-functions.sh | 17 -----------------
man/make.conf.5 | 5 ++++-
pym/_emerge/EbuildPhase.py | 2 +-
3 files changed, 5 insertions(+), 19 deletions(-)

Maybe we should simply support RESTRICT="distcc-pump" so that
incompatible ebuilds can disable it? I don't see that many bug reports
about it, and a forums search turned up this recent thread which

1. because not that many people use distcc,
2. because when they do, they quickly learn how broken it is and disable
it.

It's been just a month and a half since I rebuilt a Gentoo system with
distcc-pump, and that system ended up running well.
I don't disable distcc-pump quickly. At least not globally. I only
disable it in packages that don't compile with it.

Post by Michał Górny
RESTRICT won't be helpful because distcc-pump is also capable of silent
miscompilations and indirect breakage. If you used it at least once, my
only advice is to rebuild your entire system.

I believe giving a general warning whenever distcc-pump is used is
enough. Users should be allowed to decide whether they'll use it or
not. There are users who know when to use it, and are capable of
managing build-time inconsistencies.

Sure, provided that they are warned that any suspicious bug reports will
be rejected. Developers have better things to do than try to figure out
whether some unmaintained-for-years, half-broken feature may have been
the cause of a misbehaving system.

Post by konsolebox
Saying that distcc-pump is capable of silent miscompilations and
indirect breakage I think is also an aggressive and ungrounded
presumption.

It's not a 'presumption', it's experience. Yes, it's possible that
the breakages I've seen don't happen at the moment. But given
the generic idea that distcc-pump can cause configure checks to misfire
without actually causing build-time breakage, it's entirely possible.

Post by konsolebox
Also, if upstream suddenly decides to fix whatever needs to be fixed
on this, it would need another request to put the feature back, which
I find would be very hard to be granted.
I also agree that making specific packages use RESTRICT is more appropriate.

[konsolebox]

Post by Michał Górny
Sure, provided that they are warned that any suspicious bug reports will
be rejected. Developers have better things to do than try to figure out
whether some unmaintained-for-years, half-broken feature may have been
the cause of a misbehaving system.

Yes, even a strong warning like "Warning: Usage of distcc-pump can
lead to broken builds. Any bug report that involves a package or a
package dependency of any level that is compiled with the use of
distcc-pump will unconditionally be rejected. You might as well just
rebuild your system without distcc-pump to make a valid bug report."
is very agreeable to me.