Only bind loopback addresses when binding to local

* Only bind loopback addresses when binding to local

Today when binding to local (the default) we bind to any address that is
a loopback address, or any address on an interface that declares itself
as a loopback interface. Yet, not all addresses on loopback interfaces
are loopback addresses. This arises on macOS where there is a link-local
address assigned to the loopback interface (fe80::1%lo0) and in Docker
services where virtual IPs of the service are assigned to the loopback
interface (docker/libnetwork#1877). These situations cause problems:
 - because we do not handle the scope ID of a link-local address, we end
   up bound to an address for which publishing of that address does not
   allow that address to be reached (since we drop the scope)
 - the virtual IPs in the Docker situation are not loopback addresses,
   they are not link-local addresses, so we end up bound to interfaces
   that cause the bootstrap checks to be enforced even though the
   instance is only bound to local

We address this by only binding to actual loopback addresses, and skip
binding to any address on a loopback interface that is not a loopback
address. This lets us simplify some code where in the bootstrap checks
we were skipping link-local addresses, and in writing the ports file
where we had to skip link-local addresses because again the formatting
of them does not allow them to be connected to by another node (to be
clear, they could be connected to via the scope-qualified address, but
that information is not written out).

Relates #28029

core/src/main/java/org/elasticsearch/bootstrap/BootstrapChecks.java

@@ -132,7 +132,7 @@ final class BootstrapChecks {
         }
 
         if (enforceLimits) {
-            logger.info("bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks");
+            logger.info("bound or publishing to a non-loopback address, enforcing bootstrap checks");
         } else if (enforceBootstrapChecks) {
             logger.info("explicitly enforcing bootstrap checks");
         }
@@ -176,11 +176,10 @@ final class BootstrapChecks {
      * @return {@code true} if the checks should be enforced
      */
     static boolean enforceLimits(final BoundTransportAddress boundTransportAddress, final String discoveryType) {
-        final Predicate<TransportAddress> isLoopbackOrLinkLocalAddress =
-                t -> t.address().getAddress().isLinkLocalAddress() || t.address().getAddress().isLoopbackAddress();
+        final Predicate<TransportAddress> isLoopbackAddress = t -> t.address().getAddress().isLoopbackAddress();
         final boolean bound =
-                !(Arrays.stream(boundTransportAddress.boundAddresses()).allMatch(isLoopbackOrLinkLocalAddress) &&
-                isLoopbackOrLinkLocalAddress.test(boundTransportAddress.publishAddress()));
+                !(Arrays.stream(boundTransportAddress.boundAddresses()).allMatch(isLoopbackAddress) &&
+                isLoopbackAddress.test(boundTransportAddress.publishAddress()));
         return bound && !"single-node".equals(discoveryType);
     }
 

core/src/main/java/org/elasticsearch/common/network/NetworkUtils.java

@@ -149,17 +149,14 @@ public abstract class NetworkUtils {
     public static boolean defaultReuseAddress() {
         return Constants.WINDOWS ? false : true;
     }
-    
+
     /** Returns all interface-local scope (loopback) addresses for interfaces that are up. */
     static InetAddress[] getLoopbackAddresses() throws SocketException {
         List<InetAddress> list = new ArrayList<>();
         for (NetworkInterface intf : getInterfaces()) {
             if (intf.isUp()) {
-                // NOTE: some operating systems (e.g. BSD stack) assign a link local address to the loopback interface
-                // while technically not a loopback address, some of these treat them as one (e.g. OS X "localhost") so we must too,
-                // otherwise things just won't work out of box. So we include all addresses from loopback interfaces.
                 for (InetAddress address : Collections.list(intf.getInetAddresses())) {
-                    if (intf.isLoopback() || address.isLoopbackAddress()) {
+                    if (address.isLoopbackAddress()) {
                         list.add(address);
                     }
                 }

core/src/main/java/org/elasticsearch/node/Node.java

@@ -858,10 +858,6 @@ public class Node implements Closeable {
         try (BufferedWriter writer = Files.newBufferedWriter(tmpPortsFile, Charset.forName("UTF-8"))) {
             for (TransportAddress address : boundAddress.boundAddresses()) {
                 InetAddress inetAddress = InetAddress.getByName(address.getAddress());
-                if (inetAddress instanceof Inet6Address && inetAddress.isLinkLocalAddress()) {
-                    // no link local, just causes problems
-                    continue;
-                }
                 writer.write(NetworkAddress.format(new InetSocketAddress(inetAddress, address.getPort())) + "\n");
             }
         } catch (IOException e) {

core/src/test/java/org/elasticsearch/bootstrap/BootstrapChecksTests.java

@@ -78,7 +78,7 @@ public class BootstrapChecksTests extends ESTestCase {
     public void testLogMessageInProductionMode() throws NodeValidationException {
         final Logger logger = mock(Logger.class);
         BootstrapChecks.check(defaultContext, true, Collections.emptyList(), logger);
-        verify(logger).info("bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks");
+        verify(logger).info("bound or publishing to a non-loopback address, enforcing bootstrap checks");
         verifyNoMoreInteractions(logger);
     }
 

docs/reference/setup/bootstrap-checks.asciidoc

@@ -23,16 +23,16 @@ documented individually.
 [float]
 === Development vs. production mode
 
-By default, Elasticsearch binds to `localhost` for <<modules-http,HTTP>> and
-<<modules-transport,transport (internal)>> communication. This is fine for
-downloading and playing with Elasticsearch as well as everyday development, but it's
-useless for production systems. To join a cluster, an Elasticsearch node must be
-reachable via transport communication. To join a cluster over an external
-network interface, a node must bind transport to an external interface and not
-be using <<single-node-discovery,single-node discovery>>. Thus, we consider an
-Elasticsearch node to be in development mode if it can not form a cluster with
-another machine over an external network interface, and is otherwise in
-production mode if it can join a cluster over an external interface.
+By default, Elasticsearch binds to loopback addresses for <<modules-http,HTTP>>
+and <<modules-transport,transport (internal)>> communication. This is fine for
+downloading and playing with Elasticsearch as well as everyday development, but
+it's useless for production systems. To join a cluster, an Elasticsearch node
+must be reachable via transport communication. To join a cluster via a
+non-loopback address, a node must bind transport to a non-loopback address and
+not be using <<single-node-discovery,single-node discovery>>. Thus, we consider
+an Elasticsearch node to be in development mode if it can not form a cluster
+with another machine via a non-loopback address, and is otherwise in production
+mode if it can join a cluster via non-loopback addresses.
 
 Note that HTTP and transport can be configured independently via
 <<modules-http,`http.host`>> and <<modules-transport,`transport.host`>>; this