Service Discovery Explained: How Microservices Find Each Other

Callers that keep a fixed list of addresses keep calling the old ones. Their requests go to machines that no longer run the service, so they time out or are refused.

Picture a common setup. A podcast app runs about 40 microservices, small programs that each do one job. The setup is made up, but the problem is real.

When a listener taps play, the player service asks the episode service for the episode's details and audio link. The episode service runs as several instances, identical copies on different machines. The player found them through a config file listing six fixed IP addresses.

One evening a popular show drops a new episode. An autoscaler, a tool that adds instances when traffic rises, starts four new episode instances. It also replaces two old instances with copies on fresh machines, which have new addresses.

• The four new instances sit idle, because the config file does not mention them.
• Two of the six listed addresses now point at retired machines. About one play request in three fails after a 2 second timeout (made up).
• Someone edits the file by hand and restarts the player service 40 minutes later (made up).

The team's review note is short. Addresses now change every day, so no person should type them into files. The rest of this guide replaces that file with service discovery, one piece at a time.

Service discovery is the way a program finds where another service is running right now, using the service's name. It replaces fixed address lists with a live answer that changes as instances start, move and stop.

Think of a neighbourhood hardware shop that sends repairmen, called mistris, out to jobs. The mistri who fixed a tap in one lane yesterday may be at a site across town today.

Nobody memorises where each mistri is. The owner keeps a slate showing which mistri is at which site today, and customers read the slate before they go looking.

Service discovery works the same way. The slate is the service registry, each mistri is an instance, and the mistri's name is the service name. Reading the slate is a lookup.

Where the picture breaks: a person updates the slate now and then. A registry changes within seconds, and it must notice crashed instances on its own, because a crashed program cannot report anything.

A few words come up again and again:

• Service: a named job, like episode-service. It is the name callers know.
• Instance: one running copy of a service, with its own IP address and port, the numbered door on a machine that a program listens on. For example, 10.0.4.17:8080.
• Service registry: a small database of which instances of each service are alive, and where.
• Health check: a regular test that an instance can still do its work, such as calling its /health address.
• TTL (time to live): how long a caller may reuse an answer before it must ask again.

Here is the new shape of the podcast app. The registry now replaces the config file.

A service registry learns about instances through registration. Either each instance announces itself when it starts, or a separate tool watches the platform and registers instances for it.

Self-registration

In self-registration, the instance itself sends its name, address and port to the registry at startup. When it shuts down cleanly, it removes itself.

The weak spot is a crash. A crashed instance cannot say goodbye. So the instance usually renews its registration every few seconds, a heartbeat that says it is still alive. If heartbeats stop, the registry can drop the entry.

The cost is code. Every service, in every language the team uses, needs the registration logic.

Third-party registration

In third-party registration, another component does the work. It sees an instance start or stop on the platform and updates the registry. The service code stays free of registry logic.

Kubernetes, a platform that runs containers (packaged programs with everything they need) across many machines, works this way. Its control plane, the part that manages the cluster, records which pods back each Service, so your code never registers anything.

Health checks keep dead instances out by testing each instance regularly and hiding any instance that fails. Callers asking the registry should then get only instances that are working.

Consul, a service registry made by HashiCorp, supports several kinds of checks. Its documentation (v2.0.x, as of 19 October 2026) describes these three:

• HTTP check: calls an address over HTTP, the protocol web requests use. A 200 to 299 response is healthy, 429 (too many requests) is a warning, and anything else is a failure.
• TCP check: only tests whether a TCP connection opens. TCP is the basic network link that HTTP usually runs on.
• TTL check: expects the service to report in before a timer runs out. Silence marks it critical, Consul's word for failing.

One detail is worth copying. Consul gives a newly registered check a critical status by default, until it first passes. So a half-started instance does not receive traffic before it has proved itself.

Which instances a caller sees also depends on how it asks. Consul's DNS answers leave out only critical instances by default. Instances in warning state still get traffic unless you set only_passing to true.

Follow one episode instance through its life in the diagram below.

That last arrow needs care. While the machine's Consul agent keeps running, it reaps a long-failing service only if you set DeregisterCriticalServiceAfter. The minimum is 1 minute, and the reaper runs every 30 seconds. Without it, the instance stays listed as critical. If the whole machine leaves cleanly, Consul removes its node and services. If it just vanishes, the servers reap it after a reconnect timeout, 72 hours by default in the Consul source as of 19 October 2026.

In client-side discovery, the calling service asks the registry and picks an instance itself. In server-side discovery, the caller sends its request to a load balancer or proxy, which asks the registry and forwards the request.

Back at the hardware shop, client-side discovery is a customer reading the slate and calling the mistri himself. Server-side discovery is asking the owner to send someone over.

Client-side discovery in Python

This sketch is the player's client. A fake registry stands in for a real one, so the file runs on its own. Read find_instances first: it is the whole idea.

import random
import time

CACHE_SECONDS = 10
_cache = {}  # service name -> (expires_at, addresses)
lookups = 0  # counts trips to the registry, for this demo only

# A fake registry so the sketch runs on its own. None means "registry down".
fake_registry = {"episode-service": ["10.0.4.17:8080", "10.0.7.42:8080"]}

def registry_lookup(service):
    global lookups
    lookups += 1
    if fake_registry is None:
        raise ConnectionError("registry unreachable")
    return list(fake_registry.get(service, []))

def find_instances(service):
    expires_at, addresses = _cache.get(service, (0.0, []))
    if time.monotonic() < expires_at:
        return addresses  # fresh enough: no registry trip
    try:
        fresh = registry_lookup(service)  # passing instances only
    except ConnectionError:
        fresh = []
    if fresh:
        _cache[service] = (time.monotonic() + CACHE_SECONDS, fresh)
        return fresh
    return addresses  # registry down or empty: keep the last good list

def pick_instance(service):
    return random.choice(find_instances(service))  # simple load balancing

random.seed(7)
picks = [pick_instance("episode-service") for _ in range(1000)]
print("registry trips for 1000 picks:", lookups)
print("share per instance:", {a: picks.count(a) for a in sorted(set(picks))})

fake_registry = None                      # the registry goes down
_cache["episode-service"] = (0.0, _cache["episode-service"][1])  # cache expires
print("registry down, still picking:", pick_instance("episode-service"))

try:
    pick_instance("search-service")       # never seen, registry still down
except IndexError as err:
    print("unknown service with registry down:", type(err).__name__)

Running it with Python 3.13.16 prints this:

registry trips for 1000 picks: 1
share per instance: {'10.0.4.17:8080': 511, '10.0.7.42:8080': 489}
registry down, still picking: 10.0.4.17:8080
unknown service with registry down: IndexError

What just happened? A thousand picks cost one trip to the registry, because answers are cached for 10 seconds. The two instances shared the picks almost evenly, which is what uniform random picks give over many requests.

When the registry went down, the client kept using its last good list. For a service it had never seen, it had no list at all. random.choice then raised IndexError, which is better than calling a made-up address.

The catch is that every language needs this code. A team with services in Python, Go and Java would maintain three clients with the same caching rules.

Server-side discovery

Server-side discovery moves that logic into one shared place. The player only knows one stable name, and the proxy does the rest.

With a separate load balancer or proxy, the price is an extra network hop on every request and one more part that must stay up. That proxy must run as several copies, or it becomes the new single point of failure. Kubernetes avoids the extra hop, because it writes the forwarding rules into every node.

The table compares the three common ways to find a service, using the player as the caller.

DNS-based service discovery puts instance addresses behind an ordinary DNS name. A caller looks up the name and gets back one or more IP addresses, just as a browser does for a website.

Its big strength is that every language and tool already speaks DNS. Nothing new goes into the service code. Kubernetes and Consul both answer DNS queries.

Plain address records only give IPs, not ports. SRV records fill that gap. RFC 2782, the standard that defines them, uses names like _ldap._tcp.example.com: the service, the protocol, then the domain.

Each SRV answer carries a priority, a weight, a port and a target host. Clients try the lowest priority number first. Among equal priorities, a higher weight means a bigger share of requests.

The RFC says weights are only for static selection. They are fixed shares, not a way to react to live load.

Why caching is the weak spot

Every DNS answer has a TTL, and resolvers, operating systems and language runtimes may keep answers. Some keep them longer than the TTL. That is the evening outage again, in a new form.

Pause and predict: a Java program looks up a name, and right after that the address changes. When does the program first see the new address?

We tested it on OpenJDK 21.0.12 (Ubuntu build 21.0.12.1), changing the address in a test hosts file that Java read through its jdk.net.hosts.file option. Checks every 5 seconds got the old address up to 30 seconds and the new one at 35.

That matches the JDK's own java.security file. With no security manager installed, it caches successful lookups for 30 seconds. With a security manager, the default is to cache forever. That applies up to JDK 23; from JDK 24, a security manager cannot be enabled at all.

To check your own Java setting, run java -XshowSettings:security:properties -version and look for networkaddress.cache.ttl. On our OpenJDK 21 it was not listed at all, which means the 30 second default applies. If networkaddress.cache.ttl is not set, Java falls back to the older system property sun.net.inetaddr.ttl. Set to 5, our test saw the new address by 10 seconds. If both are set, networkaddress.cache.ttl wins: with 30 and 5, the new address still took 35 seconds.

Kubernetes gives each Service object a stable name and a stable virtual IP called the cluster IP. A virtual IP is an address that no single machine owns. It keeps the list of matching pods up to date, so callers never track pod addresses themselves.

A pod is the smallest unit Kubernetes runs, usually one container of your service. Pods come and go, and each gets its own IP. A Service uses a selector, a label match, to decide which pods belong to it.

A controller keeps scanning for matching pods and records their addresses in objects called EndpointSlices. A cluster DNS server, such as CoreDNS, answers lookups for Service names.

Here is the episode service on Kubernetes. The first document is the Service. The second is a fragment from the episode pods' container spec.

apiVersion: v1
kind: Service
metadata:
  name: episode-service
  namespace: catalog
spec:
  selector:
    app: episode-service
  ports:
    - name: http
      port: 80
      targetPort: 8080
---
# fragment: inside the episode pods' container spec
readinessProbe:
  httpGet: {path: /ready, port: 8080}
  periodSeconds: 5
  failureThreshold: 2

The selector picks every pod labelled app: episode-service. Callers use port 80, and Kubernetes forwards to port 8080 inside the pod. A namespace is a named group of resources inside a cluster. Inside the catalog namespace, the player can simply call episode-service.

From other namespaces, use episode-service.catalog, or the full name episode-service.catalog.svc.cluster.local. Here cluster.local is the usual default cluster domain. Because the port is named http, an SRV record also exists for _http._tcp on that name.

What a readiness probe really changes

The readiness probe is the health check. The kubelet, the agent on each machine that runs pods, calls /ready every 5 seconds. Two failures in a row mark the pod not ready, without restarting it.

So a pod that stops answering is marked not ready after roughly 10 seconds. With the defaults, every 10 seconds and 3 failures, it takes roughly 30 seconds.

Here the docs and the code describe it differently, as of 19 October 2026. The probes page says the controller removes the pod's IP from the EndpointSlices. In the controller source, the pod stays listed, with its ready condition set to false.

Traffic still stops. kube-proxy, the component that writes each node's service rules, sends traffic only to ready endpoints. It falls back to endpoints that are shutting down but still serving only when none are ready. So a Service is server-side discovery built into every node, with no separate proxy in the path.

You can see this yourself. Run kubectl get endpointslices -n catalog -l kubernetes.io/service-name=episode-service -o yaml. Each endpoint has conditions; a pod failing readiness shows ready: false.

A Service's cluster IP cannot be changed by an update, so callers can cache it safely. If someone deletes and recreates the Service, it usually gets a new one.

Some callers want every pod address instead, for example a database client that must reach each replica. A headless Service has no cluster IP, so its DNS name returns the IPs of the selected pods.

Most discovery failures come from old answers, a registry outage, or health checks that lie. Each one has a known guard.

• Old answers. A caller keeps a list after instances have moved. Guard: short cache times, and retrying one other instance on a connection error.
• Registry outage. Nobody can look anything up. Guard: run the registry as a cluster of several nodes, and let clients keep their last good list.
• Health checks that lie. A check passes while real requests fail. Guard: make the check exercise the same code path as a real request.
• Flapping. An instance flips between healthy and unhealthy every few seconds. Guard: require two or three failures in a row before hiding it.
• Slow shutdown. An instance stops before callers learn it is leaving. Guard: deregister or fail readiness first, wait, then stop.
• Wrong environment. A test instance registers in the production registry. Guard: separate registries or namespaces, and rules on who may register.

The registry deserves extra care because everything depends on it. Consul and etcd both use Raft, a consensus protocol in which several nodes agree on one copy of the data.

Consul recommends 3 or 5 servers for production. Three servers survive one failure, and five survive two, because a majority must still agree.

Consul also runs a gossip protocol called Serf, a way for nodes to spread news by passing messages among themselves. Consul uses it to track members and spread failure detection across the cluster.

For a safe shutdown with a 10 second client cache, the team's deploy script follows a fixed order.

1. 1

   Stop being ready

   The instance starts failing its readiness check, or deregisters itself, so the registry stops handing out its address.

2. 2

   Wait for callers to notice

   The script waits longer than the client cache time, here 15 seconds against a 10 second cache.

3. 3

   Finish work in flight

   The instance completes requests it already accepted, and refuses new connections.

4. 4

   Stop the process

   Only now does the instance exit. Callers have already moved to other instances, so nobody sees an error.

On Kubernetes, step 1 and the start of shutdown happen together. Deleting a pod marks its endpoints as terminating, with ready set to false, while the kubelet starts stopping the pod.

A preStop hook, an action the kubelet runs before it sends the stop signal, is the place for step 2's short wait. The whole shutdown must fit in the grace period, 30 seconds by default.

Explain it as the answer to one problem: instances move, so callers must ask by name. Then walk through registration, health, lookup and failure in that order.

1. 1

   State the problem

   Autoscaling and deploys give instances new addresses, so fixed address lists break. Mention a concrete case, like new instances sitting idle.

2. 2

   Add a registry

   Instances register by name, address and port, through self-registration or a platform watcher.

3. 3

   Keep it honest with health checks

   Name the check type, how often it runs, and that failing instances are hidden before they are removed.

4. 4

   Choose a lookup style

   Pick client-side, server-side or DNS, and say why for this system. On Kubernetes, say that Services and DNS already provide it.

5. 5

   Cover failure

   Say how callers behave when the registry is down, how long caches last, and how instances shut down safely.