The RESTful Web Services module works by implementing an event subscriber service, rest.resource_routes, that adds routes to Drupal based on implementations of its RestResource plugin. Each plugin returns the available routes based on HTTP methods that are enabled for the resource.
When routes are built, the \Drupal\rest\Routing\ResourceRoutes class uses the RestResource plugin manager to retrieve all the available definitions. The endpoint configuration objects are loaded and inspected. If the resource plugin provides an HTTP method that is enabled in the configuration definitions, it begins to build a new route. Verification is done against the defined supported formats and supported auth definitions. If the basic validation passes, the new route is added to the RouteCollection and returned.
If you provide a supported_formats or supported_auth value that is not available, the endpoint will still be created. There will be an error, however, if you attempt to use the route with the invalid plugin. This cannot occur when using the REST UI module, but manually providing and managing the configuration.
The default routes provided by the base class for resource plugins, \Drupal\rest\Plugin\ResourceBase class, set \Drupal\rest\RequestHandler::handle as the controller and method for the route. This method checks the passed _format parameter against the configured plugin. If the format is valid, the data is passed to the appropriate serializer. The serialized data is then returned in the request with appropriate content headers.