[RayService] Migrate from Endpoints API to EndpointSlice API for RayService #4245

seanlaii · 2025-12-01T22:18:30Z

Why are these changes needed?

The Kubernetes Endpoints API has been deprecated since v1.21 in favor of the more scalable EndpointSlice API.
The Endpoints API has limitations when handling services with many endpoints (single object size limits), while EndpointSlice supports up to 100 endpoints per slice and can scale horizontally.
This migration ensures KubeRay follows Kubernetes best practices and prepares for future Kubernetes versions where the Endpoints API may be removed.

Note:
We need to mention that the endpoints RBAC has been deprecated, and we will remove the endpoints RBAC in the new future release in the release note.

Related issue number

Closes #4127

Checks

I've made sure the tests are passing.
Testing Strategy
- Unit tests
- Manual tests
- This PR is not tested :(

win5923

Could you also open a PR in the Ray doc to mention that starting from v1.6.0, the KubeRay Operator will use EndpointSlice for RayService? Thanks!

helm-chart/kuberay-operator/templates/_helpers.tpl

seanlaii · 2025-12-03T22:35:16Z

Could you also open a PR in the Ray doc to mention that starting from v1.6.0, the KubeRay Operator will use EndpointSlice for RayService? Thanks!

Yes, will open up a PR to add the info in the Ray doc.

seanlaii · 2025-12-03T22:36:44Z

Hi @rueian @Future-Outlier @andrewsykim , please help review the PR when you get a chance. Thanks!

ray-operator/controllers/ray/rayservice_controller.go

nojnhuh · 2025-12-05T18:27:16Z

ray-operator/controllers/ray/rayservice_controller.go

+
+	// Count ready endpoints across all EndpointSlices.
+	numReadyEndpoints := 0
+	for _, endpointSlice := range endpointSliceList.Items {


If the Service happens to be IPv4/IPv6 dual-stack, then counting each individual address from all EndpointSlices will double-count Pods that have both an IPv4 and IPv6 address. Not sure if dual-stack is supported, but it would be inconsistent with how the RayService status field is documented:

// NumServeEndpoints indicates the number of Ray Pods that are actively serving or have been selected by the serve service.

Thanks for pointing this out! You are absolutely right. Although KubeRay doesn't fully support dual-stack clusters yet (see issue #4205), the logic should strictly align with the API definition of counting Pods, not addresses. I will update the implementation to deduplicate endpoints based on their Pod UID to ensure the count remains accurate regardless of the networking stack.

cc @Future-Outlier @rueian

Future-Outlier

Overall LGTM, plz ping me when you all comments are addressed, and sorry for the late review.

.buildkite/test-sample-yamls.yml

ray-operator/controllers/ray/rayservice_controller.go

ray-operator/test/support/utils.go

ray-operator/test/e2erayservice/rayservice_upgrade_test.go

Signed-off-by: seanlaii <[email protected]>

seanlaii · 2025-12-06T19:51:02Z

Hi @Future-Outlier , no worries at all! Thanks for the review. I have addressed all your comments. Please help take a look when you get a chance!

seanlaii · 2025-12-06T20:45:55Z

helm-chart/kuberay-operator/templates/_helpers.tpl

 */}}
 {{- define "role.consistentRules" -}}
 rules:
+# TODO: Remove endpoints permission in a future release.


Tracked the removal progress in #4255.

Future-Outlier · 2025-12-07T01:04:11Z

helm-chart/kuberay-operator/templates/_helpers.tpl

+# TODO: Remove endpoints permission in a future release.
+# This permission is retained for backward compatibility to support users deploying older operator
+# images with the new Helm chart. Older operator versions still use the deprecated Endpoints API.
+# This can be safely removed once those older versions are no longer supported.


thank you!!
I think it's ok to remove endpoints permission in this PR, since when you use helm install, your kuberay operator will have the old release's rbac permission.

so 3 files to change.

helm-chart/kuberay-operator/templates/_helpers.tpl

ray-operator/config/rbac/role.yaml

ray-operator/controllers/ray/rayservice_controller.go
(the kubebuilder part)

But if they use the new helm chart with old images, there will be an error. I think this is a breaking change and it might be better to decouple the helm chart version and the image version.

I think this is okay, since we upgrade the KubeRay operator via the Helm chart like this.
In this case, it will not be a breaking change, because we will have a Helm chart version that maps to each KubeRay operator version.

helm repo add kuberay https://ray-project.github.io/kuberay-helm/ helm repo update # Install both CRDs and KubeRay operator v1.4.2. helm install kuberay-operator kuberay/kuberay-operator --version 1.4.2

OK, will remove the endpoint RBAC. We might need to add this in the release doc as it will be a breaking change for users who use the helm chart with value override.

no problem, thank you!
let me add to my release draft notes now.

Removed and closed the tracking issue as well.
Please help review when you get a chance. Thank you!

Future-Outlier · 2025-12-07T01:34:52Z

ray-operator/controllers/ray/rayservice_controller.go

 // +kubebuilder:rbac:groups=core,resources=pods/status,verbs=get;list;watch;create;update;patch;delete
 // +kubebuilder:rbac:groups=core,resources=pods/proxy,verbs=get;update;patch
-// +kubebuilder:rbac:groups=core,resources=endpoints,verbs=get;list;watch
+// +kubebuilder:rbac:groups=discovery.k8s.io,resources=endpointslices,verbs=get;list;watch


Maybe move this line to the bottom of the RBAC section so that related permissions stay grouped together.

Future-Outlier · 2025-12-07T02:20:54Z

cc @rueian to merge

rueian · 2025-12-08T00:18:57Z

ray-operator/controllers/ray/rayservice_controller.go

+
+	numPods := len(uniqueNumReadyPods)
+
+	logger.Info("Counted serve-ready pods via EndpointSlices",


Wouldn't this result in too many logs? Using the debug level could be better.

Sure, updated by changing the log level.

Future-Outlier

LGTM again, thank you!

seanlaii changed the title ~~Migrate from Endpoints API to EndpointSlice API for RayService~~ [RayService] Migrate from Endpoints API to EndpointSlice API for RayService Dec 1, 2025

seanlaii force-pushed the endpointslices branch from 74c42aa to 50e6c19 Compare December 1, 2025 22:49

seanlaii marked this pull request as ready for review December 1, 2025 23:54

seanlaii requested review from MortalHappiness, andrewsykim, kevin85421 and rueian as code owners December 1, 2025 23:54

win5923 reviewed Dec 3, 2025

View reviewed changes

helm-chart/kuberay-operator/templates/_helpers.tpl Outdated Show resolved Hide resolved

seanlaii mentioned this pull request Dec 3, 2025

Remove Endpoint permission in a future release #4255

Closed

2 tasks

nojnhuh reviewed Dec 5, 2025

View reviewed changes

Future-Outlier requested changes Dec 6, 2025

View reviewed changes

seanlaii force-pushed the endpointslices branch from 095342f to 4a8c679 Compare December 6, 2025 19:45

seanlaii added 7 commits December 6, 2025 14:46

Migrate from Endpoints API to EndpointSlice API for RayService

647c824

Signed-off-by: seanlaii <[email protected]>

trigger test

2b0eaa8

add back endpoints rule for backward compatibility

f1f6cac

add comment

28b6a8e

fix comment

ca536a2

de-duplicate endpoint based on pod uid

77b0807

address comment

e8c1c24

seanlaii force-pushed the endpointslices branch from 4a8c679 to e8c1c24 Compare December 6, 2025 19:46

seanlaii force-pushed the endpointslices branch from 113acaa to db85876 Compare December 6, 2025 20:40

seanlaii commented Dec 6, 2025

View reviewed changes

change TODO message

321d7b5

seanlaii force-pushed the endpointslices branch from db85876 to 321d7b5 Compare December 6, 2025 20:49

trigger test

3f4c39f

Future-Outlier reviewed Dec 7, 2025

View reviewed changes

remove endpoint RBAC

1023900

Future-Outlier approved these changes Dec 7, 2025

View reviewed changes

Future-Outlier reviewed Dec 7, 2025

View reviewed changes

move comment

ebf2076

rueian reviewed Dec 8, 2025

View reviewed changes

change logging level

0110627

Future-Outlier approved these changes Dec 8, 2025

View reviewed changes

rueian approved these changes Dec 8, 2025

View reviewed changes

rueian merged commit e32405e into ray-project:master Dec 8, 2025
27 checks passed

seanlaii deleted the endpointslices branch December 8, 2025 15:38


		numPods := len(uniqueNumReadyPods)

		logger.Info("Counted serve-ready pods via EndpointSlices",

[RayService] Migrate from Endpoints API to EndpointSlice API for RayService #4245

[RayService] Migrate from Endpoints API to EndpointSlice API for RayService #4245

Uh oh!

Conversation

seanlaii commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Checks

Uh oh!

win5923 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

seanlaii commented Dec 3, 2025

Uh oh!

seanlaii commented Dec 3, 2025

Uh oh!

Uh oh!

nojnhuh Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

seanlaii Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Future-Outlier left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

seanlaii commented Dec 6, 2025

Uh oh!

seanlaii Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Future-Outlier Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

seanlaii Dec 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Future-Outlier Dec 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seanlaii Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Future-Outlier Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

seanlaii Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Future-Outlier Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Future-Outlier commented Dec 7, 2025

Uh oh!

rueian Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

seanlaii Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Future-Outlier left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

seanlaii commented Dec 1, 2025 •

edited

Loading

seanlaii Dec 5, 2025 •

edited

Loading

seanlaii Dec 6, 2025 •

edited

Loading

seanlaii Dec 7, 2025 •

edited

Loading

Future-Outlier Dec 7, 2025 •

edited

Loading

seanlaii Dec 8, 2025 •

edited

Loading