-
Notifications
You must be signed in to change notification settings - Fork 182
Implement shared session capability #1489
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rikatz
wants to merge
5
commits into
kubernetes:master
Choose a base branch
from
rikatz:implement-shared-session-capability
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+561
−32
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
50e2ba2
Implement vc session manager client
rikatz d278279
Implement vc session manager on credential manager
rikatz c187643
Implement vc session manager on vc client
rikatz 1aadbdd
Support using serviceaccount on session auth
rikatz 2f0981e
Add docs on shared session implementation
rikatz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
# vSphere Shared Session capability | ||
|
||
One problem that can be found when provisioning a large amount of clusters using | ||
vSphere Cloud Provider is vCenter session exhaustion. This happens because every | ||
workload cluster needs to request a new session to vSphere to do proper reconciliation. | ||
|
||
vSphere 8.0U3 and up uses a new approach of session management, that allows the | ||
creation and sharing of the sessions among different clusters. | ||
|
||
A cluster admin can implement a rest API that, once called, requests a new vCenter | ||
session and shares with CPI. This session will not count on the total generated | ||
sessions of vSphere, and instead will be a child derived session. | ||
|
||
This configuration can be applied on vSphere Cloud Provider with the usage of | ||
the following secret/credentials, instead of vSphere Username/password: | ||
|
||
```yaml | ||
apiVersion: v1 | ||
kind: Secret | ||
metadata: | ||
namespace: kube-system | ||
name: vsphere-cloud-secret | ||
stringData: | ||
your-vcenter-host.vc-session-manager-url: "https://shared-session-service.tld/session" | ||
your-vcenter-host.vc-session-manager-token: "authenticationtoken" | ||
``` | ||
The configuration above will make CPI call the shared session rest API and use the | ||
provided token to authenticate against vSphere, instead of using a username/password. | ||
The parameter provider at `vc-session-manager-token` is sent as a `Authorization: Bearer` token | ||
to the session manager, and in case this directive is not configured CPI will send the | ||
Pod Service Account token instead. | ||
|
||
Below is an example implementation of a shared session manager rest API. Starting the | ||
program below and calling `http://127.0.0.1:18080/session` should return a JSON that is expected | ||
by CPI using session manager to work: | ||
|
||
```shell | ||
$ curl 127.0.0.1:18080/session | ||
{"token":"cst-VCT-52f8d061-aace-4506-f4e6-fca78293a93f-....."} | ||
``` | ||
|
||
**NOTE**: Below implementation is **NOT PRODUCTION READY** and does not implement | ||
any kind of authentication! | ||
|
||
```go | ||
package main | ||
import ( | ||
"context" | ||
"encoding/json" | ||
"log" | ||
"net/http" | ||
"net/url" | ||
"github.com/vmware/govmomi" | ||
"github.com/vmware/govmomi/session" | ||
"github.com/vmware/govmomi/vim25" | ||
"github.com/vmware/govmomi/vim25/soap" | ||
) | ||
const ( | ||
vcURL = "https://my-vc.tld" | ||
vcUsername = "[email protected]" | ||
vcPassword = "somepassword" | ||
) | ||
var ( | ||
userPassword = url.UserPassword(vcUsername, vcPassword) | ||
) | ||
// SharedSessionResponse is the expected response of CPI when using Shared session manager | ||
type SharedSessionResponse struct { | ||
Token string `json:"token"` | ||
} | ||
|
||
func main() { | ||
ctx := context.Background() | ||
vcURL, err := soap.ParseURL(vcURL) | ||
if err != nil { | ||
panic(err) | ||
} | ||
soapClient := soap.NewClient(vcURL, false) | ||
c, err := vim25.NewClient(ctx, soapClient) | ||
if err != nil { | ||
panic(err) | ||
} | ||
client := &govmomi.Client{ | ||
Client: c, | ||
SessionManager: session.NewManager(c), | ||
} | ||
if err := client.SessionManager.Login(ctx, userPassword); err != nil { | ||
panic(err) | ||
} | ||
|
||
vcsession := func(w http.ResponseWriter, r *http.Request) { | ||
clonedtoken, err := client.SessionManager.AcquireCloneTicket(ctx) | ||
if err != nil { | ||
w.WriteHeader(http.StatusForbidden) | ||
return | ||
} | ||
token := &SharedSessionResponse{Token: clonedtoken} | ||
jsonT, err := json.Marshal(token) | ||
if err != nil { | ||
w.WriteHeader(http.StatusInternalServerError) | ||
return | ||
} | ||
w.WriteHeader(http.StatusOK) | ||
w.Write(jsonT) | ||
} | ||
|
||
http.HandleFunc("/session", vcsession) | ||
log.Printf("starting webserver on port 18080") | ||
http.ListenAndServe(":18080", nil) | ||
} | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: can just pass credential obj.