Skip to content

feat(sdk/go): add Go SDK foundation, types, and sandbox client (A)#2271

Open
rhuss wants to merge 3 commits into
NVIDIA:mainfrom
rhuss:go-sdk-a-foundation
Open

feat(sdk/go): add Go SDK foundation, types, and sandbox client (A)#2271
rhuss wants to merge 3 commits into
NVIDIA:mainfrom
rhuss:go-sdk-a-foundation

Conversation

@rhuss

@rhuss rhuss commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Context

This is the first PR in a 6-PR decomposition of the Go SDK contribution (#2044). The decomposition was discussed in the contributor meeting on 2026-07-14 to make the review process more approachable.

The first PR is intentionally the largest because it carries the shared foundation. After this merge, the SDK is usable end-to-end for sandbox management. Each subsequent PR then incrementally adds one more resource group, and after every merge the SDK is fully working with an expanded API surface.

PR What Code Tests Status
This PR (A) Foundation + types + sandbox ~4.6K ~8.5K ready for review
B Exec + file + health ~1.2K ~1.8K after A merges
C Provider + profile + config + refresh ~2.2K ~3.3K after B merges
D Policy + service + TCP + SSH ~2.6K ~3.9K after C merges
E Gateway + OIDC + edge + fakes TBD TBD after D merges
F Docs + CI ~0.5K - after E merges

What's in this PR

  • Module setup: go.mod, go.sum, Makefile, mise.toml
  • All domain types: types/ package (14 files) covering every SDK resource
  • Full ClientInterface: all 10 sub-client accessors defined upfront
  • Shared infrastructure: errors, auth primitives, gRPC connection, logging
  • Sandbox client: fully functional with converter and tests
  • Stub clients: all other resources return Unimplemented errors linking to feat(sdk/go): Go SDK PR decomposition plan #2270. Each subsequent PR replaces stubs with real implementations.
sdk/go/
├── go.mod, go.sum, Makefile, mise.toml
├── proto/                              # Proto definitions + generated .pb.go
├── openshell/v1/
│   ├── types/                          # All domain types (14 files)
│   ├── internal/converter/             # Proto-to-SDK converters
│   ├── internal/grpc/                  # Connection management
│   ├── client.go                       # ClientInterface + Client struct
│   ├── sandbox.go + sandbox_client.go  # Sandbox (real implementation)
│   ├── stub_clients.go                 # Stubs for resources not yet implemented
│   ├── auth*.go                        # Auth providers
│   ├── errors.go                       # Typed errors with IsNotFound() etc.
│   └── {exec,file,health,...}.go       # Interface definitions for all resources

How to Review

Review zones

Zone Files What to do
Must-review client.go, types/*.go, errors.go, auth*.go, sandbox.go, sandbox_client.go, internal/grpc/conn.go These define the API surface and core logic. Read carefully.
Pattern-review sandbox_client_test.go, internal/converter/sandbox.go, internal/converter/sandbox_test.go Review sandbox_client_test.go as the test pattern exemplar. Converter tests follow table-driven patterns.
Skim stub_clients.go, go.sum, Makefile, mise.toml, doc.go, interface-only files (exec.go, file.go, etc.) Stubs are mechanical. Interface files are just type declarations.
Skip proto/*.pb.go, proto/*_grpc.pb.go Generated code.

Key design decisions

  • client-go conventions: typed sub-clients per resource, watch primitives, typed errors
  • Domain types separate from proto: types/ package has no proto imports, insulating consumers from wire format changes
  • Stub pattern for incremental delivery: stubs return ErrorUnimplemented with a link to the tracking issue. Each follow-up PR replaces stubs with real implementations without modifying client.go.
  • All types upfront: the full domain model ships in this PR (~1K lines) so reviewers see the complete API shape once

What to look for

  • ClientInterface covers the right API surface
  • Type definitions match gRPC API semantics
  • Error handling follows typed error conventions (IsNotFound(), etc.)
  • Auth provider interface is extensible
  • Sandbox client test coverage is adequate

Testing

All 130 tests pass:

go test ./...   # 130 passed in 7 packages

Resolves #2044 (with remaining PRs B-F)
Part of #2270

Add the Go SDK module with the full API contract and a working sandbox
client as the first vertical slice. All other resource clients are present
as stubs returning Unimplemented errors, to be replaced with real
implementations in subsequent PRs.

Contents:
- Module setup (go.mod, Makefile, mise.toml)
- All domain types (types/ package)
- Full ClientInterface with all sub-client accessors
- Shared infrastructure (errors, auth, gRPC connection, logging)
- Sandbox client with converter and tests (fully functional)
- Stub clients for remaining resources (exec, file, health, provider,
  profile, config, refresh, policy, service, ssh, tcp)

Part of the Go SDK decomposition plan (NVIDIA#2270).
Implements NVIDIA#2044.
@rhuss rhuss requested review from a team, derekwaynecarr, maxamillion and mrunalp as code owners July 14, 2026 18:44
@copy-pr-bot

copy-pr-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

}
opts = append(opts, grpc.WithTransportCredentials(creds))
} else {
opts = append(opts, grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{MinVersion: tls.VersionTLS12})))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[codex:gpt-5.5] http:// gateway addresses are accepted but still dialed with TLS. NewConnection strips both https:// and http://, then falls through to credentials.NewTLS(...) whenever TLSConfig is nil. That diverges from the existing CLI behavior, where http:// explicitly means plaintext gRPC, and it will make Go users fail against loopback/plaintext gateway URLs unless they also know to set TLS: &TLSConfig{Insecure: true}. Please make scheme parsing drive transport selection (http:// -> insecure.NewCredentials(), https:// -> TLS, no scheme -> documented default) and add a regression test for the http:// case.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, will fix. The scheme should drive transport selection instead of being silently stripped. http:// should use insecure.NewCredentials(), https:// or no scheme should use TLS. Will add a regression test for the http:// case.

Annotations map[string]string
Environment map[string]string
UserNamespaces *bool
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[codex:gpt-5.5] The implemented sandbox client cannot express the current template resource fields. The public proto has SandboxTemplate.resources for CPU/memory/resource limits and SandboxTemplate.driver_config for driver-specific settings, and the server actively validates and forwards both into compute drivers. This SDK type only exposes image/runtime/agent socket/labels/annotations/env/user namespaces, so Create will silently be unable to create sandboxes that need CPU/memory limits or driver config such as Docker CDI devices or Kubernetes scheduling. Please add these fields to the SDK SandboxTemplate type and preserve them in both SandboxSpecToProto and SandboxFromProto.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, SandboxTemplate is missing resources and driver_config fields that the proto defines. Will add them to the SDK type and update both converter directions.

Comment thread sdk/go/proto/UPSTREAM_VERSION Outdated
@@ -0,0 +1 @@
29ce6a704cba222c29b5e0d73b90280cf5ed3b9f

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[codex:gpt-5.5] The vendored Go proto snapshot is behind the repository proto sources. For example, current proto/sandbox.proto includes NetworkEndpoint.credential_signing, signing_service, signing_region, json_rpc_max_body_bytes, mcp, and MCP params matchers, but sdk/go/proto/sandbox.proto stops at advisor_proposed. Current proto/openshell.proto also reserves volume_claim_templates, while the SDK snapshot still generates that field. Because the generated Go types and SDK domain model are built from this snapshot, Go clients cannot represent current policy features such as SigV4/MCP and may serialize a field the current API has explicitly removed. Please regenerate from the current proto/ sources in this branch and add a check that fails when sdk/go/proto/*.proto drift from the canonical proto files.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. This is a first drop based on a proto snapshot that will naturally drift. Proto freshness CI checks are planned for a later PR in the series (PR F in #2270), and proto sync is also an operational requirement in the SDK acceptance scorecard RFC (#2293). For this PR, I will regenerate from the current proto sources. The CI guard will prevent future drift.

@russellb

Copy link
Copy Markdown
Contributor

🤖 This review was generated with Claude Code using Opus 4.8 (1M context). Findings were verified by building the module, running go vet, the test suite, staticcheck, and diffing the vendored proto against the canonical proto/. Treat as reviewer input, not ground truth.

Principal Engineer Review — Go SDK foundation (A)

Reviewed by checking out the branch and reading every non-generated file. go build/go vet are clean and the 130 tests pass at ~73% package coverage. Findings are ordered by severity; all are actionable.

Blocking

1. Dead code will fail the project's own lint gate — sdk/go/openshell/v1/internal/converter/copy.go:51

boolCount is defined but never referenced anywhere in the module (verified across all .go including tests). unused is a default golangci-lint linter, and mise run ci depends on lint. Confirmed with staticcheck:

copy.go:51:6: func boolCount is unused (U1000)

This contradicts the "CI green" claim — mise run lint should be red. Delete boolCount (or wire it into the log-option validation it was presumably written for).

2. Three public Config fields are silent no-ops — sdk/go/openshell/v1/types/config.go:13-15, client.go:64

Config.Timeout, Config.RetryPolicy, and Config.Logger are declared on the public config struct but never read anywhere in the client or connection path (verified by grep). A user who sets Timeout: 30*time.Second or a RetryPolicy gets zero behavioral change, with no error and no doc warning. For the PR that "carries the shared foundation," shipping config knobs that do nothing bakes in an interface people will rely on, and later wiring them up becomes a behavior change. Either implement them (dial/context timeout, gRPC retryPolicy service-config, connection logging) or drop them from this PR and add each alongside its implementation. At minimum, document them as reserved/no-op.

High

3. Vendored proto is hand-edited and drifted from canonical proto/, and proto:sync will clobber the edits — sdk/go/proto/*, sdk/go/mise.toml:204

  • UPSTREAM_VERSION pins 29ce6a70…, which is not resolvable in this repo's history — the snapshot isn't reproducible/verifiable from here.
  • The vendored openshell.proto has been manually stripped of import "options.proto" and every [(…secret) = true] annotation (because options.proto isn't vendored).
  • It has diverged from main: e.g. volume_claim_templates = 9 is still present here but is reserved 9 on main; the newer annotations = 4 request field on main is missing.
  • proto:sync does a raw cp "$UPSTREAM_PATH/*.proto", which will re-introduce import "options.proto" and the secret annotations, immediately breaking proto:gen (protoc can't find options.proto).

Net: the generated bindings are built against a stale, hand-modified contract, and the "sync" automation is not idempotent with the manual edits. proto:check only verifies .pb.go matches the local .proto, not that the local .proto matches upstream — so drift is undetected. Recommend vendoring options.proto (or applying a scripted, repeatable transform), making proto:sync reproduce the exact committed state, and adding a check that the vendored proto equals the pinned upstream.

4. Package doc advertises functionality that returns Unimplemented in this PR — sdk/go/openshell/v1/doc.go

The package overview gives copy-paste examples for Exec().Run, Services().Expose, Providers().Profiles(), SSH().CreateSession/Tunnel, TCP().Forward, Config().Update/GetSandbox, and Policy().List (doc.go:37-357). Every one is a stub returning ErrorUnimplemented until PRs B–F. After A merges, pkg.go.dev presents these as working, and a user following the Quick Start past Sandboxes() hits runtime Unimplemented with no compile-time signal. Scope the package doc to what actually works in A, or clearly mark the not-yet-available sections.

Medium

5. internal/grpc/conn.go has zero test coverage — sdk/go/openshell/v1/internal/grpc/conn.go

The connection package ([no test files]) contains the only security-sensitive logic in the PR: TLS default selection, buildTLSCredentials, CA-pool loading, mTLS keypair loading, and the both-CertFile-and-KeyFile invariant. None of it is tested. NewConnection is easily unit-testable (temp cert files; assert error paths for bad CA / half-configured client cert / scheme stripping). Given this is the shared foundation, the crypto path deserves tests now.

6. WatchOptions fields silently ignored — sdk/go/openshell/v1/types/options.go:29-30, sandbox_client.go:174-190

Watch reads only StopOnTerminal; TimeoutSeconds and LabelSelector are never applied. Same silent no-op problem as #2. For a single-object watch, LabelSelector is meaningless — drop it (or document intent), and either honor TimeoutSeconds or remove it in favor of the documented "use context for timeout."

7. EventAdded is defined and re-exported but never emitted — sdk/go/openshell/v1/sandbox_client.go:211

The doc claims the watcher is "Modeled after k8s.io/apimachinery/pkg/watch.Interface," but the initial object and all updates are delivered as EventModified; EventAdded is dead. k8s consumers expect the first delivery to be ADDED. Either emit EventAdded for the first event (the code already handles first separately, so it's a one-line branch) or drop the constant to avoid implying semantics you don't provide.

Low / nits

8. Watch error events can be silently dropped — sandbox_client.go:233-236

The terminal EventError is sent with a non-blocking select { … default: }. If the 64-slot buffer is full (slow consumer), the stream error is dropped and the consumer only sees a closed channel — indistinguishable from clean EOF. Consider a dedicated error field on the watcher, or block on the send guarded by w.done.

9. Watch blocks until the first server event — sandbox_client.go:196

stream.Recv() runs synchronously before Watch returns, so the call blocks (bounded only by ctx) until the gateway produces the first event. Callers reasonably expect Watch to return promptly and stream thereafter. Document it, or move the first Recv into the goroutine.

10. FromGRPCError loses detail and lets non-status errors bypass typing — internal/converter/errors.go:35-52

StatusError.Details is never populated (the field is dead across the SDK), and when status.FromError returns ok=false the raw error is returned unwrapped, so a caller's IsX() checks silently return false for it. The unused Details field is misleading API surface.

11. Unchecked int → uint32 conversions — sandbox_client.go:54,57

uint32(opts[0].Limit) / uint32(opts[0].Offset) truncate on large values (only guarded against negatives by > 0). Not caught by the default linters, but would trip gosec G115 if adopted; cheap to bound-check.

12. Mock server has unsynchronized map access — sandbox_client_test.go:72,102-106,113-117

CreateSandbox, ListSandboxes, and DeleteSandbox touch s.sandboxes without holding s.mu, while GetSandbox/setPhase do lock. -race passes today only because tests don't currently overlap those calls with setPhase goroutines; latent flakiness in the file nominated as the test-pattern exemplar. Lock consistently.

13. The only non-skipped integration test can't pass — integration_test.go:25-34

TestIntegration_HealthCheck calls Health().Check(), which is a stub returning Unimplemented, so against a real gateway require.NoError fails; the others are t.Skip("TODO"). It's build-tagged so normal CI skips it, but as written it's a broken test. Skip it too, or gate on Health landing in PR B.

14. refreshableAuth holds the write lock across the network refresh — auth_refresh.go:83-112

source.Token() (a network call) runs under mu.Lock(), so every concurrent RPC's metadata fetch blocks for the full refresh duration. Correct single-flight behavior, but "coalesced" undersells that it fully serializes callers during refresh. Acceptable; worth a note.


Overall: the structure (typed sub-clients, types/ isolated from proto, converters, typed errors, watch primitives) is sound and the sandbox path is well tested. The items I'd gate merge on are #1 (lint failure), #2 (no-op config fields in the foundation), and #3/#4 (proto-sync integrity + docs overstating current capability); #5 (conn.go tests) is strongly recommended given this PR's role as the base. The rest are cleanups.


// ExecInterface is defined in exec.go

// FileInterface is defined in file.go

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[codex:gpt-5.5] gofmt -l $(find sdk/go -name "*.go") reports multiple handwritten files, including this one, types/sandbox.go, types/profile.go, types/refresh.go, types/network_policy.go, types/errors.go, stub_clients.go, internal/converter/network_policy.go, internal/converter/errors.go, and sandbox_client_test.go. Please run gofmt/goimports before merging so the committed SDK matches standard Go formatting.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Will run gofmt -w and goimports -w across all handwritten Go files before the next push.

- Make scheme parsing drive transport selection: http:// uses plaintext
  gRPC, https:// or no scheme uses TLS. Add regression tests.
- Add Resources and DriverConfig fields to SandboxTemplate and update
  both converter directions (SandboxFromProto/SandboxSpecToProto).
- Regenerate proto bindings from current canonical proto sources to
  eliminate drift (SigV4/MCP fields, params matchers, reserved fields).
- Run gofmt/goimports on all handwritten Go files.

Signed-off-by: Roland Huß <rhuss@redhat.com>

@russellb russellb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[codex:gpt-5.5] Finding 1: The Go SDK still drops active sandbox policy fields from the handwritten types/converters. The synced proto includes credential_signing, signing_service, signing_region, json_rpc_max_body_bytes, mcp, and params on L7 allow/deny rules, but PolicyNetworkEndpoint, L7Allow, L7DenyRule, and the converters omit them. Since Create sends SandboxSpecToProto, Go clients cannot express current SigV4/MCP/JSON-RPC policy controls, and server-returned policies lose these fields on round-trip. Please add SDK fields and bidirectional converter coverage for every current proto policy field. Refs: sdk/go/openshell/v1/types/network_policy.go:19, sdk/go/openshell/v1/internal/converter/network_policy.go:65, proto/sandbox.proto:131, proto/sandbox.proto:211.

[codex:gpt-5.5] Finding 2: mapToStruct ignores structpb.NewStruct errors for SandboxTemplate.Resources and DriverConfig. Invalid UTF-8 keys or unsupported map[string]any values make NewStruct return nil, err, but the SDK silently sends nil, so user-provided template config can disappear without an error. Please make sandbox spec conversion fallible, validate before CreateSandbox, or expose a safer typed representation, and add tests for invalid values. Refs: sdk/go/openshell/v1/internal/converter/copy.go:60, sdk/go/openshell/v1/internal/converter/sandbox.go:170, sdk/go/openshell/v1/sandbox_client.go:28.

- Remove dead boolCount function that would fail golangci-lint (#1)
- Emit EventAdded for the first watch event instead of EventModified,
  matching k8s watch semantics (#7)
- Add mutex locking to all mock server methods that access the shared
  sandboxes map, fixing latent race conditions (#12)
- Skip HealthCheck integration test that calls an unimplemented stub (#13)
- Scope doc.go examples: mark sections for sub-clients not yet available
  in this PR with "available in a future release" (#4)
- Document Config.Timeout/RetryPolicy/Logger and WatchOptions fields
  as reserved for future use (#2, #6)

Signed-off-by: Roland Huß <rhuss@redhat.com>
@rhuss

rhuss commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

My agent's response to #2271 (comment). Most of the things are because of this artificial split to get the PRs down to something more consumable (which was also important as I hight some size limits for code agent's review when I dropped it). But thank you very much for jumping on it, I've addressed the comments (and delayed some until we get the full combo in)


Thanks for the review. Here is my assessment, classifying each finding by root cause:

Already addressed (in a prior fix commit 3b81351):

  • Proto drift (finding 3): Synced all protos from canonical proto/ sources and regenerated Go bindings. The options.proto concern is moot: the canonical protos do not import it, so proto:sync will not break proto:gen. The volume_claim_templates field is now correctly reserved 9.
  • conn.go untested (finding 5): Added conn_test.go with tests covering all scheme/TLS paths (http:// plaintext, https:// TLS, no-scheme TLS, Insecure config).

Fixed now (commit dab9329):

  • Dead boolCount (finding 1): Deleted.
  • EventAdded never emitted (finding 7): First watch event now emits EventAdded, subsequent events emit EventModified. Tests updated.
  • Mock server races (finding 12): Added mu.Lock/Unlock to all mock server methods accessing s.sandboxes.
  • Broken integration test (finding 13): HealthCheck test now t.Skips like the others.
  • doc.go scope (finding 4): All sub-client sections not available in PR A are marked "(available in a future release)".
  • No-op fields (findings 2, 6): Config.Timeout, RetryPolicy, Logger and WatchOptions.TimeoutSeconds, LabelSelector are documented as "reserved for future use".

Deferred to later PRs (expected from the A-F split):

  • Config wiring (finding 2): The actual Timeout/RetryPolicy/Logger implementation requires the full client paths that land in PRs B-F. Documented as reserved for now.
  • LabelSelector (finding 6): Meaningless for single-object watch (reviewer agrees). Will revisit when multi-object watch is added.

Accepted as low-priority (not blocking):

  • Watch error drop (finding 8): Valid edge case with the 64-slot buffer. Will address if it surfaces in practice.
  • Watch blocks on first Recv (finding 9): Documentation issue. The blocking behavior is inherent to the name-to-ID resolution + initial state delivery pattern.
  • FromGRPCError detail loss (finding 10): Details field is dead. Will clean up alongside error handling improvements.
  • int to uint32 truncation (finding 11): Bounds would only matter at >4B. Low risk but easy to add.
  • Refresh lock scope (finding 14): Correct single-flight behavior as noted.

@rhuss rhuss marked this pull request as ready for review July 15, 2026 18:24
@russellb

Copy link
Copy Markdown
Contributor

My agent's response to #2271 (comment). Most of the things are because of this artificial split to get the PRs down to something more consumable (which was also important as I hight some size limits for code agent's review when I dropped it). But thank you very much for jumping on it, I've addressed the comments (and delayed some until we get the full combo in)

Sounds good. I figured some of it would be off, but that your agent would sort it out. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(sdk): proposal for Go SDK following client-go conventions

2 participants