Skip to content

Commit 7b32c1a

Browse files
authored
code review followups: mergeEnv correctness + lifecycle and packaging fixes (#3)
* fix(agent): mergeEnv must dedup so caller overrides reach the child The previous append(host, caller...) layout was correct in intent but wrong under libc semantics: execve hands envp through verbatim, and getenv returns the FIRST matching entry. With host appended before caller, any key that existed in the host env (PATH, HOME, USER, ...) would silently shadow the caller's override. Switch to a map-based merge so collisions resolve to a single entry, and add a regression test that pre-populates COCOON_AGENT_OVERRIDE_VAR in the host env then asserts the caller value reaches the child. * fix(cmd): silence cobra error+usage dump so exit-code failures stay quiet run() already handles real errors (logger.Error) and translates exitCodeError into an exit status. With cobra's defaults the latter path also prints "Error: exit code N\nUsage:..." to stderr, which is noise — the caller already knows the exit code and didn't ask for a usage banner. Set SilenceErrors and SilenceUsage on the root command so only run()'s intentional outputs reach the user. * fix(client): prefer ctx.Err over EOF / "closed before exit" on cancel The runCancel goroutine closes conn on ctx-cancel, which surfaces in the readLoop's Decode as io.EOF and falls through to "agent: connection closed before exit frame" — masking the real cause from the caller. Gate both the in-loop EOF branch and the post-loop !sawExit branch on ctx.Err first so caller cancel / parent timeout propagates as itself. * fix(agent): reap Serve's ctx watcher on permanent Accept error The previous watcher goroutine only exited via <-ctx.Done(), so a permanent Accept failure (e.g. EMFILE, syscall-level) returned from Serve while the goroutine kept sitting on ctx until the caller cancelled — potentially never. Rewrite the watcher as a select on ctx.Done() vs a defer-closed done channel so every Serve return path reaps it. Also drain connWG on the permanent-error exit so in-flight sessions finish before the function returns. Add TestServerWatcherExitsOnPermanentAcceptError, which uses a synthetic listener whose Accept returns a non-net.ErrClosed error, holds the parent ctx alive, and asserts the goroutine count returns to baseline after Serve exits. Without the fix the test reports a leak. * fix(cmd): log non-host vsock peer rejections + add isHostPeer test A rejected peer used to disappear silently in the Accept loop, leaving an operator with no signal that a misconfiguration or a guest-local probe was hitting the listener. Both Linux and Windows listeners now log a Warn on rejection with the peer CID/port. Threading the logger required threading a context too — the listener owns the Accept loop, which has no per-call ctx, so we capture the serve ctx at listenVsock(ctx, port) construction time and store it on the struct. This is the same ctx that the agent.Server uses, so cancellation propagates uniformly. Add cmd/transport_linux_test.go locking in the isHostPeer contract: vsock.Host accepted, vsock.Local rejected, non-vsock RemoteAddr rejected. * packaging(systemd): drop wrong modprobe target + cap restart churn vhost_vsock is the HOST-side module (driver for /dev/vhost-vsock that the hypervisor opens), not the guest-side transport — loading it inside the guest either fails outright or is a no-op. The guest needs virtio_transport_common + vmw_vsock_virtio_transport, both of which auto-load on virtio-vsock device probe. listenVsockWithRetry already covers the bind window if the device shows up late, so the ExecStartPre is purely misleading; drop it. Also add StartLimitBurst=5 + StartLimitIntervalSec=60s under [Unit] (their modern home since systemd v229) so a wedged hypervisor or viosock state can't peg journald with restart churn — five attempts in a minute is a firm enough boundary to surface real failures. * test(agent): drop dead nolint:mnd, add stdin-close + framedWriter coverage mnd is not in our enabled linter list (see .golangci.yml), so the directive on context.WithTimeout(..., 10*time.Second) is dead weight — drop it. Add two regression tests that were missing from prior coverage: - TestServerMsgStdinCloseTerminatesChildStdin: spawn `wc -c`, push a known payload, send MsgStdinClose, assert exit=0 and the byte count came back intact. Locks in the mid-stream stdin-close path that wasn't exercised by the existing cat round-trip. - TestFramedWriterAfterTerminal: directly drive framedWriter.Write after the encoder has emitted a terminal frame. The errTerminalFrameSent branch must NOT poison lastErr or fire cancel — otherwise the post-Wait err()-join would mask the legitimate exit path. * test(agent): trim errorAcceptListener to fields the watcher test exercises The closeMu/closed/addr fields were stub state that no caller ever read — remove them so the fake stays focused on its single responsibility (returning a permanent Accept error). Drops three fields, a Close mutex pair, and an Addr override branch. * fix(agent): tear down conns on permanent Accept error before joining connWG.Wait on the permanent-error path could pin Serve indefinitely if a handleConn was wedged in framedWriter.Write against a slow peer — the ctx-cancel and net.ErrClosed paths get the listener+conn teardown for free (watcher goroutine or external Close), but the permanent-error branch had no such trigger. * chore: trim verbose comments across the review branch Most of the comments added in this branch restated the code or carried multi-paragraph rationale that belongs in commit messages, not source. Collapse each to a single WHY line where one is warranted, drop the rest. -43 net comment lines, no behavior change. * test(agent): drop t.Parallel on goroutine-leak test NumGoroutine baseline is perturbed by sibling parallel tests in the package (dialTestServer spawns Serve goroutines), so the <= before assertion can stay false even with zero leak. * test(cmd): make staticAddrConn obey io.Reader/Writer contracts (0, nil) from Read violates io.Reader and can cause tight loops if the stub gets reused beyond RemoteAddr(). Return io.EOF + sane Write sink. * fix(agent): mergeEnv host dedup keeps first occurrence Comment claimed libc-getenv semantics (first match wins) but the impl overwrote merged[k] on every host duplicate, ending up with last-wins. Skip if already seen so behavior matches the documented contract. * test(agent): make watcher leak regression use specific goroutine signal Counting all goroutines is too broad — sibling tests can perturb the baseline. Match on the Serve watcher's stack frame instead so the test only fires on the goroutine it actually cares about. const block lives at the top of the file per Cocoon style. * test(agent): bound watcher stack dump helper runtime.Stack with an unbounded grow loop could hand back a huge buffer on a wedged process. Cap doubling at 16 MiB so the helper still produces a useful diagnostic without unbounded allocation. * test(agent): simplify watcher leak polling helper Collapse the dump-and-count helper now that callers only need the count; the dump path is reserved for the failure diagnostic. * build: pin golangci-lint installer to the same release tag master's install.sh recently regressed on resolving v2.9.0, breaking CI lint. Pinning the installer URL to the version tag matches upstream's recommended pattern and avoids future master breakage.
1 parent 4e91309 commit 7b32c1a

13 files changed

Lines changed: 301 additions & 25 deletions

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ GOOSES ?= linux darwin windows
4444
.PHONY: golangci-lint
4545
golangci-lint: $(GOLANGCILINT)
4646
$(GOLANGCILINT):
47-
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(GOLANGCILINT_ROOT) $(GOLANGCILINT_VERSION)
47+
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/$(GOLANGCILINT_VERSION)/install.sh | sh -s -- -b $(GOLANGCILINT_ROOT) $(GOLANGCILINT_VERSION)
4848

4949
.PHONY: gofumpt
5050
gofumpt: $(GOFMT)

agent/agent.go

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,16 @@ func (s *Server) Serve(ctx context.Context) error {
4141
logger := log.WithFunc("agent.Server.Serve")
4242
logger.Infof(ctx, "agent listening on %s", s.listener.Addr())
4343

44+
// done reaps the watcher on every return path, not just ctx-cancel.
45+
done := make(chan struct{})
46+
defer close(done)
4447
go func() {
45-
<-ctx.Done()
46-
_ = s.listener.Close()
47-
s.closeAllConns()
48+
select {
49+
case <-ctx.Done():
50+
_ = s.listener.Close()
51+
s.closeAllConns()
52+
case <-done:
53+
}
4854
}()
4955

5056
var connWG sync.WaitGroup
@@ -56,6 +62,10 @@ func (s *Server) Serve(ctx context.Context) error {
5662
return nil
5763
}
5864
logger.Error(ctx, err, "accept")
65+
// Unwedge handlers stuck on slow peers before joining.
66+
_ = s.listener.Close()
67+
s.closeAllConns()
68+
connWG.Wait()
5969
return fmt.Errorf("accept: %w", err)
6070
}
6171
connWG.Go(func() { s.handleConn(ctx, conn) })

agent/agent_test.go

Lines changed: 126 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import (
66
"errors"
77
"io"
88
"net"
9+
"runtime"
910
"strings"
1011
"sync"
1112
"testing"
@@ -15,6 +16,11 @@ import (
1516
"github.com/cocoonstack/cocoon-agent/client"
1617
)
1718

19+
const (
20+
initialGoroutineDumpSize = 1 << 16
21+
maxGoroutineDumpSize = 1 << 24
22+
)
23+
1824
// dialTestServer runs the agent over loopback TCP and dials a client conn.
1925
// Cleanup (cancel ctx, close server, wait for Serve to return, close conn)
2026
// is registered via t.Cleanup so callers don't repeat it per test.
@@ -25,7 +31,7 @@ func dialTestServer(t *testing.T) (context.Context, net.Conn) {
2531
t.Fatalf("listen tcp: %v", err)
2632
}
2733
srv := agent.NewServer(tcp)
28-
ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second) //nolint:mnd
34+
ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
2935
var wg sync.WaitGroup
3036
wg.Go(func() { _ = srv.Serve(ctx) })
3137
conn, err := net.Dial("tcp", tcp.Addr().String())
@@ -94,6 +100,32 @@ func TestServerStreamsStdin(t *testing.T) {
94100
}
95101
}
96102

103+
// TestServerMsgStdinCloseTerminatesChildStdin: child must see EOF after the
104+
// close frame and exit 0; wc -c also confirms pre-close payload arrived.
105+
func TestServerMsgStdinCloseTerminatesChildStdin(t *testing.T) {
106+
t.Parallel()
107+
ctx, conn := dialTestServer(t)
108+
109+
payload := "abcde"
110+
var stdout bytes.Buffer
111+
exit, err := client.Run(
112+
ctx, conn,
113+
[]string{"sh", "-c", "wc -c"},
114+
nil,
115+
strings.NewReader(payload),
116+
&stdout, io.Discard,
117+
)
118+
if err != nil {
119+
t.Fatalf("client run: %v", err)
120+
}
121+
if exit != 0 {
122+
t.Errorf("exit = %d, want 0", exit)
123+
}
124+
if got := strings.TrimSpace(stdout.String()); got != "5" {
125+
t.Errorf("wc -c stdout = %q, want \"5\"", got)
126+
}
127+
}
128+
97129
func TestServerRejectsNonExecFirstFrame(t *testing.T) {
98130
t.Parallel()
99131
_, conn := dialTestServer(t)
@@ -224,3 +256,96 @@ func TestServerMergesEnvWithHost(t *testing.T) {
224256
t.Errorf("host PATH not preserved on merge: %q", out)
225257
}
226258
}
259+
260+
// Not parallel: t.Setenv mutates process-global os.Environ.
261+
func TestServerMergesEnvCallerWins(t *testing.T) {
262+
t.Setenv("COCOON_AGENT_OVERRIDE_VAR", "host-value")
263+
ctx, conn := dialTestServer(t)
264+
265+
var stdout bytes.Buffer
266+
exit, err := client.Run(
267+
ctx, conn,
268+
[]string{"sh", "-c", "printf %s \"$COCOON_AGENT_OVERRIDE_VAR\""},
269+
map[string]string{"COCOON_AGENT_OVERRIDE_VAR": "caller-value"},
270+
nil, &stdout, io.Discard,
271+
)
272+
if err != nil {
273+
t.Fatalf("client run: %v", err)
274+
}
275+
if exit != 0 {
276+
t.Errorf("exit = %d, want 0", exit)
277+
}
278+
if got := stdout.String(); got != "caller-value" {
279+
t.Errorf("caller env did not win on collision: got %q, want %q", got, "caller-value")
280+
}
281+
}
282+
283+
// errorAcceptListener returns err on every Accept — drives Serve's
284+
// permanent-error return path.
285+
type errorAcceptListener struct {
286+
err error
287+
}
288+
289+
func (l *errorAcceptListener) Accept() (net.Conn, error) {
290+
return nil, l.err
291+
}
292+
293+
func (l *errorAcceptListener) Close() error { return nil }
294+
func (l *errorAcceptListener) Addr() net.Addr {
295+
return &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0}
296+
}
297+
298+
func goroutineDump() string {
299+
size := initialGoroutineDumpSize
300+
for {
301+
buf := make([]byte, size)
302+
n := runtime.Stack(buf, true)
303+
if n < len(buf) || size == maxGoroutineDumpSize {
304+
return string(buf[:n])
305+
}
306+
size = min(size*2, maxGoroutineDumpSize)
307+
}
308+
}
309+
310+
func countServeWatcherGoroutines() int {
311+
return strings.Count(goroutineDump(), "(*Server).Serve.func1")
312+
}
313+
314+
// TestServerWatcherExitsOnPermanentAcceptError: on a non-ErrClosed Accept
315+
// failure, Serve must reap the ctx watcher via the done channel rather than
316+
// leak it until the (possibly never-canceled) parent ctx fires.
317+
//
318+
// Not parallel: asserts against the specific Serve watcher goroutine and keeps
319+
// other tests from creating extra matching stacks while it samples.
320+
func TestServerWatcherExitsOnPermanentAcceptError(t *testing.T) {
321+
before := countServeWatcherGoroutines()
322+
323+
srv := agent.NewServer(&errorAcceptListener{err: errors.New("synthetic permanent accept failure")})
324+
// Long-lived ctx so only the done-channel path can release the watcher.
325+
ctx, cancel := context.WithCancel(t.Context())
326+
defer cancel()
327+
328+
errCh := make(chan error, 1)
329+
go func() { errCh <- srv.Serve(ctx) }()
330+
331+
select {
332+
case err := <-errCh:
333+
if err == nil {
334+
t.Fatal("expected permanent accept error to surface")
335+
}
336+
case <-time.After(2 * time.Second):
337+
t.Fatal("Serve did not return after permanent accept error")
338+
}
339+
340+
// Watcher exits async after close(done); poll until the specific watcher
341+
// stack count returns to baseline.
342+
deadline := time.Now().Add(2 * time.Second)
343+
for time.Now().Before(deadline) {
344+
if countServeWatcherGoroutines() <= before {
345+
return
346+
}
347+
runtime.Gosched()
348+
time.Sleep(10 * time.Millisecond)
349+
}
350+
t.Fatalf("Serve watcher goroutine still present:\n%s", goroutineDump())
351+
}

agent/exec.go

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,10 @@ import (
55
"errors"
66
"fmt"
77
"io"
8+
"maps"
89
"os"
910
"os/exec"
11+
"strings"
1012
)
1113

1214
// processController hooks platform-specific child-lifecycle steps into
@@ -144,12 +146,24 @@ func pumpStdin(ctx context.Context, w io.WriteCloser, frames <-chan Message, don
144146
}
145147
}
146148

147-
// mergeEnv layers caller env over os.Environ; caller keys win on collision.
148-
func mergeEnv(env map[string]string) []string {
149-
hostEnv := os.Environ()
150-
out := make([]string, 0, len(hostEnv)+len(env))
151-
out = append(out, hostEnv...)
152-
for k, v := range env {
149+
// mergeEnv layers caller env over os.Environ. Dedup is required: libc getenv
150+
// returns the first match, so duplicate keys would shadow caller overrides.
151+
func mergeEnv(callerEnv map[string]string) []string {
152+
host := os.Environ()
153+
merged := make(map[string]string, len(host)+len(callerEnv))
154+
for _, kv := range host {
155+
k, v, ok := strings.Cut(kv, "=")
156+
if !ok {
157+
continue
158+
}
159+
if _, dup := merged[k]; dup {
160+
continue // first occurrence wins, matching libc getenv
161+
}
162+
merged[k] = v
163+
}
164+
maps.Copy(merged, callerEnv)
165+
out := make([]string, 0, len(merged))
166+
for k, v := range merged {
153167
out = append(out, k+"="+v)
154168
}
155169
return out

agent/protocol_test.go

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,3 +147,33 @@ func TestDecodeHandlesLargeFrame(t *testing.T) {
147147
t.Errorf("payload mismatch: got %d bytes, want %d", len(got.Data), len(payload))
148148
}
149149
}
150+
151+
// Post-terminal Write must return errTerminalFrameSent without poisoning
152+
// lastErr or tripping cancel, else err()-join masks the real exit.
153+
func TestFramedWriterAfterTerminal(t *testing.T) {
154+
t.Parallel()
155+
156+
var buf bytes.Buffer
157+
enc := NewEncoder(&buf)
158+
if err := enc.Encode(Message{Type: MsgError, Message: "kaboom"}); err != nil {
159+
t.Fatalf("encode terminal: %v", err)
160+
}
161+
162+
var cancelCalled bool
163+
cancel := func() { cancelCalled = true }
164+
w := newFramedWriter(MsgStdout, enc, cancel)
165+
166+
n, err := w.Write([]byte("late stdout chunk"))
167+
if !errors.Is(err, errTerminalFrameSent) {
168+
t.Fatalf("post-terminal Write err = %v, want %v", err, errTerminalFrameSent)
169+
}
170+
if n != 0 {
171+
t.Errorf("post-terminal Write n = %d, want 0", n)
172+
}
173+
if cancelCalled {
174+
t.Error("post-terminal Write must not call cancel")
175+
}
176+
if got := w.err(); got != nil {
177+
t.Errorf("post-terminal Write must not poison lastErr, got %v", got)
178+
}
179+
}

client/client.go

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -69,12 +69,14 @@ readLoop:
6969
if e := stdinReadErr.Load(); e != nil {
7070
return 0, fmt.Errorf("read stdin: %w", *e)
7171
}
72-
if errors.Is(err, io.EOF) {
73-
break
74-
}
72+
// Prefer ctx.Err over EOF: ctx-cancel closes the conn,
73+
// surfacing as EOF here.
7574
if ctx.Err() != nil {
7675
return 0, ctx.Err()
7776
}
77+
if errors.Is(err, io.EOF) {
78+
break
79+
}
7880
return 0, fmt.Errorf("read frame: %w", err)
7981
}
8082
switch frame.Type {
@@ -105,6 +107,10 @@ readLoop:
105107
return 0, fmt.Errorf("read stdin: %w", *e)
106108
}
107109
if !sawExit {
110+
// Same ctx-cancel-races-MsgExit case as the readLoop EOF path.
111+
if ctx.Err() != nil {
112+
return 0, ctx.Err()
113+
}
108114
return 0, errors.New("agent: connection closed before exit frame")
109115
}
110116
return exitCode, nil

cmd/root.go

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@ func NewRootCmd() *cobra.Command {
2121
Use: "cocoon-agent",
2222
Short: "vsock-based command exec agent for Cocoon-managed VMs",
2323
Version: fmt.Sprintf("%s (rev=%s built=%s)", version.VERSION, version.REVISION, version.BUILTAT),
24+
// run() handles its own logging and exit codes; suppress cobra's
25+
// Error/Usage dump so child-exit failures stay quiet.
26+
SilenceErrors: true,
27+
SilenceUsage: true,
2428
}
2529
rootCmd.AddCommand(newServeCmd())
2630
rootCmd.AddCommand(newClientCmd())

cmd/serve.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ func listenVsockWithRetry(ctx context.Context, port uint32) (net.Listener, error
4444
logger := log.WithFunc("cmd.serve.listenRetry")
4545
deadline := time.Now().Add(listenRetryTimeout)
4646
for attempt := 1; ; attempt++ {
47-
lsn, err := listenVsock(port)
47+
lsn, err := listenVsock(ctx, port)
4848
if err == nil {
4949
if attempt > 1 {
5050
logger.Infof(ctx, "vsock listen succeeded on attempt %d", attempt)

cmd/transport_linux.go

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,21 +3,27 @@
33
package cmd
44

55
import (
6+
"context"
67
"fmt"
78
"io"
89
"net"
910

1011
"github.com/mdlayher/vsock"
12+
"github.com/projecteru2/core/log"
1113
)
1214

1315
var _ net.Listener = (*hostOnlyListener)(nil)
1416

15-
func listenVsock(port uint32) (net.Listener, error) {
17+
func listenVsock(ctx context.Context, port uint32) (net.Listener, error) {
1618
l, err := vsock.Listen(port, nil)
1719
if err != nil {
1820
return nil, fmt.Errorf("vsock listen: %w", err)
1921
}
20-
return &hostOnlyListener{Listener: l}, nil
22+
return &hostOnlyListener{
23+
Listener: l,
24+
ctx: ctx,
25+
logger: log.WithFunc("cmd.hostOnlyListener.Accept"),
26+
}, nil
2127
}
2228

2329
func dialVsock(cid, port uint32) (io.ReadWriteCloser, error) {
@@ -34,6 +40,9 @@ func dialVsock(cid, port uint32) (io.ReadWriteCloser, error) {
3440
// and trigger root-level command execution.
3541
type hostOnlyListener struct {
3642
net.Listener
43+
// ctx is the serve ctx, stashed for Accept-loop diagnostic logging.
44+
ctx context.Context
45+
logger *log.Fields
3746
}
3847

3948
func (l *hostOnlyListener) Accept() (net.Conn, error) {
@@ -45,6 +54,7 @@ func (l *hostOnlyListener) Accept() (net.Conn, error) {
4554
if isHostPeer(conn) {
4655
return conn, nil
4756
}
57+
l.logger.Warnf(l.ctx, "rejecting non-host vsock peer %s", conn.RemoteAddr())
4858
_ = conn.Close()
4959
}
5060
}

0 commit comments

Comments
 (0)