Skip to content

Accept FIOASYNC ioctl and F_SETOWN/F_GETOWN fcntl#81

Merged
jserv merged 1 commit into
sysprog21:mainfrom
Max042004:ioctl-fioasync
Jun 30, 2026
Merged

Accept FIOASYNC ioctl and F_SETOWN/F_GETOWN fcntl#81
jserv merged 1 commit into
sysprog21:mainfrom
Max042004:ioctl-fioasync

Conversation

@Max042004

@Max042004 Max042004 commented Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Problem

Running nginx (aarch64 Linux, e.g. nginx:alpine) under elfuse in its default master/worker mode never serves requests: the listen socket binds, the host kernel completes TCP handshakes, but every request hangs and times out. With master_process off; (single process, no fork) nginx serves correctly, which masks the real issue.

Root cause

nginx's ngx_spawn_process arms the master→worker channel socket right before fork() with:

ioctl(channel[0], FIOASYNC, &on);     /* enable SIGIO-driven I/O   */
fcntl(channel[0], F_SETOWN, ngx_pid); /* who receives that SIGIO   */

and treats failure of either as fatal — it logs an alert, ngx_close_channel(), and return NGX_INVALID_PID without ever calling fork(). elfuse answered FIOASYNC with ENOTTY and F_SETOWN with EINVAL, so the master silently ended up with zero workers. A verbose trace shows no clone(220), no accept4(242), no epoll_pwait(22) after the FIOASYNC failure — the master just spins in rt_sigsuspend, and the accepted connections are never serviced.

Fix

elfuse does not forward host SIGIO into the guest, and nginx workers receive both client I/O and channel commands via epoll rather than SIGIO, so both calls are safe to accept as no-ops:

  • sys_ioctl: FIOASYNC reads the int arg (for EFAULT parity) and returns success without arming host O_ASYNC.
  • sys_fcntl: F_SETOWN / F_SETOWN_EX accept and track no owner; F_GETOWN / F_GETOWN_EX report "no owner". glibc implements fcntl(F_GETOWN) on top of F_GETOWN_EX, so the EX form writes a struct f_owner_ex{type=F_OWNER_PID, pid=0} to stay coherent.

With this, nginx -g "daemon off;" forks its worker and serves HTTP on the stock config: GET/HEAD/404, keep-alive, and 100-way concurrency all return 200 with byte-identical bodies.

Testing

  • New tests/test-ioctl-fioasync.c (registered in tests/manifest.txt) replays nginx's pre-fork channel arming on a socketpair and a TCP socket, asserting FIOASYNC, F_SETOWN, and F_GETOWN all succeed.
  • make check: the full guest driver suite passes (the only failure observed was the pre-existing, unrelated flaky test-fork — "unexpected exit reason 0x3" — which is racy on main and untouched by this change, which only adds switch cases in sys_ioctl/sys_fcntl).

Summary by cubic

Accepts FIOASYNC ioctl and F_SETOWN/F_GETOWN (and EX variants) fcntl as safe no-ops so nginx master/worker mode forks workers and serves requests under elfuse. Fixes the hang where requests timed out because no workers were spawned.

  • Bug Fixes
    • sys_ioctl: handle LINUX_FIOASYNC as a no-op; read int arg and return EFAULT on bad pointer; add constant in abi.h.
    • sys_fcntl: accept F_SETOWN/F_SETOWN_EX; F_GETOWN returns 0; F_GETOWN_EX writes {type=F_OWNER_PID, pid=0}; read/write user memory for correct EFAULT.
    • Tests: add tests/test-ioctl-fioasync.c and manifest entry; cover socketpair and TCP sockets; verify F_SETOWN_EX with a bad pointer returns EFAULT.

Written for commit a043118. Summary will update on new commits.

Review in cubic

cubic-dev-ai[bot]

This comment was marked as resolved.

jserv

This comment was marked as duplicate.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase latest main branch and resolve conflicts.

nginx's ngx_spawn_process arms the master->worker channel socket with
ioctl(FIOASYNC) immediately followed by fcntl(F_SETOWN), right before
fork(), and treats failure of either as fatal: it logs an alert, closes
the channel, and returns NGX_INVALID_PID without ever forking the worker.

elfuse answered FIOASYNC with ENOTTY and F_SETOWN with EINVAL, so nginx
in its default master/worker mode silently ended up with zero workers:
the listen socket still accepted connections at the host kernel, but no
guest worker ever accept()ed them, so every request hung. (With
"master_process off" nginx served fine, which masked the issue.)

elfuse does not forward host SIGIO into the guest, and nginx workers
receive both client I/O and channel commands via epoll rather than
SIGIO, so both calls are safe to accept as no-ops:

  - sys_ioctl: FIOASYNC reads the int arg (for EFAULT parity) and
    returns success without arming host O_ASYNC.
  - sys_fcntl: F_SETOWN / F_SETOWN_EX accept and track no owner;
    F_GETOWN / F_GETOWN_EX report "no owner". glibc implements
    fcntl(F_GETOWN) on top of F_GETOWN_EX, so the EX form writes a
    struct f_owner_ex{type=F_OWNER_PID, pid=0} to stay coherent.

With this, "nginx -g 'daemon off;'" forks its worker and serves HTTP
(GET/HEAD/404, keep-alive, concurrency) on the default config.

Add tests/test-ioctl-fioasync.c, which replays nginx's pre-fork channel
arming on a socketpair and a TCP socket.
@jserv jserv merged commit 9d302bd into sysprog21:main Jun 30, 2026
4 checks passed
@jserv

jserv commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Thank @Max042004 for contributing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants