[PATCH] fcntl returns wrong error code

[Christopher Yeoh]

When fcntl(fd, F_DUPFD, b) is called where 'b' is greater than the
maximum allowable value EINVAL should be returned. From POSIX:

"[EINVAL] The cmd argument is invalid, or the cmd argument is F_DUPFD and
arg is negative or greater than or equal to {OPEN_MAX}, or ..."

Currently we instead return EMFILE. The following patch (against
2.4.19pre-4) fixes this behaviour:

--- linux-2.4.18/fs/fcntl.c~	Mon Sep 24 05:13:11 2001
+++ linux-2.4.18/fs/fcntl.c	Thu Mar 21 16:50:06 2002
@@ -120,8 +120,13 @@
 	int ret;
 
 	ret = locate_fd(files, file, start);
-	if (ret < 0) 
+	if (ret < 0) {
+		/* We should return EINVAL instead of EMFILE if the
+		   request for the fd starts beyond the valid range */
+		if (ret==-EMFILE && start>=current->rlim[RLIMIT_NOFILE].rlim_cur)
+			ret = -EINVAL;
 		goto out_putf;
+	}
 	allocate_fd(files, file, ret);
 	return ret;

Chris.
-- 
[e-mail]
IBM OzLabs Linux Development Group
Canberra, Australia