CS 332: Operating Systems
Adding a system call to Linux
UNDER CONSTRUCTION. Some editing yet to be done.
These instructions are valid for CS307 students during
Carleton's spring term, 2008. We were using Red Hat Enterprise
Linux installed on VMware Workstation (running in turn on Windows).
VMware
- Go to CMC 304 and login to Windows on one of the four machines in the left corner (on the
east wall beyond the whiteboard--not the ones over by the windows). The four machines are
named reddy, karp, yellow, and milner.
- Launch VMware. You should have a VMware icon on your desktop. If not, you can find
it in the Start menu (possibly under System Tools).
- Select Open from the File menu, and select
C:\My Virtual Machines\Red Hat Enterprise Linux 5\Red Hat Enterprise Linux 5.
- Click on "Start this virtual machine" in the middle of the screen (by a triangular "play" button).
- When you are given a choice of which copy of Redhat to boot from, choose
"Red Hat Enterprise Linux Server (2.6.18-8.el5)".
That's the one that has VMTools installed on it.
You definitely want VMTools, because that's what enables you to drag files from Windows
to Linux and back.
- Once Linux is booted up, login using "root" and "!jondich332".
- Right-click on the Linux desktop, open a terminal, and off you go.
Before you continue...
The big trick with the setup we are using is that you will be modifying files in the VMware
Linux installation on some machine, but other people will need to use that same machine.
Thus, it's important that you (1) revert the Linux sources to the form you found them in,
and (2) make copies of your own modified files so you can move them to other machines and
avoid the pain of having somebody else from the class delete your work. Here are some
general suggestions for you:
- Whenever you are going to modify a kernel source file, make
sure there is a copy of the original version of the file. For example, if you're going
to modify Makefile, do "cp Makefile Makefile.orig" first.
- Mike has installed VMtools on all four VMware machines inside the
kernel labeled "Red Hat Enterprise Linux Server (2.6.18-8.el5)".
Thus, when you are booted into that kernel, you can drag files between
Windows and Linux. Note that when you build a new kernel and boot into it,
the VMTools will not be available unless you install them yourself. I'll try
to obtain instructions for doing so.
- When you're finishing up a work session, you should make backup copies of all
the files you changed. To do this using a VMTools-enabled kernel, navigate to each
file and drag it from VMware to the Windows desktop. Then either e-mail the files to
yourself, or save them in your Collab space, or put them on a flash drive, etc.
Save them somewhere you can readily retrieve them later, in any case.
- Also, please copy the original versions of each modified file back into
its original position (e.g. "mv Makefile.orig Makefile"), so the next person
to work on that machine will not be messed up by your changes.
Linux source
The root directory of the Linux source code is at /usr/src/redhat/SOURCES/linux-2.6.18.
Relative paths will refer to that base from here on.
Modify the system call table
- Edit arch/i386/kernel/syscall_table.S. Add ".long sys_yourname" to the
end of the table of sys-call function pointers.
- Edit include/asm/unistd.h. Add "#define __NR_yourname [next available number]" to
the bottom of the list of __NR constants. Then, increase NR_syscalls by 1.
- Backup /usr/include/asm/unistd.h to /usr/include/asm/unistd.h.orig,
and then copy include/asm/unistd.h to /usr/include/asm.
Edit the Makefile structure
- Create a yourname/ subdirectory of the Linux source root directory.
- Edit Makefile (at the source root). The "core-y += kernel/..." line
should become "core-y += yourname/ kernel/...".
- Copy mm/Makefile to yourname/.
- Edit yourname/Makefile to have only the line:
obj-y := yourname.o
plus an appropriate comment.
Add the source for your system call
- Create yourname/yourname.c with code for your
system call. Note that the name of this function
needs to be the same as the name ("sys_yourname", or whatever)
that you used in the ".long" table in entry.S.
Here's an example.
Compile and install the new kernel
- Make sure you're in the Linux source root directory, and then execute "make".
This should take 5-10 minutes. Since you put yourname/ at
the beginning of the "core-y" line in the root Makefile, you should
see evidence fairly quickly that your new code is compiling (or not).
Everything okay? Does it say "bzImage is ready"? Then:
- make modules
- make modules_install
- make install
in that order. You may or may not be able to skip the "make modules" and "make modules_install"
steps. To save time, you might want to try and see if "make install" goes through successfully
without doing those other steps first. In any case, once "make install" completes, it will
have installed your kernel as one of the boot-up options.
Reboot
- Restart Linux, and when the OS choice screen appears, select the one that
ends with yourname.
Test your system call
- Login as root again, and create a working directory ~/yourname.
- In your working directory create a test file mytest.c or some such
thing. Here's an example.
- Compile and run your test program. Note that the $PATH environment
variable probably doesn't include ., so you'll need to execute
"./mytest" or some such thing.
- If all goes well, you will see
printf output, but not printk output. To see the printk output,
do "tail /var/log/messages".