Your Daily Source for Apache News and Information |
Breaking News | Preferences | Contribute | Triggers | Link Us | Search | About |
\n'); } if ( plugin ) { document.write(' '); } else if (!(navigator.appName && navigator.appName.indexOf("Netscape")>=0 && navigator.appVersion.indexOf("2.")>=0)){ document.write(''); } //-->
|
By Stas Bekman Forking and Executing Subprocesses from mod_perlIt's desirable to avoid forking under mod_perl. Since when you do, you are forking the entire Apache server, lock, stock and barrel. Not only is your Perl code and Perl interpreter being duplicated, but so is mod_ssl, mod_rewrite, mod_log, mod_proxy, mod_speling (it's not a typo!) or whatever modules you have used in your server, all the core routines, etc. Modern Operating Systems come with a very light version of fork which adds a little overhead when called, since it was optimized to do the absolute minimum of memory pages duplications. The copy-on-write technique is the one that allows to do so. The gist of this technique is as follows: the parent process memory pages aren't immediately copied to the child's space on fork(), but this is done only when the child or the parent modifies the data in some memory pages. Before the pages get modified they get marked as dirty and the child has no choice but to copy the pages that are to be modified since they cannot be shared any more. If you need to call a Perl program from your mod_perl code, it's better to try to covert the program into a module and call it a function without spawning a special process to do that. Of course if you cannot do that or the program is not written in Perl, you have to call via Also by trying to spawn a sub-process, you might be trying to do the ``wrong thing''. If what you really want is to send information to the browser and then do some post-processing, look into the my $r = shift; $r->register_cleanup(\&do_cleanup); sub do_cleanup{ #some clean-up code here } But when a long term process needs to be spawned, there is not much choice, but to use fork(). We cannot just run this long term process within Apache process, since it'll first keep the Apache process busy, instead of letting it do the job it was designed for. And second, if Apache will be stopped the long term process might be terminated as well, unless coded properly to detach from Apache processes group. In the following sections I'm going to discuss how to properly spawn new processes under mod_perl. Forking a New ProcessThis is a typical way to call defined (my $kid = fork) or die "Cannot fork: $!\n"; if ($kid) { # Parent runs this block } else { # Child runs this block # some code comes here CORE::exit(0); } # possibly more code here usually run by the parent When using fork(), you should check its return value, since if it returns When the process is successfully forked -- the parent receives the PID of the newly spawned child as a returned value of the It's important not to forget to explicitly call The parent process usually completes its execution path and enters the pool of free servers to wait for a new assignment. If the execution path is to be aborted earlier for some reason one should use Apache::exit() or die(). In the case of The child shares with its parent its memory pages until it has to modify some of them, which triggers a copy-on-write process which copies these pages to the child's domain before the child is allowed to modify them. But this all happens afterwards. At the moment the Freeing the Parent ProcessIn the child code you must also close all the pipes to the connection socket that were opened by the parent process (i.e. Under mod_perl, the spawned process also inherits the file descriptor that's tied to the socket through which all the communications between the server and the client happen. Therefore we need to free this stream in the forked process. If we don't do that, the server cannot be restarted while the spawned process is still running. If an attempt is made to restart the server you will get the following error: [Mon Dec 11 19:04:13 2000] [crit] (98)Address already in use: make_sock: could not bind to address 127.0.0.1 port 8000
So the simplest way is to freeing the parent process is to close all three STD* streams if we don't need them and untie the Apache socket. In addition you may want to change the process' current directory to / so the forked process won't keep the mounted partition busy, if this is to be unmounted at a later time. To summarize all this issues, here is an example of the fork that takes care of freeing the parent process. use Apache::SubProcess; defined (my $kid = fork) or die "Cannot fork: $!\n"; if ($kid) { # Parent runs this block } else { # Child runs this block $r->cleanup_for_exec(); # untie the socket chdir '/' or die "Can't chdir to /: $!"; close STDIN; close STDOUT; close STDERR; # some code comes here CORE::exit(0); } # possibly more code here usually run by the parent Of course between the freeing the parent code and child process termination the real code is to be placed. Detaching the Forked ProcessNow what happens if the forked process is running and we decided that we need to restart the web-server? This forked process will be aborted, since when parent process will die during the restart it'll kill its child processes as well. In order to avoid this we need to detach the process from its parent session, by opening a new session with help of use POSIX 'setsid'; defined (my $kid = fork) or die "Cannot fork: $!\n"; if ($kid) { # Parent runs this block } else { # Child runs this block setsid or die "Can't start a new session: $!"; ... } Now the spawned child process has a life of its own, and it doesn't depend on the parent anymore. Avoiding Zombie ProcessesNow let's talk about zombie processes. Normally, every process has its parent. Many processes are children of the A zombie is a process that doesn't have a parent. When the child quits, it reports the termination to its parent. If no parent wait()s to collect the exit status of the child, it gets ``confused'' and becomes a ghost process, that can be seen as a process, but not killed. It will be killed only when you stop the parent process that spawned it! Generally the So the proper way to do a fork is: my $r = shift; $r->send_http_header('text/plain'); defined (my $kid = fork) or die "Cannot fork: $!"; if ($kid) { waitpid($kid,0); print "Parent has finished\n"; } else { # do something CORE::exit(0); } In most cases the only reason you would want to fork is when you need to spawn a process that will take a long time to complete. So if the Apache process that spawns this new child process has to wait for it to finish, you have gained nothing. You can neither wait for its completion (because you don't have the time to), nor continue because you will get yet another zombie process. This is called a blocking call, since the process is blocked to do anything else before this call gets completed. The simplest solution is to ignore your dead children. Just add this line before the $SIG{CHLD} = 'IGNORE'; When you set the Note that you cannot localize this setting with So now the code would look like this: my $r = shift; $r->send_http_header('text/plain'); $SIG{CHLD} = 'IGNORE'; defined (my $kid = fork) or die "Cannot fork: $!\n"; if ($kid) { print "Parent has finished\n"; } else { # do something time-consuming CORE::exit(0); } Note that Another, more portable, but slightly more expensive solution is to use a double fork approach. my $r = shift; $r->send_http_header('text/plain'); defined (my $kid = fork) or die "Cannot fork: $!\n"; if ($kid) { waitpid($kid,0); } else { defined (my $grandkid = fork) or die "Kid cannot fork: $!\n"; if ($grandkid) { CORE::exit(0); } else { # code here # do something long lasting CORE::exit(0); } } Grandkid becomes a ``child of init'', i.e. the child of the process whose PID is 1. Note that the previous two solutions do allow you to know the exit status of the process, but in my example I didn't care about it. Another solution is to use a different SIGCHLD handler: use POSIX 'WNOHANG'; $SIG{CHLD} = sub { while( waitpid(-1,WNOHANG)>0 ) {} }; Which is useful when you While you test and debug your code that uses one of the above examples, You might want to write some debug information to the error_log file so you know what happens. Read perlipc manpage for more information about signal handlers. A Complete Fork ExampleNow let's put all the bits of code together and show a well written fork code that solves all the problems discussed so far. I will use an <Apache::Registry> script for this purpose: proper_fork1.pl --------------- use strict; use POSIX 'setsid'; use Apache::SubProcess; my $r = shift; $r->send_http_header("text/plain"); $SIG{CHLD} = 'IGNORE'; defined (my $kid = fork) or die "Cannot fork: $!\n"; if ($kid) { print "Parent $$ has finished, kid's PID: $kid\n"; } else { $r->cleanup_for_exec(); # untie the socket chdir '/' or die "Can't chdir to /: $!"; open STDIN, '/dev/null' or die "Can't read /dev/null: $!"; open STDOUT, '>/dev/null' or die "Can't write to /dev/null: $!"; open STDERR, '>/tmp/log' or die "Can't write to /tmp/log: $!"; setsid or die "Can't start a new session: $!"; select STDERR; local $| = 1; warn "started\n"; # do something time-consuming sleep 1, warn "$_\n" for 1..20; warn "completed\n"; CORE::exit(0); # terminate the process } The script starts with the usual declaration of the strict mode, loading the The HTTP header is sent next, with the Content-type of text/plain. It gets ready to ignore the child, to avoid zombies and the fork is called. The program gets its personality split after fork. And the if conditional evaluates to a true value for the parent process and to a false value for the child process, therefore the first block is executed by the parent and the second by the child. The parent process announces his PID and the PID of the spawned process and finishes its block. If there will be any code outside it will be executed by the parent as well. The child process starts its code by disconnecting from the socket, changing its current directory to select STDERR; local $|=1; warn "started\n"; # do something time-consuming sleep 1, warn "$_\n" for 1..20; warn "completed\n"; The localized setting of Finally the child process terminates by calling: CORE::exit(0); which makes sure that it won't get out of the block and run some code that it's not supposed to run. This code example will allow you to verify that indeed the spawned child process has its own life, and its parent is free as well. Simply issue a request that will run this script, watch that the warnings are started to be written into the /tmp/log file and issue a complete server stop and start. If everything is correct, the server will successfully restart and the long term process will still be running. You will know that it's still running, if the warnings are still printed into the /tmp/log file. You may need to raise the number of warnings to do above 20, to make sure that you don't miss the end of the run. If there are only 5 warnings to be printed, you should see the following output in this file: started 1 2 3 4 5 completed Starting a Long Running External ProgramBut what happens if we cannot just run a Perl code from the spawned process and we have a compiled utility, i.e. a program written in C. Or we have a Perl program which cannot be easily converted into a module, and thus called as a function. Of course in this case we have to use system(), exec(), When using any of these methods and when the Taint mode is enabled, we must at least add the following code to untaint the PATH environment variable and delete a few other insecure environment variables. This information can be found in the perlsec manpage. $ENV{'PATH'} = '/bin:/usr/bin'; delete @ENV{'IFS', 'CDPATH', 'ENV', 'BASH_ENV'}; Now all we have to do is to reuse the code from the previous section. First we move the core program into the external.pl file, add the shebang first line so the program will be executed by Perl, tell the program to run under Taint mode (-T) and possibly enable the warnings mode (-w) and make it executable: external.pl ----------- #!/usr/bin/perl -Tw open STDIN, '/dev/null' or die "Can't read /dev/null: $!"; open STDOUT, '>/dev/null' or die "Can't write to /dev/null: $!"; open STDERR, '>/tmp/log' or die "Can't write to /tmp/log: $!"; select STDERR; local $|=1; warn "started\n"; # do something time-consuming sleep 1, warn "$_\n" for 1..20; warn "completed\n"; Now we replace the code that moved into the external program with proper_fork_exec.pl ------------------- use strict; use POSIX 'setsid'; use Apache::SubProcess; $ENV{'PATH'} = '/bin:/usr/bin'; delete @ENV{'IFS', 'CDPATH', 'ENV', 'BASH_ENV'}; my $r = shift; $r->send_http_header("text/html"); $SIG{CHLD} = 'IGNORE'; defined (my $kid = fork) or die "Cannot fork: $!\n"; if ($kid) { print "Parent has finished, kid's PID: $kid\n"; } else { $r->cleanup_for_exec(); # untie the socket chdir '/' or die "Can't chdir to /: $!"; open STDIN, '/dev/null' or die "Can't read /dev/null: $!"; open STDOUT, '>/dev/null' or die "Can't write to /dev/null: $!"; open STDERR, '>&STDOUT' or die "Can't dup stdout: $!"; setsid or die "Can't start a new session: $!"; exec "/home/httpd/perl/external.pl" or die "Cannot execute exec: $!"; } Notice that system "/home/httpd/perl/external.pl" or die "Cannot execute system: $!"; CORE::exit(0); Another important nuance is that we have to close all STD* stream in the forked process, even if the called program does that. If the external program is written in Perl you may pass complicated data stuctures to it using one of the methods to serialize Perl data and then to restore it. The master.pl --------- # we are within the mod_perl code use Storable (); my @params = (foo => 1, bar => 2); my $params = Storable::freeze(\@params); exec "./slave.pl", $params or die "Cannot execute exec: $!"; slave.pl -------- #!/usr/bin/perl -w use Storable (); my @params = @ARGV ? @{ Storable::thaw(shift)||[] } : (); # do something As you can see, master.pl serializes the Starting a Short Running External ProgramSometimes you need to call an external program and you cannot continue before this program completes its run and optionally returns some result. In this case the fork solution doesn't help. But we have a few ways to execute this program. First using system(): system "perl -e 'print 5+5'" We believe that you will never call the perl interperter for doing this simple calculation, but for the sake of a simple example it's good enough. The problem with this approach is that we cannot get the results printed to my $result = `perl -e 'print 5+5'`; or: my $result = qx{perl -e 'print 5+5'}; the whole output of the external program will be stored in the Of course you can use other solutions, like opening a pipe ( Executing
|
|
|
About Triggers | Media Kit | Security | Triggers | Login |
All times are recorded in UTC. Linux is a trademark of Linus Torvalds. Powered by Linux 2.4, Apache 1.3, and PHP 4 Legal Notices, Licensing, Reprints, & Permissions, Privacy Policy. |