[ Thanks to and Scott Courtney for this link. ]

From: Brian Behlendorf <[email protected]>
Subject: Apache Software Foundation Server compromised, resecured.
To: <[email protected]>
Date: Wed, 30 May 2001 23:05:59 -0700 (PDT)


Earlier this month, a public server of the Apache Software Foundation
(ASF) was illegally accessed by unknown crackers.  The intrusion into
this server, which handles the public mail lists, web services, and
the source code repositories of all ASF projects was quickly
discovered, and the server immediately taken offline.  Security
specialists and administrators determined the extent of the intrusion,
repaired the damage, and brought the server back into public service.

The public server that was affected by the incident serves as a source
code repository as well as the main distribution server for binary
release of ASF software.  There is no evidence that any source or binary
code was affected by the intrusion, and the integrity of all binary
versions of ASF software has been explicitly verified.  This includes
the industry-leading Apache web server.

Specifically: on May 17th, an Apache developer with a sourceforge.net
account logged into a shell account at SourceForge, and then logged
from there into his account at apache.org.  The ssh client at
SourceForge had been compromised to log outgoing names and passwords,
so the cracker was thus able get a shell on apache.org.  After
unsuccessfully attempting to get elevated privileges using an old
installation of Bugzilla on apache.org, the cracker used a weakness in
the ssh daemon (OpenSSH 2.2) to gain root privileges.  Once root, s/he
replaced our ssh client and server with versions designed to log names
and passwords.  When they did this replacement, the nightly automated
security audits caught the change, as well as a few other trojaned
executables the cracker had left behind.  Once we discovered the
compromise, we shut down ssh entirely, and through the serial console
performed an exhaustive audit of the system.  Once a fresh copy of the
operating system was installed, backdoors removed, and passwords
zeroed out, ssh and commit access was re-enabled.  After this, an
exhaustive audit of all Apache source code and binary distributions
was performed.

The ASF is working closely with other organizations as the investigation
continues,  specifically examining the link to other intrusion(s), such
as that at SourceForge (http://sourceforge.net/) [ and php.net
(http://www.php.net/). ]

Through an extra verification step available to the ASF, the integrity
of all source code repositories is being individually verified by
developers.  This is possible because ASF source code is distributed
under an open-source license, and the source code is publicly and freely
available.  Therefore, the ASF repositories are being compared against
the thousands of copies that have been distributed around the globe.
While it was quickly determined that the source code repositories on the
ASF server were untouched by the intruders, this extra verification step
provides additional assurance that no damage was done.

As of Tuesday, May 29, most of the repository has been checked, and as
expected, no problems have been found.  A list of verified modules
will be maintained, and is available here:
http://www.apache.org/info/hack-20010519.html

Because of the possible link of the ASF server intrusion to other
computer security incidents, the investigation is ongoing.  When
complete, the ASF will offer a complete and public report.

The Apache Software Foundation strongly condemns this illegal
intrusion, and is evaluating all options, including prosecution of the
individual(s) responsible to the fullest extent of the law.  Anyone
with pertinent information relating to this or other related events
should contact .  Anyone from the media with further
interest should contact .

Thanks.

        Brian Behlendorf
        President, Apache Software Foundation

Related Stories:
Slashdot: SourceForge Server Compromised(May 29, 2001)