Deep Dive in to Citrix ADC Remote Code Execution, CVE-2019-19781

Last month, a critical vulnerability in Citrix ADC and Citrix Gateway was published under CVE-2019-19781. The vulnerability caught our attention as it suggested that an unauthenticated adversary could leverage it to compromise the device. Although the original discovery was made by Positive Technologies and Paddy Power Betfair, there were no details publicly available on how to exploit this, as such this warranted further research.

Vulnerability Analysis

Although there were no publicly available details on how to exploit this issue, the mitigation advisory from Citrix revealed a potential clue to the type of vulnerability that it was.

From this information we can see the path where presumably the vulnerability exists (/vpns/) and that is a possible directory traversal vulnerability. With this in mind, we began to look for definitions of the /vpns path in the httpd.conf file and discovered that the /vpn/portal/scripts/ is handled by the NetScaler::Portal::Handler Perl Module (Handler.pm).

Several scripts were contained in this directory, but since this vulnerability was a potential directory traversal we opted to investigate any potential code paths to file write operations.

This led to the discovery of the following code in the UsersPrefs perl module:

sub csd {
        my $self = shift;
        my $skip_read = shift || "";
  # Santity Check
    my $cgi = new CGI;
print "Content-type: text/html\n\n";

// Username variable initialized by the NSC_USER HTTP Header
    my $username = Encode::decode('utf8', $ENV{'HTTP_NSC_USER'}) || errorpage("Missing NSC_USER header.”); <- MARK THIS

    $self->{username} = $username;
...
    $self->{session} = %session;

// Constructing the path from the username.
        $self->{filename} = NetScaler::Portal::Config::c->{bookmark_dir} . Encode::encode('utf8', $username) . '.xml’;
        if($skip_read eq 1) {
                return;
        }

In short, this code essentially builds a path from from the NSC_USER HTTP header without any sanitisation. Consequently, any script that calls the csd function, will be able to trigger the directory vulnerability.

Almost all the scripts used this function, however the most interesting one to us was the newbm.pl script file:

my $cgi = new CGI;
print "Content-type: text/html\n\n";
my $user = NetScaler::Portal::UserPrefs->new();
my $doc = $user->csd();
...
my $newurl = Encode::decode('utf8', $cgi->param('url'));
my $newtitle = Encode::decode('utf8', $cgi->param('title'));
my $newdesc = Encode::decode('utf8', $cgi->param('desc'));
my $UI_inuse = Encode::decode('utf8', $cgi->param('UI_inuse'));
...
my $newBM = {   url => $newurl,
    title => $newtitle,
    descr => $newdesc,
    UI_inuse => $UI_inuse,
};
...

The script creates an array with information from several parameters and then later on, calls the filewrite function which will write the content to an XML file on disk.

if ($newBM->{url} =~ /^\/){
   push @{$doc->{filesystems}->{filesystem}}, $newBM;
 } else { # bookmark
   push @{$doc->{bookmarks}->{bookmark}}, $newBM;
 }
// Writing XML file to disk
 $user->filewrite($doc);

Essentially, we have a partial file write primitive where we control the path but not the file extension, and control some content inside the xml file. With this vulnerability alone there was little we could do, but after seeing an article by Craig Yong, it highlighted a potential avenue for exploitation using the Perl Template Toolkit.

With further research, we discovered it was possible to insert specific instructions on the XML file, that if parsed by the template engine could be used to execute commands.

An example of the template parsing can be found below:

Revisiting our exploit, we now have a partial file write primitive which we can use to inject Perl Template Toolkit instructions, but still require a method to force a script to parse the template.

Finding all the references on the code that actually used the templates, the Handler.pm module looked promising:

The $tmplfile variable is constructed from the HTTP Request Path and a new template is built and processed for that file.

Copying the test.xml file we previously created to the templates directory, we can trigger the template parsing.

To summarise, In order to exploit the vulnerability the following steps should be followed:

• Discover a way to execute perl code through the template (Requires a bypass),
• Write crafted XML file on the templates directory using the path traversal,
• Browse to the uploaded XML file, triggering the template parsing.

The last piece of the puzzle is using the template to execute arbitrary commands, which in the default configuration “is not” possible. There is an undocumented feature which allows arbitrary perl code to be executed but at the current time, we don’t plan on documenting this technique. However, using this information security teams should have sufficient details to reproduce the issue but without the granular details to execute code.

Due to the number of devices impacted, MDSec have decided to not provide a ready made exploit for this vulnerability, however we are aware of multiple actors who have now weaponised this vulnerability and felt it important to share this research so others can take appropriate action.

Update on 14 January 2020

Template injection to Remote Command Execution

As the exploit has now been made public, we decided to publish the details on how to achieve command execution by abusing the template injection. There are two methods two achieve command execution and I’ll start with the first one which is used by the public exploits.

Template BLOCK arbitrary perl code execution

After publishing our blogpost, we were made aware of the following GitHub issue on Perl Template Toolkit:

This issue describes a method on how to execute arbitrary perl code and it seems that all the public exploits use this example to achieve command execution. As this was the initial method we used to achieve command execution, we wanted to give more insights on the vulnerability itself.

According to the Template Toolkit documentation, the template variable is a special variable and contains a reference to the main template being processed. This object type is Template::Document as can be seen by trying to print it:

Having a reference to an object, we can call methods with controlled parameters. Investigating the source code for this class, we can see the new method is public and can be called:

# new(\%document)
#
# Creates a new self-contained Template::Document object which
# encapsulates a compiled Perl sub-routine, $block, any additional
# BLOCKs defined within the document ($defblocks, also Perl sub-routines)
# and additional $metadata about the document.
#------------------------------------------------------------------------

sub new {
    my ($class, $doc) = @_;
    my ($block, $defblocks, $variables, $metadata) = @$doc{ qw( BLOCK DEFBLOCKS VARIABLES METADATA ) };
    $defblocks ||= { };
    $metadata  ||= { };

    # evaluate Perl code in $block to create sub-routine reference if necessary
    unless (ref $block) {
        local $SIG{__WARN__} = \&catch_warnings;
        $COMPERR = '';

        # DON'T LOOK NOW! - blindly untainting can make you go blind!
        $block = each %{ { $block => undef } } if ${^TAINT};    #untaint

        $block = eval $block;
        return $class->error($@)
            unless defined $block;
    }

This method will take the BLOCK parameter and call eval on it, leading to arbitrary perl code execution. Additionally, there is another special variable called component that can be used to perform the same attack.

Command Injection on DATAFILE plugin

After achieving command execution with the first method, further research was done on identifying other methods to achieve command execution.
Seeking the source code for any dangerous function calls, the following code was discovered on the Datafile plugin.

sub new {
    my ($class, $context, $filename, $params) = @_;
    my ($delim, $line, @fields, @data, @results);
    my $self = [ ];
    local *FD;
    local $/ = "\n";

    $params ||= { };
    $delim = $params->{'delim'} || ':';
    $delim = quotemeta($delim);

    return $class->fail("No filename specified")
        unless $filename;

    open(FD, $filename)
        || return $class->fail("$filename: $!");

    # first line of file should contain field definitions
    while (! $line || $line =~ /^#/) {
        $line = <FD>;
        chomp $line;
        $line =~ s/\r$//;
    }

If you have ever audited perl code, you know that using the open function with only two parameters is dangerous. When prepended with a pipe |, the open function will interpret the rest of it as a command invocation, and as we have control over the filename we can use it to achieve command execution.

Creating a specially crafted template, that invokes the DATAFILE plugin leads to arbitrary command execution:

As can be seen below, the folder was created successfully:

This vulnerability was discovered by several people as the original post from Craig Yong mentioned “It is important to note, however, that certain payloads will cause NetScaler to excessively log errors until it fills up the /var partition.”. Following a Twitter conversation, we also noted that @noperator from BishopFox also used this technique to achieve command execution.

The exploit can be found here.

A demonstration of the exploitation can be shown below:

Mitigation

Citrix released a mitigation to prevent exploitation of this issue as documented in Mitigation Steps for CVE-2019-19781. We strongly recommend that any Citrix ADC users apply this mitigation as an immediate priority.

Furthermore, detection of exploitation attempts can be recognised by alerting on URLs containing a POST request with the “/vpns/” and “/../” strings, followed by a GET request to a file ending with the xml extension as highlighted by @buffaloverflow.

This blog post was written by Rio Sherri.