New anti-APT tools are no silver bullets: An independent test of APT attack detection appliances

New anti-APT tools are no silver bullets:
An independent test of APT attack detection appliances

CrySyS Lab, BME

November 26, 2014.

The term Advanced Persistent Threat (APT) refers to a potential attacker that has the capability and the intent to carry out advanced attacks against specific high profile targets in order to compromise their systems and maintain permanent control over them in a stealthy manner. APT attacks often rely on new malware, which is not yet known to and recognized by traditional anti-virus products. Therefore, a range of new solutions, specifically designed to detect APT attacks, have appeared on the market in the recent past, including Cisco’s SourceFire, Checkpoint, Damballa, Fidelis XPS, FireEye, Fortinet, LastLine, Palo Alto’s WildFire, Trend Micro’s Deep Discovery and Websense.

While these tools are useful, determining their real effectiveness is challenging, because measuring their detection rate would require testing them with new, previously unseen malware samples with characteristics similar to those of advanced malware used by APT attackers. Developing such test samples require special expertise and experience obtained either through the development of advanced targeted malware or at least through extensive analysis of known samples.

We at MRG Effitas, together with our colleagues at CrySyS Lab, decided to join our forces and perform a test of leading APT attack detection tools using custom developed samples. MRG Effitas has a lot of experience in testing anti-virus products, while the CrySyS Lab has a very good understanding of APT attacks gained through the analysis of many targeted malware campaigns. Therefore, collaborating and bringing together our complementary sets of expertise looked like a promising idea. Our goal was not to determine the detection rates of different APT attack detection products, because that would have required testing with a large set of custom developed malware samples, which was not feasible to obtain within the limited time frame and with the limited resources we had for the test. Instead, our goal was simply to implement some ideas we had for bypassing cutting-edge APT attack detection tools without actually being detected, and to test if our ideas really work in practice.

We developed 4 custom samples in 2 weeks and without access to any APT attack detection tools during the development, and then later tested with these samples 5 APT attack detection solutions in Q3 2014. All 5 tested products are well-established in the market; however, we cannot mention vendor names publicly. The result of the test was alarming:
– one of our 4 custom samples bypassed all 5 products,
– another sample of the remaining 3 samples bypassed 3 products,
– only the two simplest samples have been detected by the tested products, and even those triggered alarms with low severity in some cases.

We made the full report ( on our test available online. It contains our test methodology, including a brief description of each sample we developed for the purpose of the test, and we also present in it the test results in more details. We decided to publish this report for multiple reasons:
– First of all, we believe that our test was more appropriate for evaluating the detection capabilities of APT attack detection tools than some earlier, heavily criticized tests were, because unlike earlier tests, we used custom developed samples that resemble the malware used in APT attacks.
– Second, some of the products that we tested seem to be overestimated by the users who believe that those products are silver bullets. On the other hand, we have already emphasized at multiple occasions that these products can and will be bypassed by determined attackers. Our test is a clear proof of this, and if we could do that, then APT attackers will also be able to do that, if they have not done so yet.
– Third, we observed that some vendors of APT attack detection tools are often reluctant to participate in tests that try to evaluate the effectiveness of their products. On the one hand, we understand their caution, but on the other hand, we all know that the approach of security by obscurity has its own pitfalls. By publishing this report, we would like to encourage anti-APT tool vendors to participate in independent tests more readily and cooperatively, in order to have sufficient amount of convincing results about their products, based on which well-informed decisions can be made by the users.
– And last but not least, we believe that there are significant differences in the APT detection capabilities of the tested products, and users should be aware that not all vendors provide the same detection rate.

The test sample that bypassed all 5 tested products was developed by the CrySyS Lab. It is a custom designed sample written in C++ with a server side written in PHP. It was designed to be as stealthy as possible. It is downloaded by the victim as part of an HTML page, where it is actually hidden in an image with steganography. The downloaded page also contains scripts that extract an executable from the image when the user clicks on something that appears to be a download button. Once the sample is running, it can communicate with a remote C&C server. To hide the C&C network traffic, the sample simulates a user clicking on links in a web forum, and downloads full HTML pages with CSS style sheets and images. The real C&C traffic is hidden inside these HTTP requests. The sample allows for file download from and upload to the C&C server, as well as remote execution of commands on the victim computer.

We named this test sample BAB0, which (babo) means hobbit in Hungarian, as its objective was to stealthily bypass all state-of-the-art defenses, while actually being very simple, and this situation shows a parallel to the story of the Lord of the Rings, where Frodo, the small hobbit managed to bypass all defenses of the fearsome Sauron, the Lord of Mordor, and reached Amon Amarth, where the One Ring was finally destroyed.

We have a strong intention to publish BAB0 in the near future. This may seem to be controversial, as making the details of BAB0 publicly available can help attackers. We have a different opinion: Powerful attackers have probably been using already similar tricks, but apparently detection tools are not yet prepared to cope with them. By publishing BAB0, we push anti-APT vendors to strengthen their products, which will ultimately make the attackers’ job harder.

For further information, please contact either Zoltan Balázs ([email protected]) or Levente Buttyán ([email protected]). Please note that we cannot provide any vendor specific information about the tests, but we can help organizations to test the products integrated in their environment.

Read More

Bypass hardware firewalls – DEF CON 22

This is a follow-up post in connection with my DEF CON 22 presentation.

TL;DR: attackers having admin privileges on Linux/Windows systems can mess with the hardware firewall between the attacker and the server, and use the same ports for backdoor communication as it is allowed in the firewall (e.g. 22, 80, 443, 3389, etc). First, the attacker has to exploit the server, and only after that can bypass the firewall. If you are looking for a tool to bypass a firewall before exploiting a server, this tool won\’t help you. If you are still interested, here is the tool:

And there are some cool photos about DEF CON at the end of this post.

Introduction (the problem)

Penetration testers (or black hat hackers) sometimes face the problem of restrictive firewalls blocking the backdoor C&C communication. Sometimes it is possible to hack the server service (and get webshell, or temporary C&C via socket reuse exploits), and install the attacker tools/backdoors. But the hardware firewall between the attacker and the victim server might block the bind/reverse shell, and covert channels like ICMP/DNS/IPv6/UDP/proxy-hops might be blocked as well. There is in-the-wild malware solving this problem, e.g. the Hikit rootkit. This rootkit creates a new network interface (like software firewalls do), \”catches\” the traffic sent to the legit server service, and if the backdoor communication is found in the traffic, this data flow is handled differently, not by the legit service. At the hardware firewall level, the traffic will be allowed, as it is using the same destination TCP port as the legit service. Thus attackers can bypass hardware firewall restrictions, but not penetration testers. This is a gap between attackers and testers, which had to be closed. The idea of the hwfwbypass tool was born.


Details (the solution)

We developed a tool called hwfwbypass. The tool is a network filter kernel driver, based on the Windivert project. Admin level privileges are needed to install the tool. The kernel driver is digitally signed with a trusted signature, thanks to Nemea software development and the Windivert project.

After starting the hwfwbypass tool, when a client (attacker) connects from the TCP source port specified in the client_sourceport command line parameter, to the TCP destination port original_dstport, the kernel driver will redirect the traffic to the new_dstport on the server. The kernel driver first filters for traffic which has the TCP source and destination ports supplied in the command line argument, and if the packet matches, the packet is removed from the network flow, modified by changing the TCP destination port, and the packet is finally reinjected. If this packet is a TCP SYN packet, the (backdoor) service listening on new_dstport will handle the connection, and reply with SYN/ACK. The hwfwbypass tool will rewrite the TCP source port in this case from new_dstport to original_dstport, and reinject the new packet. The result is that we have a bidirectional TCP traffic, which uses the same port from a network perspective as the legitim service (e.g. webserver, RDP server, etc.), but is handled by a different process – which can be our backdoor server process. It is important to note that bind shells can be used with this tool, and not reverse shells.


There are additional use cases for the hwfwbypass tool:
1. The server can be used to be a hop/proxy to other servers in the same network segment, or to other servers which are not blocked by the firewall.

2. Even if the firewall allows the backdoor C&C communication without any hwfwbypass tricks, it might be useful to hide the traffic, thus log analysis/incident response won\’t reveal that port 4444 on the webserver has an abnormal traffic peak.

hwfwbypass.exe client_sourceport original_dstport new_dstport [disablechecksum] [debug]

hwfwbypass.exe 1337 3389 31337
hwfwbypass.exe 1337 3389 31337 disablechecksum debug

Here is a youtube video about the tool:

And if you are lazy, there is a metasploit post module, controlling the netcat start, uploading and starting the hwfwbypass tool, creating the new session with the stealth port, making cofee etc.


Benefits of the solution

  • It is using Windows supported network filter, thus this functionality will work in the future.
  • It ships with valid signed driver.
  • Any kind of backdoor traffic can be used with the tool. I have tested it with Netcat, Meterpreter TCP bind shell and a RAT with bind shell.
  • The server side does not need any specific tools, only the hwfwbypass and the RAT/bind shell. On the client side, one might need NetCat.

Drawbacks of the solution

  • If there is a network address translation (NAT) between the attacker and the server, the tool won\’t work. Deal with it.
  • NG (next generation) firewalls, which check the protocol, can block the backdoor communication. See improvement possibilities for the solution.


From the attacker side, one have to set the TCP source port for the backdoor client application, which can be done with netcat. We used the netcat from the Nmap project. The following command makes netcat to listen on localhost TCP port 4444, and forward all traffic to RDP.SER.VER.IP and RDP port, but with the source port set 1337.

ncat -kl 4444 -c \”ncat -p 1337 RDP.SER.VER.IP 3389\”

Improvement possibilities

1. My first idea was that the backdoor bind shell should listen on localhost only, thus even the local software firewall can be bypassed, if any. Unfortunately, it turned out that it is not as easy as I thought. The Windivert project (and the official Microsoft Network filter driver) allows the rewrite of the interfaces during packet reinject to do this trick, but it is not working. This might be because there is no real loopback interface on Windows. I\’m not sure how to solve this, at the moment I have added new firewall rules to my metasploit scripts to solve this issue – but this only supports Windows firewall.

2. Packing the whole solution into one (static) binary. I also tried this, but some solutions were not compatible with newer platforms (e.g. Windows 2012). I also had to solve to drop the kernel driver on demand, but I had no time to do that. If you feel, you can send me a quick patch and compilation instructions to solve these problems.

3. The code is a bit of a mess. It is a POC, and it worked at least once. There is some ugly code reuse in the code, which needs some object-oriented refactoring.

4. It would be nice to have some kind of wrapper code, which can encapsulate the backdoor traffic in different protocols, e.g. SSH, HTTP, HTTPS, RDP. This has to be implemented on both client and server side, as some kind of proxy tool. If you are aware of any projects (python, Java, anything) doing something like this, let me know! For example stunnel is a good solution to encapsulate the traffic in SSL.


Firewalls has been blocking network attackers in the last 20 years. Although there are ways to find holes in the firewall, a real restricted environment could block the whole attack chain. So far, there was no solution for this problem. Our tool requires admin level privileges on the attacked server to bypass the firewall, but if it is possible, the tool can be used to bypass firewalls, harden log analysis or incident response, confuse defenders.


















Read More

NEW! MRG Effitas Project 40 – MRG Effitas Online Banking Browser Security Certification Project Q2 -2014


We are pleased to announce the publication of our Online Banking / Browser Security Certification Project for the Q2 2014 period.


MRG Effitas provides two levels of testing; Level 1, where we simply test a vendor’s product and provide a report for that quarter’s assessment, and Level 2 (which includes Level 1), where we liaise with the vendors during testing and alert them to any issues found with their technology and provide all engineering and technical support required for them to counter these issues.


The purpose of Level 2 participation is that it serves as an external QA service for vendors, helping them improve the efficacy of their product and thereby enhance the protection provided to their clients.


Level I and 2 reports are published separately, but the level two report should be seen as the current level of performance of the products tested.


You can download both reports by visiting the Current Projects section on our homepage.





Read More