Web Application Firewall in GO “feat” OWASP ModSecurity Core Rule Set

Wow, that’s a mouthful! Want to turn your GO microservice or (smart?) reverse proxy server into a web application firewall? Then this article is for you.

We’ll be leveraging ModSecurity — the very same module that has been securing Apache and Nginx for years. The latest version 3.0 branch of ModSecurity can now be compiled as a standalone library. No doubt this was done to simplify the process of enabling ModSecurity as an Nginx module. But now that it is a standalone module, guess what? We can access the library from Go using Cgo.

In this installment of my Smart Reverse Proxy in Go series, we’ll compile ModSecurity, write Cgo code to access the library and integrate it to a Go reverse proxy. OK, let’s Git It Done!

Let’s install some prerequisites. I’m using Ubuntu 18.04.

$ apt-get install -y apt-utils autoconf automake build-essential git libcurl4-openssl-dev libgeoip-dev liblmdb-dev libpcre++-dev libtool libxml2-dev libyajl-dev pkgconf wget zlib1g-dev

Download the ModSecurity version 3 branch. Note: version < 3 will not work because it is written as an Apache module.

$ git clone --depth 1 -b v3/master --single-branch https://github.com/SpiderLabs/ModSecurity

Compile and build.

$ cd ModSecurity 
$ git submodule init 
$ git submodule update 
$ ./build.sh 
$ ./configure 
$ make 
$ make install

Assuming all went well, you will have a standalone ModSecurity library:

$ ls /usr/local/modsecurity/lib/
libmodsecurity.a  libmodsecurity.la  libmodsecurity.so  libmodsecurity.so.3  libmodsecurity.so.3.0.4  pkgconfig

Next we’ll need to download rules and configuration files:

$ curl -O https://raw.githubusercontent.com/lsgroup/SmartReverseProxy/master/modsecdefault.conf$ curl https://raw.githubusercontent.com/coreruleset/coreruleset/v3.3/dev/crs-setup.conf.example > crs-setup.conf$ curl -O https://raw.githubusercontent.com/coreruleset/coreruleset/v3.3/dev/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf

Now put your Go developer hat on and write our basic reverse proxy code (see Smart Rate Limiter in GO) in main.go.

package main
import (
  "encoding/json"
  "fmt"
  "log"
  "net/http"
  "net/http/httputil"
  "net/url"
  "github.com/gorilla/mux"
)
func adminHelloHandler(w http.ResponseWriter, req * http.Request) {
  w.Header().Set("Content-Type", "application/json")
  w.WriteHeader(http.StatusOK)
  resp := map[string]interface{} {"status": "smart reverse proxy admin access - hello!"}
  out, err := json.Marshal(resp)
  if err != nil {
    log.Println(err)
  }
  fmt.Fprintf(w, string(out))
}
func testHandler(w http.ResponseWriter, req * http.Request) {
  log.Print("testHandler")
  url, err := url.Parse("http://www.lightbase.io/freeforlife")
  if err != nil {
    log.Println(err)
  }
  log.Print(url)
  proxy := httputil.NewSingleHostReverseProxy(url)
  director := proxy.Director
  proxy.Director = func(req * http.Request) {
    director(req)
    req.Header.Set("X-Forwarded-Host", req.Header.Get("Host"))
    req.Host = req.URL.Host
    req.URL.Path = url.Path
  }
  proxy.ServeHTTP(w, req)
}
func main() {
    bind := ":3080"
    adminbind := "localhost:3081"
    gmux := mux.NewRouter()
    gmux.HandleFunc("/test", testHandler).Methods("GET")
    adminmux := mux.NewRouter()
    adminmux.HandleFunc("/test", adminHelloHandler).Methods("GET")
    go func() {
      if err := http.ListenAndServe(adminbind, adminmux);
      err != nil {
        log.Fatalf("unable to start server: %s", err.Error())
      }
    }()
    log.Printf("starting smart reverse proxy on [%s], administrative endpoint: [ % s] with / test ", bind, adminbind)
    initModSec()
    if err := http.ListenAndServe(bind, limitMiddleware(gmux)); err != nil {
      log.Fatalf("unable to start server: %s", err.Error())
    }
    return
}

In my previous article, we used the limitMiddleware function to implement a smart rate limiter. We’ll do the same here but instead, the limitMiddleware function will be used to perform interventions on behalf of ModSecurity.

Create a new Go file: modsec.go.

package main
/*
#cgo CPPFLAGS: -I/usr/local/modsecurity/include
#cgo LDFLAGS: /usr/local/modsecurity/lib/libmodsecurity.so
#include "modsecurity/rules_set.h"
#include "modsecurity/modsecurity.h"
#include "modsecurity/transaction.h"
#include "modsecurity/intervention.h"
ModSecurity *modsec = NULL;
RulesSet *rules = NULL;
void init() {
    if (modsec!=NULL) {
        return;
    }
    modsec = msc_init();
    rules = msc_create_rules_set();
    const char *error = NULL;
    msc_rules_add_file(rules, "modsecdefault.conf", &error);
    msc_rules_add_file(rules, "crs-setup.conf", &error);
    msc_rules_add_file(rules, "rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf", &error);
    return;
}
int process(const char *uri)
{
    Transaction *transaction = NULL;
    transaction = msc_new_transaction(modsec, rules, NULL);
    msc_process_connection(transaction, "127.0.0.1", 80, "127.0.0.1", 80);
    fprintf(stderr, "URI=%s\n", uri);
    msc_process_uri(transaction, uri, "CONNECT", "1.1");
    msc_process_request_headers(transaction);
    msc_process_request_body(transaction);
    ModSecurityIntervention intervention;
    intervention.status = 200;
    intervention.url = NULL;
    intervention.log = NULL;
    intervention.disruptive = 0;
    int inter = msc_intervention(transaction, &intervention);
    fprintf(stderr, "intervention=%i\n", inter);
    return inter;
}
*/

And yes, the code is commented out. That is how Cgo works, you write your C code inside a comment block in your Go file.

Let’s see what we did here. First we specified ModSecurity C library with LDFLAGS and header includes:

#cgo LDFLAGS: /usr/local/modsecurity/lib/libmodsecurity.so
#include "modsecurity/..."

Then we have an init() function that is meant to be run once at the start to initialize the ModSecurity objects and suck in the rules. The init() function works on the global pointers modsec and rules.

Next up is the process() function. This function will be run per http request. It will process the request as a transaction and perform the web application firewall filtering magic that is ModSecurity.

We create a new Transaction object and run it through ModSecurity’s C api functions: msc_process_connection(), msc_process_uri(), msc_process_request_headers(), and msc_process_request_body().

Once the processing functions are called, we create ModSecurityIntervention object and use it to call the msc_intervention() function.

int inter = msc_intervention(transaction, &intervention);

This is where ModSecurity determines if the request requires intervention. How paranoid or lax the library and what it checks for can be configured via configuration and rule files. The msc_intervention function returns 0 if no intervention is advised and 1 if it thinks mitigation is required.

Now onto the remaining half of our modsec.go file:

import "C"
import (
 "log"
 "net/http"
 "unsafe"
 "time"
)
func modsec(url string) int {
  Curi := C.CString(url)
  defer C.free(unsafe.Pointer(Curi))
  start := time.Now()
  inter := int(C.process(Curi))
  elapsed := time.Since(start)
  log.Printf("\n========\nmodsec()=%i, elapsed: %s", inter, elapsed)
  return inter
}
func initModSec() {
 C.init()
}
func limitMiddleware(next http.Handler) http.Handler {
 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
  log.Printf("req.URL %s", r.URL)
  inter := modsec(r.URL.String())
  if inter>0 {
    log.Printf("==== Mod Security Blocked! ====")
    http.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest)
    return
  }
  next.ServeHTTP(w, r)
 })
}

We start with the limitMiddleware function. Since it acts as a middleman for our reverse proxy engine, we have the ability to “intevene” the request and response. To see if we should intervene, we run the modsec function passing in the uri of the request.

inter := modsec(r.URL.String())

Looking at modsec(), we run our Cgo code:

inter := int(C.process(Curi))

Since the process function returns the intervention value 1 or 0, the modsec() function will return with a boolean decision on whether or not to intervene. We also put in a little timer to see how this performs:

log.Printf("\n========\nmodsec()=%i, elapsed: %s", inter, elapsed)

initModSec() is called back in our main function just before http.ListenAndServe proxy is started, which guarentees that our modsec and rules libraries are configured and ready by the time the reverse proxy starts.

OK, now we’re ready to rumble! Just remember to include modsecurity in your LD_LIBRARY_PATH.

$ export LD_LIBRARY_PATH=/usr/local/modsecurity/lib/
$ go run *.go
2020/05/27 18:33:22 starting smart reverse proxy on [:3080], administrative endpoint: [ localhost:3081] with / test

On another window, let’s throw some requests at our WAF:

$ curl localhost:3080/test

2020/05/27 18:44:41 req.URL /test
2020/05/27 18:44:41 
========
modsec()=%!i(int=0), elapsed: 3.004067ms

So that request was pretty tame. Here a known SQL Injection hack:

$ curl localhost:3080/test/artists.php?artist=0+div+1+union%23foo*%2F*bar%0D%0Aselect%23foo%0D%0A1%2C2%2Ccurrent_user

2020/05/27 18:45:38 req.URL /test/artists.php?artist=0+div+1+union%23foo*%2F*bar%0D%0Aselect%23foo%0D%0A1%2C2%2Ccurrent_user
Server log callback is not set -- [client 127.0.0.1] ModSecurity: Warning. detected SQLi using libinjection. [file "rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "45"] [id "942100"] [rev ""] [msg "SQL Injection Attack Detected via libinjection"] [data "Matched Data: 1UE1 found within ARGS:artist: 0 div 1 union#foo*/*bar\x0d\x0aselect#foo\x0d\x0a1,2,current_user"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/test/artists.php"] [unique_id "1590630338"] [ref "v33,53"] [hostname "127.0.0.1"] [uri "/test/artists.php"] [unique_id "1590630338"]
2020/05/27 18:45:38 
========
modsec()=%!i(int=1), elapsed: 1.858213ms
2020/05/27 18:45:38 ==== Mod Security Blocked! ====

ModSecurity logs the request’s rule match and returns true for intervention. And it does it in less than 2 milliseconds! This metric is just anecdotal as we’ve only loaded 1 rule file and there are over 30 rule files you might load for any real WAF implementation.

And there you have it. OWASP ModSecurity Core Rule Set implemented as a Go Web Application Firewall. It’s an indispensable part of the Smart Reverse Proxy blueprint we’ve been working on here.

Oh and in case you were wondering why the “feat” Mod Security, it’s because we’ll be implementing Shadow Daemon in a Go Web Application Firewall. So stay tuned for “feat” Shadow Daemon. And as always, keep Git’n it Done!