Automating FireDrill AdSim Configuration with InfernoAuger
In this post, we will be looking at a tool we have developed to automate many of the components of the popular adversary simulation tool, FireDrill. FireDrill is an application that is designed and developed by the company AttackIQ; more information can be found here. FireDrill provides a library of configurable attacks to help determine if your controls can either stop or detect them. These configured “scenarios” are placed into “assessments” which are simply a collection of configured tests. There are times where an assessment may need to be re-created, which was the main reason for the creation of our tool. It has since, however, evolved into so much more.
I now present our new tool… InfernoAuger! InfernoAuger can automate many components within the FireDrill application, such as: the creation and update of assessments utilizing a master configuration file, pushing FireDrill detection results into Splunk, monitoring the progress of running assessments, providing e-mail alerts to newly added scenarios, and more. There are currently five modules within InfernoAuger, all performing different tasks such as the ones mentioned above. The current list of modules are:
- Main: This is the module that can create or update an assessment based on a master build configuration file
- Detection: This module pushes assessment detection results into Splunk for further correlation or analytics
- Status: This checks the status of a currently running or previous run of an assessment and provide basic statistics on the results
- Scenarios: This retrieves a list of all MITRE ATT&CK scenarios within FireDrill and outputs information about them
- Update: This module checks for new MITRE ATT&CK scenarios since its previous run and sends an e-mail with any found
Before going into the capabilities and usage of each module, it is important to note a couple of items. First, this program is geared towards utilizing the MITRE ATT&CK framework (information can be found here), so the vast majority of its components are specific to the scenarios within FireDrill that are labelled with this framework. Secondly, our preferred SIEM of choice is Splunk, so the detection module is currently only capable of pushing results to this platform. With that out of the way, let’s get into each of the modules!
Main
Setup
As mentioned before, the “Main” module is the core component of InfernoAuger. From here, you can create a brand new assessment with the press of a key, or update an already existing one with new scenarios. This requires a specially crafted CSV file to be used as input; an example of such a CSV file is included with the code in the “samples” directory. The CSV has four main components: the name of the scenario, the scenario ID within FireDrill, the test it belongs to, and the JSON parameters for the configuration. At our organization, we divide our AdSim tests by sophistication of the attack. Due to this, the only currently supported test names are “Novice,” “Practitioner,” and “Expert.” So when creating a build configuration, your test names will have to be one of these three for each scenario.
To generate the JSON parameters for each scenario, simply log in to your FireDrill tenant, click the hamburger icon in the top left, choose Settings and then FireDrill API. You will be brought to a page that looks similar to the following:
Screenshot of the FireDrill API Portal
Once here, click Scenario Parameter Builder and search for the scenario you want to add to the build script. Once you choose and configure the scenario to your liking, simply click Copy next to the Generated Parameters JSON at the bottom of the screen and paste it into the config file. That’s it! With the parameters set and the rest of the fields in the master config filled out, you’re ready to run the main module.
Usage
python InfernoAuger.py –m Main –f <MASTER_CONFIG_LOC> <create/update>
Example to create a new assessment:
python InfernoAuger.py –m Main –f master_build.csv create
Running the above command will display the following:
You will need to provide two pieces of information: an asset group to assign to the assessment (only groups are currently supported) and a name for the assessment. An example would look like the following:
As soon as this information is entered, a progress bar will display showing the status of the assessment being created. Once that completes, you now have a fully working and scheduled assessment!
To update a currently existing assessment, the syntax is very similar:
python InfernoAuger.py –m Main –f update_config.csv update
Running the above command will re-direct to a screen where you can select the assessment you want to update. Example:
This manual intervention can also be prevented by providing the –a switch in the above command followed by the GUID of the assessment within FireDrill you want to update. After this completes, the scenario configurations in update_config.csv will now be added to that assessment!
Detection
Setup
In order to utilize the “Detection” module within InfernoAuger, the integration plugin for Splunk must be configured within FireDrill. Likewise, in order for InfernoAuger to properly parse the vendor technologies that detect attacks, the “Product” column in FireDrill for each configured sourcetype needs to be formatted as “<PRODUCT>-<SOURCETYPE>.” Here are some examples for our configuration for multiple Palo Alto firewall sourcetypes:
Likewise, to promote good security, the credentials for the Splunk connection are stored encrypted in a file. The “setup.py” script used to install the program can be used to encrypt your credentials. Please see the README.md for more information on this.
Once these three items are complete, the only other setup needed is to make sure that the InfernoAuger config.yml file is up-to-date with your specific Splunk instance’s information (hostname, port, etc).
Usage
python InfernoAuger.py –m Detection
By default, the above command will pull back detection results from the assessment specified in the config.yml file. If you want to override this behavior, you can provide the –a switch and the GUID of the assessment you want to push to Splunk.
Running this will produce a screen similar to the following:
Once this is completed, you should be able to see all detection results from that particular assessment in the lookup file specified within Splunk. Here’s an example of what the lookup file will look like, denoting the test, scenario, detection results, product, etc:
Status
Setup
No additional setup is required for the “Status” module.
Usage
python InfernoAuger.py –m Status <scheduled/ondemand>
Similar to Main’s Update function, this will display a list of assessments to check the status on, unless the –a switch is specified.
Since there are two different types of assessment runs (scheduled or on-demand), you need to specify the assessment run type via the command line to tell InfernoAuger which latest run you want to check.
To check the results from the latest scheduled test:
python InfernoAuger.py –m Status scheduled
This command will produce something similar to the following:
This screen displays how many scenarios are still running as part of the assessment, whether or not it is still in progress, and the current state of passed/failed/errored in relation to prevention results. If the assessment is still in progress, the above display will automatically refresh itself every two minutes until completion. This way, you can run the status command and watch it progress to completion.
To check the results from the latest on-demand test, the syntax is very similar:
python InfernoAuger.py –m Status ondemand
This will display the same chart as the above scheduled one.
Scenarios
Setup
No additional setup is required for the “Scenarios” module.
Usage
python InfernoAuger.py –m Scenarios
When this command is run, a CSV file will be created in “data/scenarios/” called “FireDrillScenarios.csv.” This file will contain all of the MITRE ATT&CK tagged scenarios within the FireDrill platform. It contains the following information about each scenario: name, scenario ID, MITRE ID, operating system platforms, and parameters. If a scenario is tagged with multiple MITRE IDs, the program creates a new line for each of those IDs.
Here is an example of what the script will output:
Update
Setup
No setup is inherently required for the “Update” module, but there are a few things to note. The module depends on two files in the “data/update” folder called “currentscenarios.txt” and “results.txt.” These files contain information to determine what scenarios were pulled back the last time the module was run. If they do not exist in this location, the module will create them and pull back all MITRE ATT&CK scenarios as if they were new. So while nothing is required to set up for this module, it is important to note that the first time this runs (or if the files disappear), it will pull back everything as if it was new.
Usage
python InfernoAuger.py –m Update
The above will check to see how many scenarios are currently available in FireDrill and compare it with the number in the “results.txt” file mentioned above. If they are the same, the module will exit out similar to the following:
If it finds new scenarios, it will look like the following:
After this completes, two things will happen:
- The user specified in the config.yml will receive an e-mail containing the newly found scenarios. The e-mail will look similar to the following:
- A file will be created in “data/update” called “newscenarios.csv” which will have the new scenarios formatted to be ingested with the “Main” module (with the exception of the JSON parameters which will need to be populated). An example of the CSV looks like the following:
This has been a high-level overview of InfernoAuger and its’ various modules. The code, along with more information in its’ README.md, can be found on our GitHub page here. If you have any questions related to the tool or discover any bugs or issues, please feel free to report them on our GitHub page. Thank you for reading and we sincerely hope this helps you out with all of your AttackIQ FireDrill automation needs!