Blastin and Castin - Part 2

Part 1 Review:

Just a refresher of what we covered in our first part of this series:

• Our phishing problem statement

• Making ESA logs more useful

• Making and using the EmailSearcher to learn about your email traffic 

---

Now that the data is in Splunk and searchable, you can start to pattern phishing behavior. Based on the patterns we were seeing in our environment, we started creating searches to proactively spot campaigns prior to our users reporting them (or clicking and not reporting). We created a Phishing Insight dashboard to display our new searches.

Finding Phishing Emails:

A few of our more active searches are:

• Messages Delivered from Raw IP DNS
  • Searches the summary for messages where the DNS address is a raw IP. We then do a country lookup for priority as well as a filter to only search delivered messages.

  • index=summary_esa  
    | regex email_dns_name="^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$" 
    | iplocation email_dns_name 
    | stats latest(_time) as _time values(Country) as Country values(message_subject) AS message_subject values(attachment) AS attachment values(sender) AS sender values(recipient) AS recipient values(vendor_action) AS vendor_action by email_dns_name
    | fillnull value="" attachment 
    |eval action=if((vendor_action="Message done done") OR (vendor_action LIKE "%released") AND (vendor_action!="Quarantine Quarantined done released"),"Delivered", "Not Delivered")   
    | where action="Delivered" AND Country!="United States"  
    | search NOT [| inputlookup ess_whitelist_Phishing.csv
        | eval current_date=strftime(now(), "%Y-%m-%d") 
        | eval expire_date=strptime(expire_date, "%Y-%m-%d")
        | eval expire_date=strftime(expire_date, "%Y-%m-%d")
        | where expire_date > current_date
        | fields message_subject attachment sender]
    | search NOT [    | inputlookup ess_whitelist_email_spoofers.csv
       | fillnull value=* message_subject sender recipient | eval current_date=strftime(now(), "%Y-%m-%d") 
       | eval expire_date=strptime(expire_date, "%Y-%m-%d")
       | eval expire_date=strftime(expire_date, "%Y-%m-%d")
       | where expire_date > current_date
       | fields sender dns_name message_subject recipient 
     | rename dns_name AS email_dns_name ] 
    | where NOT like(sender, "%bounce%") AND NOT like(sender, "%return%") AND NOT like(sender, "%mail@%") AND NOT like(sender, "%contact@%") AND NOT like(sender, "%info@") 
    | fields - vendor_action 
    | sort -_time
    

• Potential Phish that Contains An Attachment
  • Searches the summary for messages that contain an attachment with the most commonly abused file attachments. We then do logic to calculate a blocked rate based on the patterns we observed in our environment. We noticed that spammers usually have old email address lists so some messages bounce. We use this against them to figure out a blocked rate. We also built in logic to examine the amount of time messages trickle in based on bot behavior or trying to avoid spam filters by slipping messages in over an extended time.

  • index=summary_esa attachment!="" sender!="*@<yourdomain>.com" 
    | eval period_count =if(match(attachment,"(?:[A-Za-z]+(?:\.[A-Za-z]+)+){2,}"),"true","false") 
    | where (attachment_type="doc" OR attachment_type="docm" OR attachment_type="xls" OR attachment_type="xlsm" OR attachment_type="pdf") OR (period_count="true" AND attachment_type!="pdf") 
    | search NOT [
         | inputlookup ess_whitelist_Phishing.csv
         | eval current_date=strftime(now(), "%Y-%m-%d")
         | eval expire_date=strptime(expire_date, "%Y-%m-%d")
         | eval expire_date=strftime(expire_date, "%Y-%m-%d")
         | where expire_date > current_date
         | fields message_subject attachment sender]  
    |eval action=if((vendor_action="Message done done") OR (vendor_action LIKE "%released") AND (vendor_action!="Quarantine Quarantined done released"),"Delivered", "Not Delivered") 
    |stats latest(_time) as _time dc(internal_message_id) as message_count,  count(eval(action="Not Delivered")) as blocked_messages,  count(eval(action="Delivered")) as allowed_messages,  values(attachment) as attachment,  max(time_start) as last_sent,   min(time_start) as first_sent by message_subject  
    |eval difference=last_sent-first_sent  
    |fields - first_sent last_sent  
    |eval blocked_rate=blocked_messages/message_count   
    |where  blocked_rate >=.18 and  message_count >=5 and  difference > 600 
    |eval difference=tostring(difference,"duration")  
    |rename difference as time_span  
    |sort desc message_count, blocked_rate 
    | rename message_subject AS subject | eval desc="Attachment:".attachment." Allowed Count:".allowed_messages 
    | table _time subject, desc, attachment 
    | sort - _time
    

• Spoofed Messages
  • Searches the summary for messages that are delivered with a sender domain of our company's domain but the email DNS name is not from our domain

  • index=summary_esa  
    | regex email_dns_name!="^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$" 
    | fillnull value=* message_subject sender recipient
    |rex field=sender ".*@(?<sender_domain>.*)" 
    |eval action=if((vendor_action="Message done done") OR (vendor_action LIKE "%released") AND (vendor_action!="Quarantine Quarantined done released"),"Delivered", "Not Delivered") 
    | search (sender_domain=*<yourdomain>.com AND email_dns_name!=*<yourdomain>.com AND action="Delivered") 
    | search NOT [
        | inputlookup ess_whitelist_email_spoofers.csv
        | fillnull value=* message_subject sender recipient | eval current_date=strftime(now(), "%Y-%m-%d")
        | eval expire_date=strptime(expire_date, "%Y-%m-%d")
        | eval expire_date=strftime(expire_date, "%Y-%m-%d") 
        | where expire_date > current_date
        | fields sender dns_name message_subject recipient | rename dns_name AS email_dns_name ] 
    | stats latest(_time) as _time dc(recipient) AS recipient_count, values(recipient) AS recipient, values(sender) AS sender, values(email_dns_name) AS email_dns_name, values(vendor_action) AS action by message_subject
    | rename message_subject AS subject 
    | sort - _time
    

• Emails Delivered from Blocked Top Level Domains
  • We created a list of Top Level Domains that we consider threat domains. We then search the summary index for any messages that were successfully delivered from one of these domains.

  • index=summary_esa   
    | rex field=email_dns_name "[@\.](?<domain>\w+)\.(?<tld>\w+)$" 
    |rex field=tld "(?<top_tld>[^\d]*)" 
    | stats latest(_time) as _time values(message_subject) AS message_subject values(vendor_action) AS vendor_action values(attachment) AS attachment values(email_dns_name) AS email_dns_name values(top_tld) AS top_tld by sender
    |eval action=if((vendor_action="Message done done") OR (vendor_action LIKE "%released") AND (vendor_action!="Quarantine Quarantined done released"),"Delivered", "Not Delivered")  
    | where action="Delivered" 
    | search  [      | inputlookup wsa_blocked_domain.csv      | fields top_tld] 
    | sort - _time
    

• Potential Phish from New Sender No Attachment
  • This one is pretty complicated but one of our heavy hitters. We use a scheduled report to populate a csv with sending domains. We time stamp first seen and last seen for each domain. The report automatically updates the last seen value each time a domain is discovered through the search to keep the csv clean and up to date.
  • Report GitHub: https://github.com/security-storm/phishing/blob/master/reports/ESA-Domain-Sender-Lookup.spl
  • Report Schedule: Earliest -20m, Latest -10m, Runs every 10 minutes
  • Now that we have a csv with a list of domains and when those domains were first discovered in our environment, we can flag on brand new domains. We look at all of the domains that are first seen within the last 7 hours to identify new domains being spun up and used for phishing. We then apply similar logic to other searches to identify block rate and duration.

  • index=summary_esa[| inputlookup afi_esa_domain_sender.csv | eval lasthour=relative_time(now(),"-7h") | where first_seen>lasthour| fields sender_domain | eval sender="*@".sender_domain | fields sender] 
    | rex field=sender ".*@(?<sender_domain>.*)"
    | fillnull value="" message_subject attachment 
    | stats latest(time_start) AS last_seen dc(recipient) AS recipient_count values(message_subject) AS message_subject values(attachment) AS attachment values(vendor_action) AS vendor_action earliest(time_start) AS first_seenvalues(sender_domain) as sender_domain values(email_dns_name) as src by sender 
    |eval action=if((vendor_action="Message done done") OR (vendor_action LIKE "%released") AND (vendor_action!="Quarantine Quarantined done released"),"Delivered", "Not Delivered")
    |eval difference=last_seen-first_seen 
    |fields - first_seen 
    |mvcombine attachment
    |search sender!="bounce@*" AND sender!="noreply@*" AND sender!="prvs=*"
    |where recipient_count>3 AND action="Delivered" AND difference > 600 AND attachment=""
    |eval difference=tostring(difference,"duration")
    |rename difference AS time_span message_subject AS subject
    | eval last_delivered=strftime(last_seen, "%m/%d/%Y %I:%M:%S %p")
    | fields - last_seen

Now, before you ask, there are definitely false positives that we find using our searches. We regularly tune the potential phish searches based on changes in phishing patterns, but there are always a few false positives that get triggered on. In order to handle these and keep the noise down, we created whitelist dashboards that our searches look at to find exceptions to the search criteria. We have one specifically for legitimate spoofed emails to handle companies that are allowed to send on behalf of our company as well as a separate whitelist for legitimate email that are flagged as possible phishing campaigns. We have built logic into our searches to bypass results once the expiration date has passed, so items can be added short-term to prevent long-term accidental whitelisting.

The sourcecode for this dashboard can be found at:

https://github.com/security-storm/phishing/blob/master/dashboards/whitelist-phishing.xml

Phishing Signatures:

The next step in our process was to actually signature true positive campaigns so we can gather metrics around phishing activity in our environment. We created a Phishing Signature dashboard that provides a friendly interface to add a row to a lookup table in the background.

Each entry adds a new line to the signature lookup with the pertinent information to be used for metrics as well as visibility into campaigns to help with patterning.

The sourcecode for this dashboard can be found at:

https://github.com/security-storm/phishing/blob/master/dashboards/phishing-signature.xml

Phishing Metrics:

Once data is being populated into the signature lookup file, it can start to be utilized for metrics. We created yet another dashboard just for displaying metrics. This dashboard can also be scheduled into a report, which is something we were asked for all the time.

The dashboard is full of metrics based on those signatured events. We pull in the following metrics as of today, but are adding more regularly:

• Number of phishing-related notables (in Splunk Enterprise Security) that were handled (more on this in our part 4)
• Number of emails dropped by Cisco AMP, which was a control we put in as part of this effort (to be discussed in part 3)
• Number of emails that were attempted as part of all of the signatured campaigns
  • This searches the summary index for each email that was attempted using the criteria entered in the signature dashboard (subject, sender, attachment) and counts the total number of messages that match that specific criteria
• List of confirmed phishing campaigns, which just pulls in the signature lookup file
• Pie chart displaying the classification of the campaign as entered in the signature dashboard
  • Link, attachment, empty "feeler" email, or information request
• Top email subjects that were aborted
• Top email subjects dropped specifically by Cisco AMP
• Count of emails delivered from our top threat domains (.ro, .cn, etc) using a lookup we created
• Count of potential phishing emails from Raw IP DNS by country
• List and count of most targeted email addresses based on a search looking up the recipients of each signatured campaign
  • Many times we work with these individuals directly to provide increased education or even change their email address if it is bad enough

The sourcecode for this dashboard can be found at:

https://github.com/security-storm/phishing/blob/master/dashboards/phishing-metrics.xml

---

Up Next in Part 3:

In our next post we will be discussing what we do once we identify a possible campaign and how we have become proactive to prevent users from clicking.

• Exchange Removal Script

• Sandbox Sample Retrieval

• Enabling Email Controls