Skip to content
This repository has been archived by the owner on Dec 14, 2024. It is now read-only.

Commit

Permalink
Merge branch 'release/v4.1'
Browse files Browse the repository at this point in the history
  • Loading branch information
btorresgil committed Apr 2, 2014
2 parents aa015ac + 2896b4c commit 395fc4e
Show file tree
Hide file tree
Showing 33 changed files with 8,374 additions and 544 deletions.
130 changes: 28 additions & 102 deletions README.md
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -8,55 +8,43 @@ Field extractions and sample reports,
and dashboards for the Palo Alto
Networks Firewall

#### Version ####
#### Latest Version ####

* Splunk Version: 6.x
* App Version: 4.0.2
* Last Modified: Mar 2013
* App Version: 4.1
* Last Modified: Apr 2013
* Authors:
* Monzy Merza - Splunk, Inc.
* Brian Torres-Gil - Palo Alto Networks

#### Credits ####
#### Version Compatibility ####

Many Thanks to Contributors, Advisors, Testers:

* Joel 'JayKul' Bennett, David Dorsey
* David Hazekamp, Mike Munn, Adam Sealey
* David Markquardt, Gerald Kannapathy
* Will Hayes, Marc Benoit, Jeff Hillon
* Genti Zaimi, Scott Brenner, Steve Brown
Splunk 6.x -- Palo Alto Networks App 4.x
Splunk 5.x -- Palo Alto Networks App 3.x

#### Support ####

Further documentation can be found at:
https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks/wiki

For fastest response to support, setup, help or feedback,
please click the __Ask a Question__ button at http://apps.splunk.com/app/491

For bugs or feature requests, you can also open an issue on github at
https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks/issues

## IMPORTANT ##

This app ONLY works on Splunk 6.x

For Splunk 5.x, use version 3.x of this app.
## Quick Start Guide ##

## Dependencies ##

No dependencies

## Installing ##
Install the app:

- Unpack the tar ball into `$SPLUNK_HOME/etc/apps`
- Restart Splunk

Note: After restart, it can take up to 5 minutes for new data to show up in the dashboards.

## Configuring ##
Note: If upgrading from a previous version, please read the __Upgrade Notes__ below.

### Setup Screen and Custom Commands ###

The first time you run the app from the web ui, you will be presented with a setup screen. The credentials are only needed if you wish to use the `panblock` and `panupdate` custom commands. The WildFire API is only needed if you are a WildFire subscriber and want Splunk to index WildFire analysis reports from the cloud when a malware sample is analyzed. These credentials will be stored in Splunk using encryption the same way other Splunk credentials are stored.
The first time you run the app from the web ui, you will be presented with a setup screen. The credentials are only needed if you wish to use the `pantag`, `panblock`, `panupdate` custom commands. The WildFire API is only needed if you are a WildFire subscriber and want Splunk to index WildFire analysis reports from the cloud when a malware sample is analyzed. These credentials will be stored in Splunk using encryption the same way other Splunk credentials are stored.

If you do not wish to use these extra features, you can enter garbage values.

Expand Down Expand Up @@ -89,98 +77,36 @@ Example: (Palo Alto Networks firewalls default to udp port 514)

#### Configure the Firewall ####

Next, on the Palo Alto Networks firewall or Panorama management center, create a Log Forwarding object to send desired syslogs to the Splunk Server. Refer to the Palo Alto Networks documentation for details on log forwarding. https://live.paloaltonetworks.com/community/documentation

Note: Palo Alto Networks devices have a variety of different logs including traffic, threat, url filtering, malware, etc. This app works with the all the default log types. Customized log types may not work, if they are not defined in the Palo Alto Networks syslog configuration documentation (PANOS-Syslog-Integration-TN-RevM).

## Hints and Tips ##

### Source types ###

As Splunk indexes your Palo Alto Networks firewall data, the app will rename the sourcetypes to pan_threat, pan_traffic, pan_config, and pan_system depending on the logging facility.

Log can be further filtered by type during search by using predefined macros. The following macros are available in the search bar to filter on logs of a specific type.

- pan_traffic
- pan_threat
- pan_url
- pan_file
- pan_data
- pan_wildfire
- pan_wildfire_report
- pan_config
- pan_system

Use these macros in the search bar by surrounding them with back-ticks.

### WildFire Cloud Integration ###
On the Palo Alto Networks firewall or Panorama management center, create a Log Forwarding object to send desired syslogs to the Splunk Server. Refer to the Palo Alto Networks documentation for details on log forwarding. https://live.paloaltonetworks.com/community/documentation

WildFire analysis reports can be retrieved dynamically from the WildFire cloud after each analysis. This retrieval requires a WildFire API Key from https://wildfire.paloaltonetworks.com
Note: It can take up to 5 minutes for new data to show up in the dashboards. Palo Alto Networks devices have a variety of different logs including traffic, threat, url filtering, malware, etc. This app works with the all the default log types. Customized log types may not work, if they are not defined in the Palo Alto Networks syslog configuration documentation (PANOS-Syslog-Integration-TN-RevM).

Malware analysis reports from the WildFire Cloud are dynamically downloaded and indexed when a WildFire log is received from a firewall.
### Upgrade Notes ###

### NetFlow ###
Starting in version 4.1 of this app, all of the dashboards use the Splunk 6 Datamodel feature, which allows for pivot of Palo Alto Networks data and better control and acceleration of summary indexes used by the dashboards. This replaces the TSIDX feature from Splunk 5.

NetFlow graphs and charts are based on NetFlow data produced by Palo Alto Networks devices and converted to syslog messages by 3rd party software - NetFlow Integrator. Download a 30-day free trial of NetFlow Integrator at https://www.netflowlogic.com/downloads
If you are upgrading the app from a pre-4.1 version to 4.1 or higher, then you may delete the TSIDX files that were generated by the previous version of the app. To delete the TSIDX files, look under $SPLUNK_HOME$/var/lib/splunk/tsidxstats/ and remove any directories that start with `pan_`. There could be up to 10 directories.

Steps to configure:
After upgrade to 4.1 or higher, Splunk will backfill the datamodel with historic data up to 1 year old. It may take some time for historic data to show up in the dashboards, but it will be available in the pivot interface and search immediately. The time range for historic data to be available in the dashboards can be adjusted in the datamodel accelerations settings.

- Install NetFlow Integrator on a separate server or together with Splunk Forwarder
- Point Palo Alto Networks device NetFlow settings to NetFlow Integrator server, default port 9995 with PAN-OS Field Types enabled (see [Administrator's Guide](https://live.paloaltonetworks.com/community/documentation/content?filterID=contentstatus[published]~category[administrators-guide]&filterID=contentstatus[published]~objecttype~objecttype[document]&itemView=detail))
- Enable NetFlow in the Splunk for Palo Alto Networks app setup page
- Restart Splunk for the previous change to take effect
- Add NetFlow Integrator output pointing to Splunk UDP port 10514
- Create Splunk UDP data input `sourcetype=flowintegrator`, which receives syslog messages on UDP port 10514, and `index=flowintegrator`.
- Enable NetFlow Integrator Palo Alto Networks Rules (10030 through 10035) and Converter (20093)

If you have any questions, or require any assistance with configuration please contact NetFlow Logic at https://netflowlogic.zendesk.com/home

### High Performance Value Store (HPVS) ###

The app uses the HPVS feature introduced in Splunk 5.0 and 6.0. This feature provides a tremendous performance improvement for dashboards and views. The views and dashboards make use of saved searches that store data on your search head. This means that disk storage on your search head will be consumed as a result of these searches. If you turn off these saved searches, your dashboards will not render. Or dashboard rendering will be really, really slow. Please post a question to answers.splunk.com if you'd like to explore alternatives.

### Lookups ###

Lookups are provided for the threat_id and app field to provide additional information about threats and applications on the network.

### Using the form fields on the dashboards ###

All the dashboards work without any filtering values for the form fields. If you want to filter based on a field you should use asterisks before and after the search terms unless you are absolutely sure of the filter value.

Keep in mind that searches that have longer time ranges may take a little longer to return the results.

### Modifying dashboards ###

Dashboards are built with SimpleXML, so they can be modified using the Splunk GUI. To do this, click the __Edit__ menu in the top right of the dashboard and select __Edit Panels__. You can drag panels to new positions, change the visualization (pie, column, area, etc), and modify the searches. If you modify a dashboard and want to recover the original dashboard, delete the modified dashboard file in `$SPLUNK_HOME/etc/apps/SplunkforPaloAltoNetworks/local/data/ui/views` and restart Splunk.
If you have customized the built-in dashboards of a previous app version, then they will no longer work because the customized dashboards will still use TSIDX. Remove your custom dashboards from the `local` directory of the app to use the new datamodel-based dashboards. You can add your customizations to the new dashboards.

## What's new in this version ##

Version 4.0.2

- Fix: Overview dashboard optimizations
- Fix: Top Applications panel would sometimes show error
- Fix: Traffic dashboard form filter works
Version 4.1

Version 4.0.1
If upgrading from a previous version, please read the __Upgrade Notes__ above.

- Fix: Config dashboard shows all events
- Fix: Better handling of navbar changes

Version 4.0

- Splunk 6 support
- Dashboards converted to Splunk 6 SimpleXML, meaning dashboards can now:
- Print
- Export as pdf
- Produce scheduled reports
- Use pre-populated dropdowns in filters
- Change using SplunkWeb by editing the panels
- Maps converted to Splunk 6 built-in maps (removes dependencies on other apps)
- Updated navbar including icons and colors
- PAN-OS Data model including acceleration
- Data model accelerated dashboards (replaces TSIDX-based dashboards)
- New command: `pantag` - tag IP addresses on the firewall into Dynamic Address Groups
- IP Classification - add metadata to your CIDR blocks, classifying them as internet/external/dmz/datacenter/etc.
- Applipedia change notifications and highlighting - know when Palo Alto Networks releases new application signatures and if those applications are on your network

## Installing from Git ##

This app is available on [Splunk Apps](http://apps.splunk.com/app/491) and [Github](https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks). Optionally, you can clone the github repository to install the app.
From the directory `$SPLUNK_HOME/etc/apps/`, type the following command:

git clone https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks.git SplunkforPaloAltoNetworks

1 change: 1 addition & 0 deletions bin/lib/pan/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__version__ = '0.2.0-current'
176 changes: 176 additions & 0 deletions bin/lib/pan/commit.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
#
# Copyright (c) 2013 Kevin Steves <[email protected]>
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
#

from __future__ import print_function
import sys

valid_part = set([
'device-and-network-excluded',
'policy-and-objects-excluded',
'shared-object-excluded',
'no-vsys',
'vsys',
])

_part_xml = {
'device-and-network-excluded':
'<device-and-network>excluded</device-and-network>',
'policy-and-objects-excluded':
'<policy-and-objects>excluded</policy-and-objects>',
'shared-object-excluded':
'<shared-object>excluded</shared-object>',
'no-vsys':
'<no-vsys></no-vsys>',
'vsys':
'<member>%s</member>',
}


def valid_part(part):
return part in _valid_part


class PanCommit:
def __init__(self,
debug=0,
force=False,
commit_all=False,
merge_with_candidate=False):
self.debug = debug
self._force = force
self._commit_all = commit_all
self._merge_with_candidate = merge_with_candidate
self.partial = set()
self._vsys = set()
self._device = None
self._device_group = None

def force(self):
self._force = True

def commit_all(self):
self._commit_all = True

def merge_with_candidate(self):
self._merge_with_candidate = True

def device_and_network_excluded(self):
part = 'device-and-network-excluded'
self.partial.add(part)

def policy_and_objects_excluded(self):
part = 'policy-and-objects-excluded'
self.partial.add(part)

def shared_object_excluded(self):
part = 'shared-object-excluded'
self.partial.add(part)

def no_vsys(self):
part = 'no-vsys'
self.partial.add(part)

def vsys(self, vsys):
if not self._commit_all:
part = 'vsys'
self.partial.add(part)

if type(vsys) == type(''):
vsys = [vsys]
for name in vsys:
self._vsys.add(name)

def device(self, serial):
self._device = serial

def device_group(self, device_group):
self._device_group = device_group

def cmd(self):
if self._commit_all:
return self.__commit_all()
else:
return self.__commit()

def __commit_all(self):
s = '<commit-all><shared-policy>'

if self._device:
s += '<device>%s</device>' % self._device

if self._device_group:
s += '<device-group>%s</device-group>' % self._device_group

# default when no <merge-with-candidate-cfg/> is 'yes'
# we default to 'no' like the Web UI
merge_xml = '<merge-with-candidate-cfg>%s</merge-with-candidate-cfg>'
if self._merge_with_candidate:
merge = 'yes'
else:
merge = 'no'
s += merge_xml % merge

if self._vsys:
s += '<vsys>%s</vsys>' % self._vsys.pop()

s += '</shared-policy></commit-all>'

if self.debug:
print('commit-all cmd:', s, file=sys.stderr)

return s

def __commit(self):
s = '<commit>'

if self._force:
s += '<force>'

if self.partial:
s += '<partial>'
for part in self.partial:
if part in _part_xml:
if part == 'vsys':
s += '<vsys>'
for name in self._vsys:
xml_vsys = _part_xml[part] % name
s += xml_vsys
s += '</vsys>'
else:
s += _part_xml[part]
if self.partial:
s += '</partial>'

if self._force:
s += '</force>'

s += '</commit>'

if self.debug:
print('commit cmd:', s, file=sys.stderr)

return s

if __name__ == '__main__':
import pan.commit

c = pan.commit.PanCommit()
c.force()
c.device_and_network_excluded()
c.policy_and_objects_excluded()
c.shared_object_excluded()
c.vsys(['vsys4', 'vsys5'])
print('cmd:', c.cmd())
Loading

0 comments on commit 395fc4e

Please sign in to comment.