You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/creating_parsers/index.md
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,13 +3,12 @@
3
3
!!! note "Prerequisites"
4
4
Before reading this section, make sure you are familiar with [Sources](../sources/index.md).
5
5
6
-
This and subsequent sections describe how to create new parsers. SC4S parsers perform operations that would normally be performed during index time, including line-breaking, source and sourcetype setting. You can write your own parser if the parsers available in the SC4S package do not meet your needs or you want to add support for new sourcetype.
6
+
This and subsequent sections describe how to create new parsers. SC4S parsers perform operations that would normally be performed during index time, including line-breaking, source and sourcetype setting. You can write your own parser if the parsers available in the SC4S package do not meet your needs or you want to add support for a new sourcetype.
7
7
8
8
## Before you start
9
9
10
10
* Make sure you have read our [contribution standards](../CONTRIBUTING.md).
11
11
* Obtain a raw log message that you want to parse. If you do not know how to do it, refer to [Obtain raw message events](../troubleshooting/troubleshoot_resources.md#obtain-raw-message-events).
12
-
* Create a new branch in the repository where you will apply your changes.
13
12
* Prepare your testing environment. With Python>=3.11.0:
14
13
15
14
```
@@ -35,15 +34,15 @@ The SC4S parser consists of `application` and `block parser` blocks. The `applic
35
34
If you find a similar parser in SC4S, you can use it as a reference. In the parser, make sure you assign the proper sourcetype, index, vendor, product, and template. The template shows how your message should be parsed before sending them to Splunk.
36
35
37
36
38
-
The application filter will match all messages that start with the string `Carbon Black App Control event:`, and those events will be parsed by `block parser app-syslog-vmware_cb-protect()`. This parser then will route the message to index: `epintel`, set the sourcetype, source, vendor and product fields, and use speciefed template.
37
+
The application filter will match all messages that start with the string `Carbon Black App Control event:`, and those events will be parsed by `block parser app-syslog-vmware_cb-protect()`. This parser then will route the message to index: `epintel`, set the sourcetype, source, vendor and product fields, and use the specified template.
To learn more about creating filters and parse blocks see pages: [Filter Messages](filter_message.md) and [Parse Messages](parse_message.md).
43
42
44
43
### Adding parser to SC4S Lite package
45
44
46
-
For SC4S lite, parsers are grouped into `addons`. Create a folder (if it doesn't already exist) in `package/lite/etc/addons` with the name of vendor. In this folder also create an `addon_metadata.yaml` file with vendor name:
45
+
For SC4S lite, parsers are grouped into `addons`. Create a folder (if it does not already exist) in `package/lite/etc/addons` with the name of vendor. In this folder, also create an `addon_metadata.yaml` file with vendor name:
Copy file name to clipboardExpand all lines: docs/creating_parsers/parse_message.md
+8-2Lines changed: 8 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,7 +39,7 @@ rewrite {
39
39
};
40
40
```
41
41
42
-
**`r_set_splunk_dest_update_v2`** — overrides specific fields already set by `r_set_splunk_dest_default`. It accepts `index`, `source`, `sourcetype`, and `template` options. You can also use the `condition` option for a conditional expression.
42
+
**`r_set_splunk_dest_update_v2`** — overrides specific fields already set by `r_set_splunk_dest_default`. It accepts `index`, `source`, `sourcetype`, `class`, and `template` options. You can also use the `condition` option for a conditional expression.
Copy file name to clipboardExpand all lines: docs/sources/index.md
+9-3Lines changed: 9 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,16 +20,22 @@ New supported sources are added regularly. Please submit an [issue](https://gith
20
20
21
21
Many sources can be self supported. While we encourage sharing new sources via the github project to promote consistency and develop best-practices there is no requirement to engage in the community.
22
22
23
-
Sources that are *compliant* with RFC 5424, RFC 5425, RFC 5426, or RFC 6587 can be onboarded as [simple sources](simple.md).
23
+
* Sources that are compliant with RFC 5424, RFC 5425, RFC 5426, or RFC 6587 can be onboarded as [simple sources](simple.md).
24
+
* Sources compatible with RFC 3164 can also be onboarded as [simple sources](simple.md). Note that incorrect use of the syslog version, or “creative” formats in the timestamp or other fields may prevent this method of onboarding and will require writing custom parsers.
25
+
26
+
Other popular log formats that are supported by SC4S are:
27
+
28
+
* Common Event Format [CEF](base/cef.md), also known as ArcSight format.
29
+
* Log Extended Format [LEEF](base/leef.md).
24
30
25
31
## Common Patterns
26
32
27
33
This section covers the most basic and common patterns for onboarding data with user-made parsers. If you want to read more, see [Create a Parser](../creating_parsers/index.md).
28
34
29
35
### Almost Syslog
30
36
31
-
Sources sending legacy non conformant 3164 like streams can be assisted by the creation of an "Almost Syslog" Parser. In an such a parser the goal is to process the syslog header allowing other parsers
32
-
to correctly parse and handle the event. The following example is take from a currently supported format where the source product used epoch in the time stamp field.
37
+
Sources sending legacy non conformant 3164 like streams can be assisted by the creation of an "Almost Syslog" Parser. In such a parser the goal is to process the syslog header allowing other parsers
38
+
to correctly parse and handle the event. The following example is taken from a currently supported format where the source product used epoch in the timestamp field.
Copy file name to clipboardExpand all lines: docs/sources/vendor/Cisco/cisco_meraki.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
1
## Meraki (MR, MS, MX)
2
2
3
3
## Key facts
4
-
* Cisco Meraki messages are not distinctive, which means that it's impossible to parse the sourcetype based on the log message.
5
-
* Because of the above you should either configure known Cisco Meraki hosts in SC4S, or open unique ports for Cisco Meraki devices.
4
+
* Cisco Meraki messages are not distinctive, which means that it is impossible to parse the sourcetype based on the log message.
5
+
* Because of the above, you should either configure known Cisco Meraki hosts in SC4S, or open unique ports for Cisco Meraki devices.
6
6
*[Splunk Add-on for Cisco Meraki 2.1.0](https://splunkbase.splunk.com/app/5580) does not support syslog. Use [TA-meraki](https://splunkbase.splunk.com/app/3018) instead. `TA-meraki 1.1.5` requires sourcetype `meraki`.
* Run `tcpdump` on the collection interface and display the results in ASCII. You will see events similar to the following buried in the packet contents:
@@ -67,10 +59,10 @@ You can also read the logs using Wireshark from the .pcap file. From Wireshark g
67
59
68
60
* For most other sourcetypes, the `RAWMSG` is not displayed, but can be
69
61
viewed by changing the output template to one of the JSON variants, including t_JSON_3164 or t_JSON_5424, depending on RFC message type. See
70
-
[SC4S metadata configuration](https://splunk-connect-for-syslog.readthedocs.io/en/develop/configuration/#sc4s-metadata-configuration) for
62
+
[SC4S metadata configuration](../configuration.md#sc4s-metadata-configuration) for
71
63
more details.
72
64
73
-
* In order to send `RAWMSG` to Splunk regardless the sourcetype you can also temporarily place the following final filter in the local parser directory:
65
+
* In order to send `RAWMSG` to Splunk regardless of the sourcetype you can also temporarily place the following final filter in the local parser directory:
0 commit comments