> For the complete documentation index, see [llms.txt](https://botes.gitbook.io/botes-dataset/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://botes.gitbook.io/botes-dataset/botes-prerequisites.md).
# BOTES Prerequisites
{% hint style="info" %}
Scripts have been tested on the latest CentOS 7 minimal image version (currently 7.7-1908).
{% endhint %}
## Install Python 3.7.X
{% tabs %}
{% tab title="CentOS" %}
```
sudo yum update
sudo yum install gcc openssl-devel bzip2-devel libffi-devel python-devel wget
cd /usr/src/
wget https://www.python.org/ftp/python/3.7.4/Python-3.7.4.tgz
tar xzf Python-3.7.4.tgz
cd Python-3.7.4
./configure --enable-optimizations
make altinstall
# Check Python version to validate installation
python3.7 --version
```
{% endtab %}
{% tab title="Other" %}
{% endtab %}
{% endtabs %}
## Install jq
jq allow easy and fast JSON processing. (More information here : )
{% tabs %}
{% tab title="CentOS" %}
```
sudo yum install epel-release
sudo yum install jq
```
{% endtab %}
{% tab title="Other" %}
{% endtab %}
{% endtabs %}
## Install Go
{% tabs %}
{% tab title="CentOS" %}
```
cd /usr/local
wget https://dl.google.com/go/go1.12.7.linux-amd64.tar.gz
tar -zxf go1.12.7.linux-amd64.tar.gz
export GOROOT=/usr/local/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
# Check Go version to validate installation
go version
```
{% endtab %}
{% tab title="Other" %}
{% endtab %}
{% endtabs %}
## Install Python packages
{% tabs %}
{% tab title="CentOS" %}
```
# BOTES Script
python3.7 -m pip install --upgrade pip
python3.7 -m pip install pyyaml
# ECS Script
sudo yum install epel-release
sudo yum install python-pip
python -m pip install --upgrade pip
python -m pip install pyyaml
python -m pip install autopep8==1.4.3
python -m pip install yamllint==1.13.0
```
{% endtab %}
{% tab title="Other" %}
{% endtab %}
{% endtabs %}
## Install Git
{% tabs %}
{% tab title="CentOS" %}
```
sudo yum install git
```
{% endtab %}
{% tab title="Other" %}
{% endtab %}
{% endtabs %}
## Logstash configuration
Download Logstash configuration files from the following links and copy them in `/etc/logstash/conf.d/` folder.
{% hint style="info" %}
Change file paths in configuration according to your environement.
{% endhint %}
{% hint style="info" %}
An output configuration file is also available. The specified Index name match with name or pattern specified in the Elasticsearch Index Mapping template file available here : .
If you want to change Index name in Output file, don't forget to change Index name or pattern in Elasticsearch Index Mapping template file.
{% endhint %}
| Logstash configuration file | Download Link |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| fgt\_event | |
| fgt\_traffic | |
| fgt\_utm | |
| iis | |
| nessus-scan | |
| stream-dhcp | |
| stream-dns | |
| stream-http | |
| stream-icmp | |
| stream-ip | |
| stream-ldap | |
| stream-mapi | |
| stream-sip | |
| stream-smb | |
| stream-snmp | |
| stream-tcp | |
| suricata | |
| winevent-application | |
| winevent-security | |
| winevent-system | |
| winregistry | |
| xmlwineventlog-sysmon | |
| elasticsearch output | |
## Elasticsearch Index Template
Elasticsearch Index Mapping for BOTES Dataset, already modified and ready to be used can be downloaded here :
Upload Elasticsearch Index Mapping template with the following command and your environment :
```
curl -XPUT 'http://"$ES_Instance:9200"/_template/botes' \
-H 'Content-Type: application/json' \
-d@"$path_to_template"/template.json
```
### Date format
Because some dates are in specific format, Elasticsearch Index Template need to be modified. Following formats have been added in Date fields :
```
{
// Modified index pattern to match with BOTES index name
"index_patterns": [
"botes*"
],
...
"botes": {
"file": {
"pctime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss.SSS"
"request": {
"date": {
"type": "date",
"format": "EEE, dd MMM yyyy HH:mm:ss zzz"
"request": {
"date": {
"type": "date",
"format": "EEE, dd MMM yyyy HH:mm:ss zzz"
"date": {
"type": "date",
"format": "EEE, dd MMM yyyy HH:mm:ss zzz"
...
"event": {
"created": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss.SSS z||strict_date_optional_time||epoch_millis"
...
"file": {
"ctime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss.SSS||strict_date_optional_time||epoch_millis"
...
"nessus": {
"scan": {
"end": {
"type": "date",
"format": "E MMM d HH:mm:ss yyyy"
...
"tls": {
"certificate": {
"validity": {
"end": {
"type": "date",
"format": "MMM dd HH:mm:ss yyyy zzz||MMM d HH:mm:ss yyyy zzz"
},
"start": {
"type": "date",
"format": "MMM dd HH:mm:ss yyyy zzz||MMM d HH:mm:ss yyyy zzz"
}
...
"winlog": {
"process": {
"time_utc": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss.SSS"
...
```
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://botes.gitbook.io/botes-dataset/botes-prerequisites.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.