POCs Prerequisites

This page describe which packages are needed for the CEP POCs, how install and configure them.

Note: For this demo, infrastructure is pretty small and simple, there is absolutely no security or reliability functionnalities configured. Don't use those configurations in production.

Demo has been tested on the latest CentOS 8 version (currently 8.0-1905).

Server-side prerequisites

System

CentOS7/8

yum -y install epel-release
yum update
yum upgrade

vi nano /etc/selinux/config
#Change SELINUX value to
SELINUX=disabled

systemctl stop firewalld
systemctl disable firewalld

reboot

Others

Elastic repo

Download and install Elastic public signing key :

CentOS7/8

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Others

Create a new yum repo file and add the following lines :

CentOS7/8

vi /etc/yum.repos.d/elasticsearch.repo

[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Java

Installation

CentOS7/8

sudo yum install java-1.8.0-openjdk

Elasticsearch

Installation

CentOS7/8

yum install elasticsearch

Configuration

Modify/add the following lines in Elasticsearch configuration file :

sudo vi /etc/elasticsearch/elasticsearch.yml


cluster.name: BOTES
node.name: Glooper
# If you have to change default data path 
# Don't forget to change permissions for data folder
# chown -R elasticsearch:elasticsearch /opt/data/elasticsearch/
path.data: /opt/data/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: localhost
http.port: 9200
cluster.initial_master_nodes: ["$Your_Server_IP"]

Configure system and Elasticsearch for JVM memory usage :

sudo vi /etc/elasticsearch/jvm.options


-Xms2g
# Xmx must but set no more than 50% of total memory and no more than 32Gb
-Xmx2g

CentOS7/8

sudo vi /etc/security/limits.conf

# Add the following lines
elasticsearch   soft  memlock   unlimited
elasticsearch   hard  memlock   unlimited


sudo vi /etc/sysconfig/elasticsearch

# Modify the folloing lines
MAX_OPEN_FILES=65535
MAX_LOCKED_MEMORY=unlimited


sudo vi /usr/lib/systemd/system/elasticsearch.service

# Add the following line
LimitMEMLOCK=infinity

Others

Then start and enable Elasticsearch :

systemctl start elasticsearch
systemctl enable elasticsearch

Logstash

Installation

CentOS7/8

yum install logstash

Configuration

Logstash output configuration for Kafka can be downloaded here : https://botes.s3-us-west-1.amazonaws.com/botes-logstash-configuration/full-config/output-kafka/output-kafka.conf

Logstash input configurations for BOTES JSON files can be downloaded on Logstash Configuration section here: BOTES Prerequisites.

Then start and enable Logstash :

systemctl start logstash
systemctl enable logstash

Kibana

Installation

CentOS7/8

yum install kibana

Configuration

Modify/add the following lines in Kibana configuration file :

sudo vi /etc/kibana/kibana.yml


server.port: 5601
server.host: "$Your_Server_IP"
server.name: "BOTES"
elasticsearch.hosts: ["http://localhost:9200"]
# Node can be slow if in Raspberry Pi for example
elasticsearch.requestTimeout: 300000

Then start and enable Kibana :

systemctl start kibana
systemctl enable kibana

Zookeeper

Installation

CentOS7/8

cd /opt
wget http://apache-mirror.8birdsvideo.com/zookeeper/stable/apache-zookeeper-3.5.5.tar.gz
tar zxf apache-zookeeper-3.5.5.tar.gz
rm -f apache-zookeeper-3.5.5.tar.gz
ln -s /opt/apache-zookeeper-3.5.5/ /opt/zookeeper

mkdir /opt/zookeeper/logs
mkdir /opt/zookeeper/data

sudo useradd zk -m
sudo usermod --shell /bin/bash zk
sudo usermod -aG sudo zk
sudo chown -R zk:zk /opt/zookeeper/logs/
sudo chown -R zk:zk /opt/zookeeper/data/

Configuration

Create and add the following lines to Zookeeper configuration file :

CentOS7/8

vi /opt/zookeeper/conf/server.configuration


tickTime=2000
dataDir=/opt/zookeeper/data
dataLogDir/opt/zookeeper/logs
clientPort=2181
clientPortAddress=localhost

Others

Systemd script

Create and add the following lines to Zookeeper systemd script :

CentOS7/8

[Unit]
Description=Zookeeper Daemon
Documentation=http://zookeeper.apache.org
Requires=network.target
After=network.target


[Service]
Type=forking
WorkingDirectory=/opt/zookeeper
User=zk
Group=zk
ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/server.configuration
ExecStop=/opt/zookeeper/bin/zkServer.sh stop /opt/zookeeper/conf/server.configuration
ExecReload=/opt/zookeeper/bin/zkServer.sh restart /opt/zookeeper/conf/server.configuration
TimeoutSec=30
Restart=on-failure


[Install]
WantedBy=default.target

Others

sudo vi /etc/systemd/system/zookeeper.service


[Unit]
Description=Zookeeper Daemon
Documentation=http://zookeeper.apache.org
Requires=network.target
After=network.target


[Service]
Type=forking
WorkingDirectory=/opt/zookeeper
User=zk
Group=zk
ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/server.configuration
ExecStop=/opt/zookeeper/bin/zkServer.sh stop /opt/zookeeper/conf/server.configuration
ExecReload=/opt/zookeeper/bin/zkServer.sh restart /opt/zookeeper/conf/server.configuration
TimeoutSec=30
Restart=on-failure


[Install]
WantedBy=default.target

Then start and enable Zookeeper :

systemctl start zookeeper
systemctl enable zookeeper

Kafka

Installation

CentOS7/8

cd /opt
wget http://apache-mirror.8birdsvideo.com/kafka/2.3.0/kafka_2.11-2.3.0.tgz
tar zxf kafka_2.11-2.3.0.tgz
rm -f kafka_2.11-2.3.0.tgz
ln -s /opt/kafka_2.11-2.3.0/ /opt/kafka

Others

Configuration

Modify/add the following lines in Kafka configuration file :

CentOS7/8

vi /opt/kafka/config/server.properties


broker.id=0
listeners=PLAINTEXT://localhost:9092
log.retention.hours=24
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000

Others

Systemd script

CentOS7/8

sudo vi /etc/systemd/system/kafka.service


[Unit]
Description=Apache Kafka server (broker)
Documentation=http://kafka.apache.org/documentation.html
Requires=network.target remote-fs.target
After=network.target remote-fs.target zookeeper.service


[Service]
Type=simple
User=zk
Group=zk
Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh


[Install]
WantedBy=multi-user.target

Others

Then start and enable Kafka :

systemctl start kafka
systemctl enable kafka

Maven

Installation

CentOS7/8

cd /opt
wget http://apache.mirrors.ionfish.org/maven/maven-3/3.6.2/binaries/apache-maven-3.6.2-bin.tar.gz
tar xzf apache-maven-3.6.2-bin.tar.gz
rm -f apache-maven-3.6.2-bin.tar.gz
ln -s /opt/apache-maven-3.6.2/ /opt/maven
export M2_HOME=/opt/maven
export PATH=${M2_HOME}/bin:${PATH}

Flink

Installation

CentOS7/8

cd /opt
wget http://apache-mirror.8birdsvideo.com/flink/flink-1.9.0/flink-1.9.0-bin-scala_2.11.tgz
tar zxf flink-1.9.0-bin-scala_2.11.tgz
rm -f flink-1.9.0-bin-scala_2.11.tgz
ln -s /opt/flink-1.9.0/ /opt/flink
export FLINK_HOME=/opt/flink/
export PATH=$PATH:$FLINK_HOME/bin

Client-side prerequisites

The following prerequisites apply to clients. It can be configuration or software to collect logs, configuration to make logs more verbose or configuration to make host vulnerable to specifics CVE for the needs of a POC.

Windows

Sysmon

System Monitor (Sysmon) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time.

Download Sysmon on Microsoft website : https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon

For my tests I use the Sysmon configuration from ion-storm GitHub (Forked from SwiftOnSecurity). Sysmon configuration can be downloaded here : https://github.com/ion-storm/sysmon-config/blob/master/sysmonconfig-export.xml

Extract "Sysmon.zip" folder where you want and copy "sysmonconfig-export.xml" in the folder that has been created. (In my case directly in C:\).

Open a CMD as Administrator and launch the following commands to install Sysmon and apply the configuration :

Windows

cd C:\Sysmon
Sysmon.exe –accepteula –i sysmonconfig-export.xml

Winlogbeat

Installation

Winlogbeat is a lightweight events shipper provided by Elastic and which can be used to send events to Logstash or/and Elasticsearch for example.

Download Winlogbeat on Elastic website : https://www.elastic.co/fr/downloads/beats/winlogbeat

Extract "Sysmon.zip" folder where you want. (In my case directly in C:\)

Open a Powershell as Administrator and launch the following commands to install Winlog beat :

Windows

cd C:\winlogbeat-7.4.1-windows-x86_64
set-executionpolicy unrestricted
.\install-service-winlogbeat.ps1

Then go to "Services" and try to start Winlogbeat to validate installation.

Configuration

To configure Winlogbeat, modify the "winlogbeat.yml" file following your need. It's possible to send logs directly to Logstash and/or Elasticsearch, to add Event logs modules/sources, add processors and so on...

winlogbeat.yml configuration example :

Windows

#======================= Winlogbeat specific options ===========================

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h

  - name: System

  - name: Security
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js

  - name: Microsoft-Windows-Sysmon/Operational
    processors:
      - script:
          lang: javascript
          id: sysmon
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js

  - name: Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational
  
#===================== Elasticsearch template settings ===========================

setup.template.settings:
  index.number_of_shards: 1 
  
#================================ Kibana =========================================

setup.kibana:
  host: "192.168.126.42:5601"
  
#================================ Logstash output ================================

output.logstash:
  hosts: ["192.168.126.42:5044"]
  
#================================ Processors =====================================

processors:
  - add_locale: ~
  - add_host_metadata: ~
  - add_cloud_metadata: ~

With this configuration, Winlogbeat will send logs to Logstash only, include Application, System, Security, Sysmon and TerminalServices-RemoteConnectionManager logs. The processors "add_locale" will also include Timezone in logs.

Linux

Metasploit

Metasploit can be easily installed with the following command :

CentOS7/8

curl https://raw.githubusercontent.com/rapid7/metasploit-omnibus/master/config/templates/metasploit-framework-wrappers/msfupdate.erb > \ 
msfinstall && chmod 755 msfinstall && ./msfinstall

Then to launch the Metasploit console, use :

CentOS7/8

cd /$YourMetasploitFrameworkLocation/bin/
./msfconsole