The Twistlock Console is deployed as a part of the gitops. Once deployed the process of setting up daemonsets is currently a manual process. In order to install the following is required:
The Twistlock Console is deployed as a part of the gitops. Once deployed the process of setting up daemonsets is currently a manual process. In order to install the following is required:
### Prerequisites
### Prerequisites
...
@@ -40,6 +39,7 @@ cd twstlock
...
@@ -40,6 +39,7 @@ cd twstlock
```
```
Apply kustomized manifest
Apply kustomized manifest
```
```
kubectl -k ./
kubectl -k ./
```
```
...
@@ -69,7 +69,7 @@ Initially there is no users associated with twistlock console. Go to the extern
...
@@ -69,7 +69,7 @@ Initially there is no users associated with twistlock console. Go to the extern
//Add Administrator
//Add Administrator
if ! curl -k -H 'Content-Type: application/json' -X POST \
if ! curl -k -H 'Content-Type: application/json' -X POST \
https://$TWISTLOCK_EXTERNAL_ROUTE/api/v1/signup; then
<https://$TWISTLOCK_EXTERNAL_ROUTE/api/v1/signup>; then
echo "Error creating Twistlock Console user $TWISTLOCK_CONSOLE_USER"
echo "Error creating Twistlock Console user $TWISTLOCK_CONSOLE_USER"
exit 1
exit 1
...
@@ -187,14 +187,15 @@ fi
...
@@ -187,14 +187,15 @@ fi
## Integrating with SAML
## Integrating with SAML
Integrating Prisma Cloud with SAML consists of setting up your IdP, then configuring Prisma Cloud to integrate with it. for keycloak integration we will use use ADFS as the IdP.
Integrating Prisma Cloud with SAML consists of setting up your IdP, then configuring Prisma Cloud to integrate with it. for keycloak integration we will use use ADFS as the IdP.
The following information is required to setup up Prisma Cloud in Keycloak:
The following information is required to setup up Prisma Cloud in Keycloak:
* The Client ID. THis is the name of the client in keycloak. For SAML you will need the x509 certificate for this Client
* The Client ID. THis is the name of the client in keycloak. For SAML you will need the x509 certificate for this Client
CLIENT_ID=il2_twistlock (or whatever your client name)
CLIENT_ID=il2_twistlock (or whatever your client name)
* X590 certificate from the keycloak client install download To imput this into twistlock by teh web page or by the api, be aware teh pem format is strictly enforced. If you are having issues, test the certificate using opensource tools. Ensure there are 3 lines in the cert; BEGIN/CRLF/Cert/CRLF/END
* X590 certificate from the keycloak client install download To imput this into twistlock by teh web page or by the api, be aware teh pem format is strictly enforced. If you are having issues, test the certificate using opensource tools. Ensure there are 3 lines in the cert; BEGIN/CRLF/Cert/CRLF/END
X_509_CERT="just the certificate"
X_509_CERT="just the certificate"
1. Follow the keycloak instructions under docs/keycloak named `configure-keycloak.md`
1. Follow the keycloak instructions under docs/keycloak named `configure-keycloak.md`
If you only have one pod in the twistlock namespace, the defender did not install properly or at all. Run the steps to install defender again before continuing on.
Here are some examples of a filter for specific containers
twistlock-console
```
{
"query": {
"match_phrase": {
kubernetes.container_name:twistlock-console
}
}
}
```
twistlock-defender
```
{
"query": {
"match_phrase": {
kubernetes.labels.app:twistlock-defender
}
}
}
```
In the KQL field you can text search within a source field such as twistlock-defender
```
kubernetes.labels.app: "twistlock-defender"
```
```
kubernetes.namespace_name:twistlock kubernetes.labels.app:twistlock-defender stream:stdout log: F [31m ERRO 2020-07-14T19:13:25.646 defender.go:331 [0m Failed to initialize GeoLite2 db: open /prisma-static-data/GeoLite2-Country.mmdb: no such file or directory docker.container_id:c0f14b6ba111ef0af3761484dd77a19a5a9f054a4853f757d303be838cad6e6a kubernetes.container_name:twistlock-defender kubernetes.pod_name:twistlock-defender-ds-dtdjv kubernetes.container_image:registry-auth.twistlock.com/tw_bbzc81abegfiqtnruvspkazws2ze0dby/twistlock/defender:defender_20_04_169 kubernetes.container_image_id:registry-
```
```
kubernetes.container_name:twistlock-console
```
```
kubernetes.container_name:twistlock-console kubernetes.namespace_name:twistlock stream:stdout log: F [31m ERRO 2020-07-14T20:01:10.932 kubernetes_profile_resolver.go:38 [0m Failed to fetch Istio resources in 863da02e-15f2-d3da-f74d-0256f77292ad: 1 error occurred: docker.container_id:8303db1aa9e2a694b5db5a454c07127944ee0a4799f3e15f190eaa0eec53ca63 kubernetes.pod_name:twistlock-console-7d77c954d-lnjxp kubernetes.container_image:registry.dsop.io/platform-one/apps/twistlock/console:20.04.169 kubernetes.container_image_id:registry.dsop.io/platform-one/apps/twistlock/console@sha256:db77c64af682161c52da2bbee5fb55f38c0bcd46cacdb4c1148f24d094f18a10 kubernetes.pod_id:c979ebe6-f636-41b8-bfff-eab27fd48692
Integrating Prisma Cloud with SAML consists of setting up your IdP, then configuring Prisma Cloud to integrate with it. for keycloak integration we will use use ADFS as the IdP.
Setting up Prisma Cloud in Keycloak
1. Follow the keycloak instructions under docs/keycloak named `configure-keycloak.md`
2. In Keycloak select the baby-yoda realm
3. On the left column, select "Clients", then new client.
4. Select load file and choose the "client.json" if available. If not, use the 'saml_example.json.md' for the correct settings. The client info can be manually entered if the client isn't available. Go into the configuration and select "Save".
5. In the left column Create a `Client Scope` for twistlock with a SAML Protocol. Return to yout twistlock client and Add the Client scope to the `Your twistlock client` client.
6. Back in the Client configuration, under "Scope" Add the twistlock scope just created.
7. Select the "Installation" tab, the download the connection file in `Mod Auth Mellon format`
_This is needed for the keycloak connection string._
8. Create a user in keycloak for twistlock. Add the user to the IL2 Group.
### The following is required for manual configuration
1. Navigate to the Twistlock URL and create an admin user, then add a license key.
2. Navigate to "Manage" -> "Authentication" in the left navigation bar.
3. Select "SAML" then the enable switch.
4. Open the installation file from keycloak.
a. The Identity Provider SSO is `https://keycloak.fences.dsop.io/auth/realms/your-realm/protocol/saml`
b. The Identity Provider is `https://keycloak.fences.dsop.io/auth/realms/your-realm`
c. The root URL is `https://twistlock.fences.dsop.io`
5. Paste the client certificate token in the x509 area. The certificate must be in pem format and include the header and footer. When completed select "Save".
If this fails, the certificate is not formatted correctly. Copy the cert to a file and test its validity.
Copy the certificate into a vi session and ensure there are three lines:
-----BEGIN CERTIFICATE-----
(certificate from step 7 keycloak install file)
-----END CERTIFICATE-----
*note: when SAML is added, the twistlock console will default to keycloak. If you need to bypass the saml auth process add "#!/login" the the end of the root url.*
6. Create a twistlock user using the same name as in step
7. There should be a "SAML box to select. If this selection is not visible, go to a different tab, then return to users.
@@ -11,6 +11,23 @@ This installation follows the Twistlock documented guidance. Twistlock document
...
@@ -11,6 +11,23 @@ This installation follows the Twistlock documented guidance. Twistlock document
The Twistlock Console is deployed as a part of the gitops. Once deployed the process of setting up daemonsets is currently a manual process. For this installation the following information is needed:
The Twistlock Console is deployed as a part of the gitops. Once deployed the process of setting up daemonsets is currently a manual process. For this installation the following information is needed:
## Table of contents
- Application Overview
- Prerequisites
- Deployment
- Initial Configuration
- Daily Application Use
- Integrations
- Prometheus.md
- Elastic.md
- Keycloak.md
- Troubleshooting Tips
### Applicaiton overview
Twistlock monitors Docker for container deployment and Kubernetes for container orchestration, along with other cloud platforms. Twistlock provides continuous monitoring of containers, in addition to multi-tenancy which allows the user to defend, monitor, and manage multiple projects at once. Twistlock allows for adding firewall rules to individual applications, detecting and blocking anomalies, analyzing events, monitoring memory space, monitoring container compliance, and providing customizable access controls. Continuous Integration provides developers with the status of vulnerabilities found with each build they run, as opposed to running a different tool to see the status of each builds’ CVEs and their severity. ACAS has capability to scan entire servers, however, does not provide the container security Twistlock offers. Container security is a leading issue right now and Twistlock provides the tools necessary to address those.
The application needs a administrator, the license file needs to be installed, then a defender.yaml needs to be generated and deployed. This has been consolidated in a script called twistlock_setup.sh.
The application needs a administrator, the license file needs to be installed, then a defender.yaml needs to be generated and deployed. This has been consolidated in a script called twistlock_setup.sh.
This process requires kubectl to be installed and able to communicate with the DSOP cluster.
This process requires kubectl to be installed and able to communicate with the DSOP cluster.
#### Add an Administrator
#### Add an Administrator
...
@@ -71,6 +94,7 @@ fi
...
@@ -71,6 +94,7 @@ fi
```
```
#### Install the license
#### Install the license
The License can be added directly from the TWISTLOCK_EXTERNAL_ROUTE. When first logging in the admin user will be prompted for a license. The following script will install the license:
The License can be added directly from the TWISTLOCK_EXTERNAL_ROUTE. When first logging in the admin user will be prompted for a license. The following script will install the license:
```
```
...
@@ -80,31 +104,37 @@ if ! curl -k \
...
@@ -80,31 +104,37 @@ if ! curl -k \
-H 'Content-Type: application/json' \
-H 'Content-Type: application/json' \
-X POST \
-X POST \
-d "{\"key\": \"$LICENSE_KEY\"}" \
-d "{\"key\": \"$LICENSE_KEY\"}" \
https://$TWISTLOCK_EXTERNAL_ROUTE/api/v1/settings/license; then
https://$TWISTLOCK_EXTERNAL_ROUTE/api/v1/settings/license; then
echo "Error uploading Twistlock license to console"
echo "Error uploading Twistlock license to console"
exit 1
exit 1
fi
fi
```
```
Notes: curl has some difficulties with special characters. During the initial setup using a password without special characters is recommended. This password needs to be changed to a complex password or the account removed when keycloak is integrated.
#### Install Defender with Twistcli
Notes: curl has some difficulties with special characters. During the initial setup using a password without special characters is recommended. This password needs to be changed to a complex password or the account removed when keycloak is integrated.
#### Install Defender with Twistcli
Defender can be installed from console, script or by command line. The twistlock CLI is provided as a part of the installation. This can be found in the Manage/System/Download. After download ensure the file is made executable.
Defender can be installed from console, script or by command line. The twistlock CLI is provided as a part of the installation. This can be found in the Manage/System/Download. After download ensure the file is made executable.
```
```
$ chmod +x twistcli
chmod +x twistcli
```
```
The "Bearer" token can be found in the twistlock application Manage/Authorization/User Certificates. Alternatively, run the following commands
The "Bearer" token can be found in the twistlock application Manage/Authorization/User Certificates. Alternatively, run the following commands
#####Download the daemonset.yaml. The default Image is set to the Prisma server. The image should be hardened. To pull images from Platform 1. The image URL needs to be changed:
##### Download the daemonset.yaml. The default Image is set to the Prisma server. The image should be hardened. To pull images from Platform 1. The image URL needs to be changed
Note: The Console and Defender must use the same version. If your deploymnet is using 20.04.169 then edit the image accordingly.
Note: The Console and Defender must use the same version. If your deploymnet is using 20.04.169 then edit the image accordingly.
2) Install Defender
2) Install Defender
```
```
kubectl apply -f defender.yaml
kubectl apply -f defender.yaml
```
```
### Install Defender from the Console UI
### Install Defender from the Console UI
The Daemonset generator is located in the Twistlock Console Under Manage -> Defenders -> Deploy -> Daemonset
The Daemonset generator is located in the Twistlock Console Under Manage -> Defenders -> Deploy -> Daemonset
...
@@ -145,144 +181,84 @@ Run Defenders as privileged - On
...
@@ -145,144 +181,84 @@ Run Defenders as privileged - On
Nodes use Container Runtime Interface (CRI), not Docker - On
Nodes use Container Runtime Interface (CRI), not Docker - On
Nodes runs inside containerized environment - Off
Nodes runs inside containerized environment - Off
### Elasticsearch configuration
### Daily Use
Before running the configuration, be sure to have Defender installed.
Follow the steps in either 'Install Defender' or 'Install Defender with Twistcli'
Runtime Defense provides predictive and threat based active protection for running containers. Runtime defense serves the purpose of detecting suspicious activity or the presence of malware within a container. Predictive protection detects anomalous behavior. The Defend - Runtime tab allows for machine learning, where models are built with known good activity so anomalous behavior can be identified. Container runtime protection creates models for images and host runtime protection creates models for processes. Runtime rules are applied to containers and hosts in addition to the autonomous model already created. The rules provide further administrative control to explicitly allow or block an object. The Monitor - Runtime tab displays the container and host models created, along with the container and host audits that list any abnormal behavior happening with a container or image. Using Twistlock Runtime Defense enables security staff to quickly identify and isolate suspicious activity, launch an investigation, and remediate the vulnerability.
create an index pattern for fluentd if not already created
#### Access
```
logstash-*
Twistlock is configured to use SAML and map groups from keycloak to roles within Twistlock.
```
Build filter for twistlock namespace
The roles are as listed:
```
{
- Administrator
"query": {
- Can manage all aspects of Twistlock installation
"match_phrase": {
- Full read-write access to all Twistlock settings and data
"kubernetes.namespace_name": "twistlock"
- Create and update security policies
}
- Create and update access control policies
}
- Create and update the list of users and groups that can access Twistlock
}
- Assign roles to users and groups
```
- Designated for members of Twistlock security administrators team
- Operator
There should be 4 pods in the twistlock namespace
- Can create and update all Twistlock settings
```
- View audit data
kubectl get pods -n twistlock
- Manage the rules that define the policies
NAME READY STATUS RESTARTS AGE
- Designated for members of Twistlock security operations team
- Can install, manage, and remove Defenders from environment
twistlock-defender-ds-dtdjv 1/1 Running 0 5s
- Manage hosts that Twistlock protects - Read-only access to settings and log files
twistlock-defender-ds-rgs7q 1/1 Running 0 5s
- Designated for members of Twistlock DevOps team
```
- Auditor
:warning: **CAUTION**:
- Read-only access to all Twistlock data, settings, and logs
If you only have one pod in the twistlock namespace, the defender did not install properly or at all. Run the steps to install defender again before continuing on.
- Designated for members of Twistlock compliance team
- DevOps User
Here are some examples of a filter for specific containers
- Read-only access to all tabs under Monitor > Vulnerabilities
- Access to Manage > Collections to group resources and organize environment
twistlock-console
- Designated for members of Twistlock DevOps team
```
- Access User
{
- Can run Docker client commands on the hosts that are protected by Defender
"query": {
- Designated for members of the Twistlock engineering team
"match_phrase": {
- CI User
kubernetes.container_name:twistlock-console
- Can only run the plugin
}
- Has no other access to configure Twistlock or view data
}
- Minimal amount of access required to run the plugins
}
```
#### Collections
twistlock-defender
Collections can be used to partition views, which provide a convenient way to browse data from related resources. Collections can also be used to optionally enforce which views specific users and groups can see. They can control access to data on a need-to-know bases or assigned collections. While a single Console manages data from Defenders spread across all hosts, collections let you segment that data into different views based on attributes. Collections are created with pattern matching expressions that are evaluated against attributes such as image name, container name, labels, and namespace. Selecting a collection reduces the scope displayed in Console to just the relevant components.
```
{
#### Assigned Collections
"query": {
"match_phrase": {
When admins create users and groups, they must grant access to at least one collection. By default, users and groups are assigned access to a set called All collections, which contains all objects in the system. All collections is effectively the same as manually creating a collection with a wildcard (*) for every resource type.
kubernetes.labels.app:twistlock-defender
Users with admin or operator roles can always see all resources in the system. They can also see all collections, and utilize them to filter views. When creating users or groups with the admin or operator role, there is no option for assigning collections.
}
}
Collections cannot be deleted as long as they’ve been assigned to users or groups. This enforcement mechanism ensures that users and groups are never left stateless. Click on a specific collection to see who is using them.
}
```
Changes to a user or group’s assigned collections only take affect after users re-login.
Creating Collections Procedure
In the KQL field you can text search within a source field such as twistlock-defender
```
- Manage > Collections
kubernetes.labels.app: "twistlock-defender"
- Add Collection
```
- Create a new collection
```
- Add name to collection
kubernetes.namespace_name:twistlock kubernetes.labels.app:twistlock-defender stream:stdout log: F [31m ERRO 2020-07-14T19:13:25.646 defender.go:331 [0m Failed to initialize GeoLite2 db: open /prisma-static-data/GeoLite2-Country.mmdb: no such file or directory docker.container_id:c0f14b6ba111ef0af3761484dd77a19a5a9f054a4853f757d303be838cad6e6a kubernetes.container_name:twistlock-defender kubernetes.pod_name:twistlock-defender-ds-dtdjv kubernetes.container_image:registry-auth.twistlock.com/tw_bbzc81abegfiqtnruvspkazws2ze0dby/twistlock/defender:defender_20_04_169 kubernetes.container_image_id:registry-
- Specify a filter
```
- Save
```
Note: The collection selects all images with specified image filter in the specified namespace, based on what you choose as your filters. You cannot have collections that specify both containers and images. A wildcard must be in one of the fields, of the collection won’t be applied correctly. If you want to create collections that apply to both a container and an image, then this must be done through two separate collections. Filtering on both collections at the same time will yield the desired result.
kubernetes.container_name:twistlock-console
```
#### Assigning Collection Procedure
```
kubernetes.container_name:twistlock-console kubernetes.namespace_name:twistlock stream:stdout log: F [31m ERRO 2020-07-14T20:01:10.932 kubernetes_profile_resolver.go:38 [0m Failed to fetch Istio resources in 863da02e-15f2-d3da-f74d-0256f77292ad: 1 error occurred: docker.container_id:8303db1aa9e2a694b5db5a454c07127944ee0a4799f3e15f190eaa0eec53ca63 kubernetes.pod_name:twistlock-console-7d77c954d-lnjxp kubernetes.container_image:registry.dsop.io/platform-one/apps/twistlock/console:20.04.169 kubernetes.container_image_id:registry.dsop.io/platform-one/apps/twistlock/console@sha256:db77c64af682161c52da2bbee5fb55f38c0bcd46cacdb4c1148f24d094f18a10 kubernetes.pod_id:c979ebe6-f636-41b8-bfff-eab27fd48692
- Ensure one or more collections are created
```
- Manage > Authentication > {Users | Groups}
- Add users or Add group
# Monitoring
- Selected Auditor or DevOps User role
- Within permissions, select one or more collections
## Prometheus Monitoring
Note: If left unspecified, default is All Collections
Twistlock Prometheus metrics collection is implemented following the documentation:
The Collections column shows to which collection a resource belongs. The color assigned to a collection distinguishes objects that belong to specific collections. This is useful when multiple collections are displayed simultaneously. Collections can also be assigned arbitrary text tags to make it easier for users to associate other metadata with a collection.
To enable prometheus metrics in twistlock:
```
cd app/monitoring/prometheus
```
```
kubectl apply -k .
```
## Integrating with SAML
Integrating Prisma Cloud with SAML consists of setting up your IdP, then configuring Prisma Cloud to integrate with it. for keycloak integration we will use use ADFS as the IdP.
Setting up Prisma Cloud in Keycloak
1. Follow the keycloak instructions under docs/keycloak named `configure-keycloak.md`
2. In Keycloak select the baby-yoda realm
3. On the left column, select "Clients", then new client.
4. Select load file and choose the "client.json" if available. If not, use the 'saml_example.json.md' for the correct settings. The client info can be manually entered if the client isn't available. Go into the configuration and select "Save".
5. In the left column Create a `Client Scope` for twistlock with a SAML Protocol. Return to yout twistlock client and Add the Client scope to the `Your twistlock client` client.
6. Back in the Client configuration, under "Scope" Add the twistlock scope just created.
7. Select the "Installation" tab, the download the connection file in `Mod Auth Mellon format`
_This is needed for the keycloak connection string._
8. Create a user in keycloak for twistlock. Add the user to the IL2 Group.
### The following is required for manual configuration
1. Navigate to the Twistlock URL and create an admin user, then add a license key.
2. Navigate to "Manage" -> "Authentication" in the left navigation bar.
3. Select "SAML" then the enable switch.
4. Open the installation file from keycloak.
a. The Identity Provider SSO is `https://keycloak.fences.dsop.io/auth/realms/your-realm/protocol/saml`
b. The Identity Provider is `https://keycloak.fences.dsop.io/auth/realms/your-realm`
c. The root URL is `https://twistlock.fences.dsop.io`
5. Paste the client certificate token in the x509 area. The certificate must be in pem format and include the header and footer. When completed select "Save".
If this fails, the certificate is not formatted correctly. Copy the cert to a file and test its validity.
Copy the certificate into a vi session and ensure there are three lines:
-----BEGIN CERTIFICATE-----
(certificate from step 7 keycloak install file)
-----END CERTIFICATE-----
*note: when SAML is added, the twistlock console will default to keycloak. If you need to bypass the saml auth process add "#!/login" the the end of the root url.*
6. Create a twistlock user using the same name as in step
7. There should be a "SAML box to select. If this selection is not visible, go to a different tab, then return to users.
