UNCLASSIFIED

You need to sign in or sign up before continuing.
Commit bc1d2b18 authored by Gerard Fulton's avatar Gerard Fulton
Browse files

Merge branch 'one' into 'master'

One

See merge request !1
parents b684e454 d612a6cd
# These three ARGs must be present with default variables
# Default variables will be replaced by build/hardening pipelines, but defaults should lead to a successful build outside of the pipelines
# If your container is not based on either the ubi7/ubi8 DCAR images, then it should be based on an image/s ultimately based on ubi7/ubi8
# Note that you will not be able to pull conatiners from nexus-docker-secure.levelup-dev.io into your local dev machine
ARG BASE_REGISTRY=nexus-docker-secure.levelup-dev.io
ARG BASE_IMAGE=dsop/openjdk-v1.8
ARG PRODUCT_API_VERSION=1
ARG PRODUCT_API_RELEASE=6.0
# FROM statement must reference the base image using the three ARGs established
FROM ${BASE_REGISTRY}/${BASE_IMAGE}
# ENV LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8
# 'LABEL' instructions should include at least the following information and any other helpful details.
LABEL product_api_commit=$API_COMMIT \
source="https://nifi.apache.org/" \
name="nifi" \
maintainer="dev@nifi.apache.org" \
vendor="Apache" \
version=$PRODUCT_API_VERSION \
release=$PRODUCT_API_RELEASE \
summary="Image of Apache Nifi" \
description="This builds a working image of Apache Nifi."
ENV VENDOR=nifi \
PACKAGE_NAME=nifi_dependencies.tgz
# You must have these next three variables set exactly as is if you need to pull any files (i.e. dependencies for your container) from the Nexus repo
# Look at the next section
ARG NEXUS_SERVER=${NEXUS_SERVER}
ARG NEXUS_USERNAME=${NEXUS_USERNAME}
ARG NEXUS_PASSWORD=${NEXUS_PASSWORD}
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates gnupg dirmngr bzip2 unzip xz-utils jq xmlstarlet && rm -rf /var/lib/apt/lists/*
ENV LANG=C.UTF-8
RUN echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"' > /usr/local/bin/docker-java-home && chmod +x /usr/local/bin/docker-java-home
RUN ln -svT "/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)" /docker-java-home
RUN /var/lib/dpkg/info/ca-certificates.postinst configure
ARG UID=1000
ARG GID=1000
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_VERSION_DIR=nifi-${PRODUCT_API_VERSION}.${PRODUCT_API_RELEASE}
ENV NIFI_HOME=${NIFI_BASE_DIR}/${NIFI_VERSION_DIR}
RUN mkdir ${NIFI_BASE_DIR}
RUN groupadd -g ${GID} nifi && useradd -u ${UID} -g ${GID} -m nifi && chown -R nifi:nifi ${NIFI_BASE_DIR}
WORKDIR ${NIFI_BASE_DIR}
RUN curl -k -fu ${NEXUS_USERNAME}:${NEXUS_PASSWORD} https://${NEXUS_SERVER}/repository/dsop/${VENDOR}/${PACKAGE_NAME} -o ${PACKAGE_NAME} \
&& tar -zxvf ${PACKAGE_NAME} -C ${NIFI_BASE_DIR} \
&& mkdir -p ${NIFI_HOME}/conf/templates \
&& chown -R nifi:nifi ${NIFI_BASE_DIR}
USER nifi
EXPOSE 10000/tcp 8080/tcp 8443/tcp
WORKDIR /opt/nifi/nifi-1.6.0
CMD ["../scripts/start.sh"]
# This Dockerfile health check is a requirement. Please add an appropriate health check for your container.
HEALTHCHECK --timeout=15s CMD /opt/nifi/nifi-1.6.0/bin/nifi.sh status
# In order to make this container build outside of the pipeline you need to run the following:
# docker build --build-arg NEXUS_USERNAME=username --build-arg NEXUS_PASSWORD=password --build-arg NEXUS_SERVER=https://server.com -t nifi:1.6.0 .
# Make sure to replace username, password, and server address with the appropriate variables if you have your own repository.
# Otherwise, you might need to test locally by pointing curl to the location of the files on the internet
This diff is collapsed.
Apache NiFi
Copyright 2014-2018 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
This product includes the following work from the Apache Hadoop project under Apache Software License V2:
BoundedByteArrayOutputStream.java adapted to SoftLimitBoundedByteArrayOutputStream.java
This product includes derived works from the Apache Software License V2 library python-evtx (https://github.com/williballenthin/python-evtx)
Copyright 2012, 2013 Willi Ballenthin william.ballenthin@mandiant.com
while at Mandiant http://www.mandiant.com
The derived work is adapted from Evtx/Evtx.py, Evtx/BinaryParser.py, Evtx/Nodes.py, Evtx/Views.py
and can be found in the org.apache.nifi.processors.evtx.parser package.
This includes derived works from the Apache Storm (ASLv2 licensed) project (https://github.com/apache/storm):
Copyright 2015 The Apache Software Foundation
The derived work is adapted from
org/apache/storm/hive/common/HiveWriter.java
org/apache/storm/hive/common/HiveOptions.java
and can be found in the org.apache.nifi.util.hive package
This includes derived works from the Apache Hive (ASLv2 licensed) project (https://github.com/apache/hive):
Copyright 2008-2016 The Apache Software Foundation
The derived work is adapted from
release-1.2.1/ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java
and can be found in the org.apache.hadoop.hive.ql.io.orc package
This includes derived works from the Apache Software License V2 library Jolt (https://github.com/bazaarvoice/jolt)
Copyright 2013-2014 Bazaarvoice, Inc
The derived work is adapted from com.bazaarvoice.jolt.chainr.ChainrBuilder.java, com.bazaarvoice.jolt.chainr.spec.ChainrSpec.java,
com.bazaarvoice.jolt.chainr.spec.ChainrEntry.java and can be found in the org.apache.nifi.processors.standard.util.jolt.TransformFactory.java class.
This includes derived works from Elastic Logstash (https://github.com/elastic/logstash/tree/v1.4.0/) and modified by Anthony Corbacho, and contributors
available under an Apache Software License V2.
Copyright 2009-2013 Jordan Sissel, Pete Fritchman, and contributors.
Copyright 2014 Anthony Corbacho, and contributors.
The derived work consists in modifications from patterns/grok-patterns
and can be found in the file nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/TestExtractGrok/patterns
This includes derived works from Maxmind GeoIP2 Java available under Apache Software License V2. Portions of test code found in
https://github.com/maxmind/GeoIP2-java/blob/v2.2.0/src/test/java/com/maxmind/geoip2/TestTransport.java
Copyright (c) 2013 by MaxMind, Inc.
Are used in unit tests found in nifi/nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/test/java/org/apache/nifi/processors/TestGeoEnrichIP.java
This includes derived works from Apache Calcite available under Apache Software License V2. Portions of code found in
https://github.com/apache/calcite/blob/master/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvProjectTableScanRule.java
and
https://github.com/apache/calcite/blob/master/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvTableScan.java
Copyright 2012-2017 The Apache Software Foundation
The code can be found in nifi-nar-bundles/nifi-standard-nar/nifi-standard-processors/../FlowFileProjectTableScanRule
and nifi-nar-bundles/nifi-standard-nar/nifi-standard-processors/../FlowFileTableScan
# opensource Apache nifi v1.6.0
This is derived work from https://nifi.apache.org/
## Build
`docker build -t image-name Dockerfile --no-cache .`
## Usage
`docker run --name nifi --rm image-name`
## Running
`docker run --name nifi --rm image-name`
## Resources
CPU: 2
RAM: 6GB
DISK: 40GB
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBF5Kx2wBEADSBJfmScGiiuoSkr5LHte6EaH2NiFZr6egNABgxHpA0EmRmZm8
7b6Twhm1bOou5GZFZtBwG6EeH3GCD4f/Xd3m6RMoD5gbWIeVzgqT/uqloRUKxQdH
QSxU8qurXLSHXnRmfpvFP4mpMkPMJUwhROnhV8ogYV/AYEvLRR0BYA+jUdhfLgnK
P8iT9OX55Rg3khUNoO8Z917GYTynZM0h9sy5ghcP19zW0SS/Lg2Rt4VvyQurwYHK
SZT04DaUXmcoLP4cCfMgbVGHbE5bSdZVIzJufobStSYQN86+xrMz8OMSlopa5Oyi
jlM+rJLIR3i6EkpB77GbGEKnSHLYnHLvglxEENjZX3gAyAzmQpOidjSjZ2ji4FJQ
P5Wl6tWk9F4iP1tDz/q3bCE04l4TuDc2hKVAAHRT+goIwygfStXLSTg5g/857lKZ
ojuAGhzeJQO2RYDV2ck+PIZ1FY15dv7TRHT72RAljGHBTJdCseNk51Z4BeqGUmg5
vKWGTqcjgg4A4Be1DqUxQooMbkOcKNV24IYcZqoelIfWrFE4lzhYNZSrpxUgiGqY
SSg8Yl/MjMtb5Y8FlaWIDCw+ZzvtX5MYJ+mW5xNgdJWnN4qkaG+TdYM8AhUoobTF
L69d3YFbyPzeBFkGLW8MMyKOFt3D9+1izhdRRaTHP3EOCxkkAtn88MAsnQARAQAB
tB5qY2F0dXBlcyA8bWFyay5rZW5uZXlAbmdjLmNvbT6JAlQEEwEIAD4WIQRDPap+
C1ig84UYmM2KX+zS9K8QFwUCXkrHbAIbAwUJAeEzgAULCQgHAgYVCgkICwIEFgID
AQIeAQIXgAAKCRCKX+zS9K8QF6RhD/9jsc3Gb6wQIitN/eEVS+DkcaA8sYrHKyJC
JuUUb3lk5TSzE1D5JRZSNz/3rSkjhdpe8ttEEr/MffB9QP6gk6BbEHrS5ID/+t4+
rteVY8jEloZu7W4TWqCti/7ETI09MzGr8HPKLMLlpNSPR/arHfkupKgLmrV7cycI
5XhYQjjxS6UIoGJI2+AOp8XUoneqq+4xRrhSg7o82OlMrwJj0ezolxPa2lcBVT0n
+SsEmhvBj//nXbU66KTfgi/JwGWaeqn1RFlKdbasObrzxUQmVOozbV2KU0C245Ao
c3Fkowg3H39kzjdaARHLQ5l3lgQzzAp9bstwwDdejJMtKtq+Cj239TULhwHgU3AL
IbOULNCI2ilD0Ccrf60EY8U3SmR0Izp8CpHMxkYD1uvRhAlXpFXBdu0UFCtTdl2o
oTrIrUflEEC9sC3UrduhX728sbh91cQUUGhGT3tP9qREm1NUGp1z8z88/k6OKN5D
mUCC3FOMOG//Pu8U7b88h94it5XIJWoaGj+uiBwz6OKmsMSF6MIaN6GXXSaqbGgu
dVq2MqsCNCUvEXsoIook4TkC3yD7MoO7bKlZZX0V8Yaf3XJbaSU1/qnMkAPK8qUE
zcfDSI4zDcvDZKf6aU5rike4tlhK2HRKSxjIZbLd4AIqYfhKusVZ7gCddAiTUOLO
vEzxlZjbCrkCDQReSsdsARAA0zEm7mD6doV0b8rclh1Zm+lV5B/v5WJbJkmZ4DRr
pKFHRZaVvVta80WNxQcPzqK+VRht1RX5wBJnm62knR8vWAHrLA3bgOomuF0MYSNd
l+bHbcwByTzQDmd2C25ecIBIy73Mx22kz4dehh321Q5WxuYtE6yzKdvCH25rUTho
SzqMh6T6DONKuPAC8VHEeK2htXmfskMSGZBXUr4cOUR7FWUe8HV00J+KCMDtYD6b
C/lwypkeZUMor+fqnEqKm+gLwhysfllP+ex/Pt7Y8EYWnNF/dx0kpqb3wqzCeUzN
uDjQScRYvGbAEPSgArMy38FMY+NHfylgjyTYJ0mJBcpYEaOW2wSMuTorPFkQYoGk
/6V0MFHXg9zwawsboYdDaKhVaR8WoFuObXvzuCSp2SkQlsBjlXAAFPE5FLqhNvD0
WBeCDnhmfhzidfOV3PemYxQzfYENgXOphSmtXgjATXMyg8B6yNSPFZoGlu6SHfs+
D2XJ0k0rCiSIn482lg0GWeC5juqgLoxKWunWSfCS2nwdxpa7z1Bur3TIS6C2KmBZ
r/1PSw6THVYOC1rmvSISJWPw0uif0b9FDbeQ3AwHtG4hc2sDB+sWlzlNepHJGJQC
uScKhqsYTlF9t+XtUNAtK+1xaa4oEh9mW++DiihAjCsAsh3JwFlPaD8n7CRDT08T
b7EAEQEAAYkCPAQYAQgAJhYhBEM9qn4LWKDzhRiYzYpf7NL0rxAXBQJeSsdsAhsM
BQkB4TOAAAoJEIpf7NL0rxAXg18QAKSa2GrfPxjxAQjkQ/XuN6SXqtXl/aeh04CJ
I0P3Mox4m5CG9iqJqXzR33K2ozQOryqMwI6184T5DAl4QDJqD6IsKwn3egtESAx9
LJ02vuD4xosxCi1IrI0NZ+yzt3bdYxbLkXJ1jztgg7OfRWDjw961qBp2lalvFajs
yXnXJcenA3JPpWZjyM9xGMsaA35VoHAQVRa4fE3Nbg75aOgYqD60osGDMkfVLo1T
BUrvTjQCJ8YNBcNA4SHsPfQ3cR6kLKXv7T0H0Noy9Bk9/+lowt1TSbEZC4VnuK5C
ZP826r4V1SwRH8mZ5cD8Eh2/8C1/aj5EikHTR3ddBSA5tNTbVdRRrK+paAG6gklL
GctmtfBG1/6bvwRByeYUTZkUbtQA9HySIIDXxRXKn0Uvqu9Uj8Qim7CObF7uXY+s
jRLmii/57FJzkFlZNjuOQxUJ46YqGRo99n+u72CEoHoTFMyR9vsT7HMVmwlAjEFP
k5Nv4R90S8i3pDJi7Iyz+vQkZwx1fr9+TYGcsx6rLYJM5TM2lZujSIPqzXAoVffA
YFIICsoHp+OIOOwJSvN6jMZhZ2DgzKh4yLP8B9w7W75odHtkysZ+FUEPiySOohjY
BKI2ks/b7UGAALMI43S4DTdNGGTK/Ex/N33pdxYJWGv43OwQOThN0VSpuUo6iarg
HcEoL0Sl
=EXyX
-----END PGP PUBLIC KEY BLOCK-----
568b12564109718ea1228ee3045366a22ee870a6a66f6d010fffadbd94547047 nifi_dependencies.tgz
set -ex
### Environment Variables ###
VENDOR=nifi
CONTAINER=nifi
TARBALL=nifi_dependencies.tgz
NEXUS_REPO=https://${NEXUS_SERVER}/repository/dsop/${VENDOR}/${TARBALL}
BUCKET=https://up-iron-bank-dependencies-repo.s3-us-gov-west-1.amazonaws.com/put-folder/${TARBALL}
## Create temporary dir to work from, copying necessary files (i.e GPG/SHA files) to it, and making it the working dir ##
TMPDIR=$(mktemp -d -p /tmp) && cp scripts/*.{key,sig,sha} $TMPDIR && cd $TMPDIR
## Verify tarball downloaded with GPG signature and hash ##
# Download tarball from S3 (or whatever other public repo you choose)
curl ${BUCKET} -O
# Import GPG signature and verify tarball with GPG signature
gpg --import ${VENDOR}.key
gpg --verify ${TARBALL}.sig $TARBALL
# Verify sha256sum hash
sha256sum ${TARBALL}.sha --check --status
## Nexus Repo Upload ##
curl -k -fu ${NEXUS_USERNAME}:${NEXUS_PASSWORD} -T $TARBALL $NEXUS_REPO
## Clean up ##
cd -
rm -rf $TMPDIR
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment