Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions C2.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,6 @@ be requested via C2 DESCRIBE manifest command.
# c2 agent class -- must be defined to run agent
nifi.c2.agent.class=<your agent class>

# configure SSL Context service for REST Protocol
#nifi.c2.rest.ssl.context.service

# specify encoding strategy for c2 requests (gzip, none)
#nifi.c2.rest.request.encoding=none

Expand Down
11 changes: 10 additions & 1 deletion CONFIGURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1034,7 +1034,16 @@ Here's an example of an alert appender with its available properties:
appender.alert1.batch.size=100 KB
appender.alert1.buffer.limit=1 MB
appender.alert1.level=TRACE
appender.alert1.ssl.context.service=<Name of the SSLContextService>

If you want to use SSL connection for the alert appender, you can set up the certificate configuration properties in the minifi.properties file:

# setup the client certificate and private key PEM files
nifi.security.client.certificate=./conf/client.pem
nifi.security.client.private.key=./conf/client.pem
# setup the client private key passphrase file
nifi.security.client.pass.phrase=./conf/password
# setup the client CA certificate file
nifi.security.client.ca.certificate=./conf/nifi-cert.pem

### Log levels
After the appenders are defined you can set the log level and logging target for each of them. Appenders can be set to log everything on a specific log level, but you can also define some appenders to only write logs from specific namespaces or classes with different log levels. The log level can be set to one of the following values: OFF, TRACE, DEBUG, INFO, WARN, ERROR, CRITICAL. The log levels can be set in the following way:
Expand Down
8 changes: 2 additions & 6 deletions OPS.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,13 +69,9 @@ The executable is stored in the bin directory and is titled minifi-controller. A

### SSL

To use secure connection for the C2 commands, both the MiNiFi C++ agent and the controller have to be started with either of the following configurations.
To use secure connection for the C2 commands, both the MiNiFi C++ agent and the controller have to be started with the following configurations.

To retrieve the SSL configuration properties from the flow config, set the following property to the name of the SSL context service holding the information:

$ controller.ssl.context.service=SSLContextService

Otherwise if you prefer to retrieve the SSL configuration properties from the minifi.properties file set the nifi.remote.input.secure property value to true and configure the security properties in the minifi.properties file.
To retrieve the SSL configuration properties from the minifi.properties file set the nifi.remote.input.secure property value to true and configure the security properties in the minifi.properties file.

$ nifi.remote.input.secure=true
$ nifi.security.client.certificate=/path/to/cert/mycert.crt
Expand Down
1 change: 0 additions & 1 deletion conf/minifi-log.properties.in
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ appender.rolling.max_file_size=20 MB
#appender.alert1.batch.size=100 KB
#appender.alert1.buffer.limit=1 MB
#appender.alert1.level=TRACE
#appender.alert1.ssl.context.service=<Name of the SSLContextService>

# Uncomment if you do not want to include the UUID of the component at the end of log lines
#logger.include.uuid=false
Expand Down
2 changes: 0 additions & 2 deletions conf/minifi.properties.in
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,6 @@ nifi.content.repository.class.name=DatabaseContentRepository
#nifi.c2.rest.path.base=
#nifi.c2.rest.url=
#nifi.c2.rest.url.ack=
#nifi.c2.rest.ssl.context.service=
nifi.c2.root.classes=DeviceInfoNode,AgentInformation,FlowInformation,AssetInformation
## Minimize heartbeat payload size by excluding agent manifest from the heartbeat
nifi.c2.full.heartbeat=false
Expand Down Expand Up @@ -129,7 +128,6 @@ nifi.c2.full.heartbeat=false
#controller.socket.host=localhost
#controller.socket.port=9998
#controller.socket.local.any.interface=false
#controller.ssl.context.service=SSLContextService

## specify the destination of c2 directed assets
nifi.asset.directory=@MINIFI_PATH_ASSET_DIR@
Expand Down
49 changes: 4 additions & 45 deletions controller/MiNiFiController.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -38,55 +38,14 @@

namespace minifi = org::apache::nifi::minifi;

std::shared_ptr<minifi::core::controller::ControllerService> getControllerService(const std::shared_ptr<minifi::Configure> &configuration,
const std::string &service_name) {
std::string nifi_configuration_class_name = "adaptiveconfiguration";

minifi::core::extension::ExtensionManagerImpl::get().initialize(configuration);

configuration->get(minifi::Configure::nifi_configuration_class_name, nifi_configuration_class_name);
auto flow_configuration = minifi::core::createFlowConfiguration(
minifi::core::ConfigurationContext{
.flow_file_repo = nullptr,
.content_repo = nullptr,
.configuration = configuration,
.path = configuration->get(minifi::Configure::nifi_flow_configuration_file)},
nifi_configuration_class_name);

auto root = flow_configuration->getRoot();
if (!root) {
return nullptr;
}
auto controller = root->findControllerService(service_name);
if (!controller) {
return nullptr;
}
return controller->getControllerServiceImplementation();
}

std::shared_ptr<minifi::controllers::SSLContextServiceInterface> getSSLContextService(const std::shared_ptr<minifi::Configure>& configuration) {
std::shared_ptr<minifi::controllers::SSLContextServiceInterface> secure_context;
std::string context_name;
// if the user wishes to use a controller service we need to instantiate the flow
if (configuration->get(minifi::Configure::controller_ssl_context_service, context_name) && !context_name.empty()) {
const auto service = getControllerService(configuration, context_name);
if (nullptr != service) {
secure_context = std::dynamic_pointer_cast<minifi::controllers::SSLContextServiceInterface>(service);
}
if (secure_context == nullptr) {
throw minifi::Exception(minifi::GENERAL_EXCEPTION, "SSL Context was set, but the context name '" + context_name + "' could not be found");
}
}

if (nullptr == secure_context) {
std::string secureStr;
if (configuration->get(minifi::Configure::nifi_remote_input_secure, secureStr) && minifi::utils::string::toBool(secureStr).value_or(false)) {
secure_context = std::make_shared<minifi::controllers::SSLContextService>("ControllerSocketProtocolSSL", configuration);
secure_context->onEnable();
}
} else {
std::string secure_str;
if (configuration->get(minifi::Configure::nifi_remote_input_secure, secure_str) && minifi::utils::string::toBool(secure_str).value_or(false)) {
secure_context = std::make_shared<minifi::controllers::SSLContextService>("ControllerSocketProtocolSSL", configuration);
secure_context->onEnable();
}

return secure_context;
}

Expand Down
75 changes: 2 additions & 73 deletions controller/tests/ControllerTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -212,40 +212,10 @@ class TestControllerSocketReporter : public c2::ControllerSocketReporter {
}
};

class TestControllerServiceProvider : public core::controller::ControllerServiceProviderImpl {
public:
explicit TestControllerServiceProvider(std::shared_ptr<controllers::SSLContextServiceInterface> ssl_context_service)
: core::controller::ControllerServiceProviderImpl("TestControllerServiceProvider"),
ssl_context_service_(std::move(ssl_context_service)) {
}
std::shared_ptr<core::controller::ControllerService> getControllerService(const std::string&) const override {
return is_ssl_ ? ssl_context_service_ : nullptr;
}

std::shared_ptr<core::controller::ControllerServiceNode> createControllerService(const std::string&, const std::string&) override {
return nullptr;
}
void clearControllerServices() override {
}
void enableAllControllerServices() override {
}
void disableAllControllerServices() override {
}

void setSsl() {
is_ssl_ = true;
}

private:
bool is_ssl_{};
std::shared_ptr<controllers::SSLContextServiceInterface> ssl_context_service_;
};

class ControllerTestFixture {
public:
enum class ConnectionType {
UNSECURE,
SSL_FROM_SERVICE_PROVIDER,
SSL_FROM_CONFIGURATION
};

Expand All @@ -259,10 +229,8 @@ class ControllerTestFixture {
configuration_->set(minifi::Configure::nifi_security_client_private_key, (minifi::utils::file::FileUtils::get_executable_dir() / "resources" / "minifi-cpp-flow.key").string());
configuration_->set(minifi::Configure::nifi_security_client_pass_phrase, "abcdefgh");
configuration_->set(minifi::Configure::nifi_security_client_ca_certificate, (minifi::utils::file::FileUtils::get_executable_dir() / "resources" / "root-ca.pem").string());
configuration_->set(minifi::Configure::controller_ssl_context_service, "SSLContextService");
ssl_context_service_ = std::make_shared<controllers::SSLContextService>("SSLContextService", configuration_);
ssl_context_service_->onEnable();
controller_service_provider_ = std::make_unique<TestControllerServiceProvider>(ssl_context_service_);
controller_socket_data_.host = "localhost";
controller_socket_data_.port = 9997;
}
Expand All @@ -271,10 +239,7 @@ class ControllerTestFixture {
if (connection_type_ == ConnectionType::SSL_FROM_CONFIGURATION) {
configuration_->set(minifi::Configure::nifi_remote_input_secure, "true");
}
controller_socket_protocol_ = std::make_unique<minifi::c2::ControllerSocketProtocol>(*controller_service_provider_, *update_sink_, configuration_, reporter);
if (connection_type_ == ConnectionType::SSL_FROM_SERVICE_PROVIDER) {
controller_service_provider_->setSsl();
}
controller_socket_protocol_ = std::make_unique<minifi::c2::ControllerSocketProtocol>(*update_sink_, configuration_, reporter);
controller_socket_protocol_->initialize();
}

Expand All @@ -294,15 +259,11 @@ class ControllerTestFixture {
std::unique_ptr<TestUpdateSink> update_sink_;
std::unique_ptr<minifi::c2::ControllerSocketProtocol> controller_socket_protocol_;
std::shared_ptr<controllers::SSLContextServiceInterface> ssl_context_service_;
std::unique_ptr<TestControllerServiceProvider> controller_service_provider_;

minifi::utils::net::SocketData controller_socket_data_;
};

TEST_CASE_METHOD(ControllerTestFixture, "Test listComponents", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand All @@ -329,10 +290,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "Test listComponents", "[controllerTests
}

TEST_CASE_METHOD(ControllerTestFixture, "TestClear", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand All @@ -356,10 +313,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "TestClear", "[controllerTests]") {
}

TEST_CASE_METHOD(ControllerTestFixture, "TestUpdate", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand All @@ -382,10 +335,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "TestUpdate", "[controllerTests]") {
}

TEST_CASE_METHOD(ControllerTestFixture, "Test connection getters on empty flow", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand Down Expand Up @@ -422,10 +371,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "Test connection getters on empty flow",
}

TEST_CASE_METHOD(ControllerTestFixture, "Test connection getters", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand Down Expand Up @@ -476,10 +421,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "Test connection getters", "[controllerT
}

TEST_CASE_METHOD(ControllerTestFixture, "Test manifest getter", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand All @@ -499,10 +440,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "Test manifest getter", "[controllerTest
}

TEST_CASE_METHOD(ControllerTestFixture, "Test jstack getter", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand All @@ -527,10 +464,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "Test jstack getter", "[controllerTests]
}

TEST_CASE_METHOD(ControllerTestFixture, "Test debug bundle getter", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand Down Expand Up @@ -581,10 +514,6 @@ TEST_CASE_METHOD(ControllerTestFixture, "Debug bundle retrieval fails if target
}

TEST_CASE_METHOD(ControllerTestFixture, "Test flow status getter", "[controllerTests]") {
SECTION("With SSL from service provider") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_SERVICE_PROVIDER);
}

SECTION("With SSL from properties") {
setConnectionType(ControllerTestFixture::ConnectionType::SSL_FROM_CONFIGURATION);
}
Expand Down
31 changes: 25 additions & 6 deletions core-framework/include/http/BaseHTTPClient.h
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,31 @@
#include "utils/ByteArrayCallback.h"
#include "utils/gsl.h"

namespace org::apache::nifi::minifi::http {
enum class HttpRequestMethod {
Get, Post, Put, Patch, Delete, Connect, Head, Options, Trace
};
} // namespace org::apache::nifi::minifi::http

namespace magic_enum::customize {
using HttpRequestMethod = org::apache::nifi::minifi::http::HttpRequestMethod;
template<>
constexpr customize_t enum_name<HttpRequestMethod>(HttpRequestMethod type) noexcept {
switch (type) {
case HttpRequestMethod::Get: return "GET";
case HttpRequestMethod::Post: return "POST";
case HttpRequestMethod::Put: return "PUT";
case HttpRequestMethod::Patch: return "PATCH";
case HttpRequestMethod::Delete: return "DELETE";
case HttpRequestMethod::Connect: return "CONNECT";
case HttpRequestMethod::Head: return "HEAD";
case HttpRequestMethod::Options: return "OPTIONS";
case HttpRequestMethod::Trace: return "TRACE";
}
return invalid_tag;
}
} // namespace magic_enum::customize

namespace org::apache::nifi::minifi::http {

struct HTTPProxy {
Expand Down Expand Up @@ -177,12 +202,6 @@ namespace HTTPRequestResponse {
int seek_callback(void *p, int64_t offset, int);
}

#undef DELETE // this is a macro in winnt.h

enum class HttpRequestMethod {
GET, POST, PUT, PATCH, DELETE, CONNECT, HEAD, OPTIONS, TRACE
};

class BaseHTTPClient {
public:
BaseHTTPClient() = default;
Expand Down
7 changes: 3 additions & 4 deletions core-framework/src/http/BaseHTTPClient.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ std::string get_token(BaseHTTPClient* client, const std::string& username, const

client->setContentType("application/x-www-form-urlencoded");

client->set_request_method(HttpRequestMethod::POST);
client->set_request_method(HttpRequestMethod::Post);

std::string payload = "username=" + username + "&" + "password=" + password;

Expand Down Expand Up @@ -110,7 +110,7 @@ URL::URL(const std::string& url_input) {
std::string::const_iterator end_of_port = std::find_first_of(current_pos, url_input.end(), std::begin(PORT_TERMINATORS), std::end(PORT_TERMINATORS));
const auto port_number = parsePortNumber(std::string{current_pos, end_of_port});
if (port_number) {
port_ = *port_number;
port_ = port_number;
} else {
logger_->log_error("Could not parse the port number in URL '{}'", url_input);
return;
Expand Down Expand Up @@ -226,8 +226,7 @@ size_t HTTPUploadByteArrayInputCallback::getDataChunk(char *data, size_t size) {
if (ptr == nullptr) {
return 0;
}
if (len > size)
len = size;
len = std::min(len, size);
memcpy(data, ptr, len);
pos += len;
seek(pos);
Expand Down
Loading
Loading