Apache NiFi 0.2.0 发布，数据处理和分发系统

Apache NiFi 0.2.0 发布了，该项目目前还处于 Apache 基金会的孵化阶段。

Apache NiFi 是一个易于使用、功能强大而且可靠的数据处理和分发系统。 Apache NiFi 是为数据流设计。它支持高度可配置的指示图的数据路由、转换和系统中介逻辑。

架构：

集群管理器：

主界面：

关键特性包括:

• 基于web的用户界面

• 无缝体验设计、控制和监视

• 高度可配置的

• 数据丢失容错和保证交付

• 低延迟和高吞吐量

• 动态优先级

• 流可以在运行时修改

• 背压 Back presure

• 数据来源

• 从始至终跟踪数据流

• 为扩展设计

• 构建自己数据处理器

• 支持快速开发和有效的测试

• 安全

• SSL,SSH,HTTPS加密内容,等等……

• 可插拔的基于角色的验证/授权

主要改进内容包括：

Sub-task

• [ NIFI-626 ] - ExecuteSQL processor for executing arbitrary SQL queries

Bug

• [ NIFI-16 ] - Site-to-site should use canonical hostnames when sending peer list

• [ NIFI-37 ] - If provenance receive/create/fork/join/clone event registered against flowfile not created in session...

• [ NIFI-41 ] - Shoud not be able to change destination of Connection if component has its FlowFiles

• [ NIFI-80 ] - When provenance repo retrieves an event that has attributes > some threshold, truncate the value

• [ NIFI-126 ] - Mock Framework should detect when session.create() called instead of session.create(FlowFile) / session.create(Collection<FlowFile>) without emitting CREATE/RECEIVE Provenance Event

• [ NIFI-129 ] - Root Group Ports show that they are not transmitting even when there are active threads receiving data

• [ NIFI-155 ] - Test Failure: MulticastProtocolListenerTest and ClusterServiceDiscoveryTest

• [ NIFI-230 ] - Volatile Content Repository shows negative values available in 'System Diagnostics'

• [ NIFI-313 ] - When template is deleted, UI still shows "Displaying X of Y" instead of "Displaying X of (Y-1)"

• [ NIFI-334 ] - When WriteAheadFlowFileRepo is restoring FlowFiles, it needs to look at the swapped flow files, Id's before setting the max id for the FlowFile ID.

• [ NIFI-373 ] - If NiFi UI is secure but cluster communication is not, NiFi doesn't start but throws NullPointer instead of nice error message

• [ NIFI-378 ] - MergeContent in Defragment mode will merge fragments without checking index

• [ NIFI-379 ] - MergeContent does not merge when Maximum number of Bins is set to 1

• [ NIFI-427 ] - Too many steps to delete flows

• [ NIFI-472 ] - When running NiFi with the "run.as" property specified in the bootstrap.conf file, the run.as user should own the nifi.pid file

• [ NIFI-482 ] - users are currently able to evaluate a function against the result of an aggregrate function

• [ NIFI-487 ] - JsonPath processors can mark valid JSON as invalid

• [ NIFI-488 ] - Bootstrap not capturing output stream or error stream

• [ NIFI-532 ] - Exception handling in RunNifi

• [ NIFI-545 ] - DataFlowDaoImpl writeDataFlow creates an unused dataflow

• [ NIFI-546 ] - StandardDataFlows constructor declears it throws NullPointerException but does not

• [ NIFI-547 ] - Data Viewer: Better support for Unicode characters

• [ NIFI-564 ] - ReplaceText not working as expected when forming POST requests for InvokeHTTP

• [ NIFI-565 ] - System-Level bulletins generated on nodes do not show in the UI when clustered

• [ NIFI-571 ] - Findbugs issue in StandardControllerServiceProvider

• [ NIFI-578 ] - FlowFileNode does not set clusterNodeIdentifier

• [ NIFI-580 ] - Typo in documentation for EncryptContent

• [ NIFI-582 ] - Inconsistent naming convention of IDs for the same component in NiFi UIs

• [ NIFI-591 ] - CompressContent uses wrong mime type for BZIP2

• [ NIFI-594 ] - TestExtractHL7Attributes writes to System output

• [ NIFI-602 ] - Error on shutdown bin/nifi.sh stop

• [ NIFI-613 ] - ScanContent throws exception on dictionary with blank line

• [ NIFI-625 ] - ReplaceText does not substitute Expression Language if embedded within curly braces

• [ NIFI-630 ] - Process Group ID

• [ NIFI-632 ] - Documentation Generation: Nar ClassLoader

• [ NIFI-633 ] - Documentation Generation: Logger

• [ NIFI-637 ] - Improve Nifi Windows Batch Scripts

• [ NIFI-638 ] - Rest API documentation is not clickable

• [ NIFI-643 ] - SplitJson creates a cycle of provenance events

• [ NIFI-647 ] - Duplicating provenance reporting via session creates unexpected behavior

• [ NIFI-657 ] - MongoDB unit tests fail

• [ NIFI-661 ] - Controller Services tab not showing resons for "Invalid" state in Firefox

• [ NIFI-662 ] - Unit test failures for ExecuteStreamCommand

• [ NIFI-663 ] - TestExecuteStreamCommand tests are not OS safe

• [ NIFI-664 ] - WebUI - Invalid argument to setTimeout when disabling ControllerServices

• [ NIFI-670 ] - Copying and pasting a processor using a controller service generates copies of the controller service

• [ NIFI-672 ] - Documentation: Admin Guide-Controlling Levels of Access example incorrect

• [ NIFI-674 ] - FileSystemRepository should not create new threads in its constructor

• [ NIFI-676 ] - DBCPService throws NPE if using OtherDB

• [ NIFI-677 ] - DBCPService should allow user to specify Connection URL

• [ NIFI-678 ] - DBCPService should not require password

• [ NIFI-682 ] - EvaluateXPath docs missing closing parenthesis

• [ NIFI-683 ] - When connecting to cluster, splash screen remains indefinitely if error sent back

• [ NIFI-691 ] - Default config causes NiFi to stop processing after a while if more than 50% of disk used

• [ NIFI-693 ] - Cannot download template

• [ NIFI-694 ] - If Enabling Controller Service fails, no indication is provided to user

• [ NIFI-695 ] - Cancel Enable/Disable controller service should provide immediate feedback to user

• [ NIFI-699 ] - Remove Enable checkbox from Controller Service Configuration Dialog

• [ NIFI-701 ] - Downloaded templates are missing ControllerServices

• [ NIFI-702 ] - PutFile writes too much data when used with VolatileContentRepository

• [ NIFI-703 ] - When calling Session.get(int), it's possible to not get any FlowFiles even when queue is not empty

• [ NIFI-705 ] - TestHandleHttpRequest#testRequestAddedToService fails in Ubuntu 14.04

• [ NIFI-711 ] - Exception thrown when routing FlowFile to multiple connections

• [ NIFI-717 ] - nifi-bootstrap.log written to directory relative to current working directory

• [ NIFI-718 ] - nifi.sh install does not properly install nifi as a linux service

• [ NIFI-719 ] - For Reporting Tasks, ConfigurationContext should  provide scheduling information

• [ NIFI-720 ] - If Reporting Task fails to start properly and is then stopped, it can continue to run once it is able to start

• [ NIFI-721 ] - Typos in ExtractText

• [ NIFI-722 ] - Reporting Task Run Schedule

• [ NIFI-728 ] - Unit test failure on multi-core/fast system builds: nifi-distributed-cache-server

• [ NIFI-731 ] - If content repo is unable to destroy content as fast as it is generated, nifi performance becomes very erratic

• [ NIFI-733 ] - GetKafka group identifier is ignored

• [ NIFI-734 ] - GetKafka Kafka Timeout property is ignored

• [ NIFI-735 ] - DocGenerator should call @OnShutdown after its done generating the docs

• [ NIFI-738 ] - Do not write conversion error messages to flow file content

• [ NIFI-743 ] - .getSolr-mock-processor and .httpCache-mock-processor files in conf dir

• [ NIFI-745 ] - Disabling Controller Service stuck

• [ NIFI-749 ] - Unit tests fail on Windows for InvokeHTTP

• [ NIFI-753 ] - NPE when reading in provenance data

Improvement

• [ NIFI-75 ] - Mock Framework should fail if session.create() used and no appropriate prov event created...

• [ NIFI-349 ] - NiFi should set namespaceAware flag on document builders

• [ NIFI-413 ] - KafkaPut should support compression option

• [ NIFI-419 ] - SSLContextService should allow configurable TLS version

• [ NIFI-454 ] - Distributed cache server unit tests should use random ports

• [ NIFI-548 ] - Listen UDP should support generation of a flowfile per datagram

• [ NIFI-551 ] - Improve error handling for ConvertJSONToAvro processor

• [ NIFI-553 ] - Remove experimental tag from some processors

• [ NIFI-583 ] - Provide ExecuteStreamCommand option of streaming contents over STDIN of an incoming flowfile

• [ NIFI-600 ] - Add compression support to PutHDFS and GetHDFS

• [ NIFI-606 ] - Add a NiFi Storm Spout

• [ NIFI-618 ] - Allow components to be taken out of a processor group

• [ NIFI-642 ] - Eliminate hardcoded HDFS compression codec classnames.

• [ NIFI-644 ] - Data Viewer: Use mime.type when known

• [ NIFI-645 ] - Set content archive and content viewer on by default

• [ NIFI-654 ] - Update dependency versions

• [ NIFI-679 ] - InvokeHTTP - Add support for basic authentication

• [ NIFI-680 ] - Processor docs don't always need to mention Sensitive properties or EL

• [ NIFI-685 ] - Proxy Support for GetHTTP and PostHTTP processors

• [ NIFI-704 ] - StandardProcessorTestRunner should allow you to wait before calling OnUnScheduled methods

• [ NIFI-707 ] - GetHTTP should set mime.type attribute based on HTTP Content-Type header

• [ NIFI-724 ] - Controller Services and Reporting Tasks should be able to emit bulletins

• [ NIFI-737 ] - Typo in developer guide 'lot message'

• [ NIFI-750 ] - NiFi Storm Spout should allow FlowFile attributes to be put into Tuple values

New Feature

• [ NIFI-322 ] - New Database Connection Pooling Controller Service

• [ NIFI-570 ] - MongoDB Processors

• [ NIFI-601 ] - Use dynamic properties as environment variables in ExecuteStreamCommand

Task

• [ NIFI-584 ] - Remove javadoc author tags

• [ NIFI-656 ] - Release Manager: Release 0.2.0

• [ NIFI-659 ] - rat-plugin needs to be configured to exclude child and parent items