转载

Spring for Apache Hadoop 2.1 正式发布

Spring for Apache Hadoop 提供了 Spring 框架用于创建和运行 Hadoop MapReduce Hive Pig 作业的功能,包括 HDFS HBase 。如果你需要简单的基于 Hadoop 进行作业调度,你可添加 Spring for Apache Hadoop 命名空间到你的 Spring 项目即可快速使用 Hadoop 了,使用该项目无需了解过多 Hadoop 技巧。

Spring for Apache Hadoop 是 Spring Data 的子模块之一。

Spring for Apache Hadoop 2.1 正式发布,经过 6 个月的努力,2.1 终于正式发布啦!

从 Spring for Apache Hadoop 2.1 版本开始,只支持 Hadoop 2.0 APIs,不再向后兼容旧版的 Hadop v1 版本。如果你需要支持 Hadoop v1 版本,请使用 2.0.4 或者 1.1.0 版本。

主要改进内容如下:

支持 Configuration 和 Boot;

Store:

  • Added support for append mode in the HDFS store writers.

  • The Kite SDK dataset support updated to 0.17.0. This means there are some changes to the API. The use of a namespace in addition to the basePath is now mandatory. The DatasetTemplate now also uses ViewCallbacks instead of a partition expression for querying the data.

YARN:

  • Support for container grouping and clustering in Spring YARN, which brings functionality for running multiple container types within a single YARN application.

  • A new REST API for submitted apps and an improved application model with new client side commands and a command line shell.

  • To see examples of these features look at the yarn-store-groups example app or at the Spring XD implementation for running on YARN .

此版本支持:

  • Apache Hadoop 2.4.1 (2.1.0.RELEASE-hadoop24)

  • Apache Hadoop 2.5.2 (2.1.0.RELEASE-hadoop25)

  • Apache Hadoop 2.6.0 (2.1.0.RELEASE)

  • Pivotal HD 2.1 (2.1.0.RELEASE-phd21)

  • Cloudera CDH5 5.3.0 (2.1.0.RELEASE-cdh5)

  • Hortonworks HDP 2.2 (2.1.0.RELEASE-hdp22)

默认的版本是 Apache Hadoop 2.6.0。

未来计划开发的特性:

  • Better Java Configuration support.

  • Add better support for for Hiveserver2 including a batch tasklet.

  • Basic support for a batch tasklet to run Spark apps.

  • Better boot support throughout the different modules.

  • Improved security support (i.e. the YARN Boot CLI interaction, etc).

  • Enhancements to have seamless integration with spring-cloud components (i.e. spring-cloud-cluster).

更多内容请看 发行说明

文章转载自: 开源中国社区 [ http://www.oschina.net

]

本文标题:Spring for Apache Hadoop 2.1 正式发布

本文地址:

http://www.oschina.net/news/59561/spring-for-apache-hadoop-2-1-0

正文到此结束
Loading...