【Hadoop Diary】调试篇<2>
自从上次调通hbase后很久没有碰hadoop了，今日想写一个mapreduce的小程序。于是先运行了下自带的wordcount示例程序，却报错了。
信息如下：
kevin@ubuntu:~/usr/hadoop/hadoop$ ./bin/hadoop jar hadoop-examples-1.2.1.jar wordcount readme.txt output 15/05/11 08:20:04 INFO input.FileInputFormat: Total input paths to process : 1 15/05/11 08:20:04 INFO util.NativeCodeLoader: Loaded the native-hadoop library 15/05/11 08:20:04 WARN snappy.LoadSnappy: Snappy native library not loaded 15/05/11 08:20:05 INFO mapred.JobClient: Running job: job_201505110806_0003 15/05/11 08:20:06 INFO mapred.JobClient: map 0% reduce 0% 15/05/11 08:20:06 INFO mapred.JobClient: Task Id : attempt_201505110806_0003_m_000002_0, Status : FAILED Error initializing attempt_201505110806_0003_m_000002_0: java.io.IOException: Exception reading file:/~/usr/hadoop/hadoop/tmp/mapred/local/ttprivate/taskTracker/kevin/jobcache/job_201505110806_0003/jobToken at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135) at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178) at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226) at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File file:/~/usr/hadoop/hadoop/tmp/mapred/local/ttprivate/taskTracker/kevin/jobcache/job_201505110806_0003/jobToken does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125) at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436) at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129) ... 5 more
错误是无法读取jobToken的文件，由于程序不会出错，推测是上次在配置hbase时修改hadoop的core-site.xml文件出错了。在stackoverflow上有一个同样的问题，有人建议删除 该配置文件中关于缓存文件目录的配置，因为这一部分是会被系统默认创建的，不需要手动配置。即下面这一部分：
1 <!-- In: conf/core-site.xml --> 2 <property> 3 <name>hadoop.tmp.dir</name> 4 <value>/app/hadoop/tmp</value> 5 <description>A base for other temporary directories.</description> 6 </property>
删掉这个问题就解决了。关闭hadoop，重启hadoop。出现新问题 JobTracker is in safe mode：
15/05/11 08:47:40 ERROR security.UserGroupInformation: PriviledgedActionException as:kevin cause:org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.mapred.SafeModeException: JobTracker is in safe mode at org.apache.hadoop.mapred.JobTracker.checkSafeMode(JobTracker.java:5188) at org.apache.hadoop.mapred.JobTracker.getStagingAreaDir(JobTracker.java:3677) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)
碰巧stackoverflow上也有相关的问题，应该是权限的问题，运行以下命令即可：
bin/hadoop dfsadmin -safemode leave
问题均解决，wordcount成功运行。
正文到此结束
热门推荐
相关文章
近期评论
-
文章和留言都翻到11页了 没有OOM
-
-
朋友，翻页到11页，及以后，会出现OOM，无法访问
-
-
搞个gitee的项目
-
-
版本号是多少，你可以下载哪个代码仓库，jdk选1.8 直接跑就行
-
-
-
Loading...
![[HBLOG]公众号](https://www.liuhaihua.cn/img/qrcode_gzh.jpg)
