50070端口占用
启动nameNode失败，查看启动日志
/var/log/hadoop/hdfs/hadoop-hdfs-namenode-tmaster.log
显示50070端口被占用
查看端口占用情况：netstat -alnp | grep 50070 被 MySQL占用
查看MySQL端口情况
发现大量 端口被MySQL占用，状态全部为 TIME WAIT
因为此MySQL只用来HDP集群，用到的只有hive和ambari。但是hive已经挂了，怀疑是ambari问题。查看ambari日志，发现是有个SQL没有关闭MySQL连接
21 Feb 2017 19:15:29,519 WARN [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] StatementUtils:48 - Statement close FAILED.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘OPTION SQL_SELECT_LIMIT=DEFAULT’ at line 1
原因：Ambari默认的 mysql jdbc 驱动不支持 5.6以上版本
解决方法：下载新的 mysql jdbc驱动
ln -s mysql-connector-java-5.1.40-bin.jar mysql-connector-java.jar
ambari-server setup –jdbc-db=mysql –jdbc-driver=/usr/share/java/mysql-connector-java.jar
提交jar包运行，报错
java.io.IOException: No FileSystem for scheme: hdfs
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at kafka.MyUtils.processFile(MyUtils.java:34)
at kafka.KafkaToParquetAutoOffset.main(KafkaToParquetAutoOffset.java:132)
conf配置有问题：core-site添加 或者代码添加
| fs.file.impl | org.apache.hadoop.fs.LocalFileSystem |
|---|---|
| fs.hdfs.impl | org.apache.hadoop.hdfs.DistributedFileSystem |
Configuration conf = new Configuration();
conf.set(“fs.hdfs.impl”, org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
conf.set(“fs.file.impl”, org.apache.hadoop.fs.LocalFileSystem.class.getName());