CDH 5.2.0 spark history server issue
5.2.0-1.cdh5.2.0.p0.36 has been released couple of days ago.
It has a small bug with its spark bit, which make unable to start spark history server.
The problem is the default value of Spark History Location (=spark.eventLog.dir) is /user/spark/applicationHistory and it doesn’t append hdfs header by default, hence get recognised as local drive rather than HDFS.
Apply hdfs:///user/spark/applicationHistory instead.