Failed to execute user defined function(anonfun$concatStr$1: (map<string,string>, string) => string)

    科技2024-03-26  98

    自定义spark UDF函数如下:

    def concatStr: UserDefinedFunction = udf((list: Map[String, String], key: String) => { val param = list.getOrElse(key, "") key.concat(param) })

    今天出现报错信息: org.apache.spark.SparkException: Failed to execute user defined function(anonfun$concatStr$1: (map<string,string>, string) => string)

    原因:不是函数的问题,而是调用函数时,传入的第二个参数不是数值,而是 null 解决:修改第二个参数的赋值逻辑即可!

    Processed: 0.016, SQL: 9