I use IntelliJ for Apache Spark debugging, but I get an error every time.

It hasn't been long since I committed to apache park, but I recently had an error when I debacked the park object (park LR) from the example module.
Specifically, the following message appears:

C:\Users\frpgm\IdeaProject\junichi\common\sketch\src\main\java\org\apache\park\util\sketch\Murmur3_x86_32.java
Error: (69,22) java: Symbol not found
  symbols:variable Platform
  Location: Class org.apache.park.util.sketch.Murmur3_x86_32
C:\Users\frpgm\IdeaProject\junichi\common\sketch\src\main\java\org\apache\park\util\Sketch\CountMinSketchImpl.java
Error: (220,51) java: Cannot find symbol
  symbols:variable Platform
  Location: Class org.apache.park.util.sketch.CountMinSketchImpl
Error: (221,51) java: Symbol not found
  symbols:variable Platform
  Location: Class org.apache.park.util.sketch.CountMinSketchImpl
C:\Users\frpgm\IdeaProject\junichi\common\sketch\src\main\java\org\apache\park\util\sketch\BloomFilterImpl.java
Error: (87,51) java: Cannot find symbol
  symbols:variable Platform
  Location: Class org.apache.park.util.sketch.BloomFilterImpl
Error: (88,51) java: Cannot find symbol
  symbols:variable Platform
  Location: Class org.apache.park.util.sketch.BloomFilterImpl
Error: (110,51) java: Cannot find symbol
  symbols:variable Platform
  Location: Class org.apache.park.util.sketch.BloomFilterImpl
Error: (111,51) java: Cannot find symbol
  symbols:variable Platform
  Location: Class org.apache.park.util.sketch.BloomFilterImpl

If possible, I would appreciate it if you could let me know the run/debug configuration settings.

java apache scala spark

2022-09-30 19:13

1 Answers

C:\Users\frpgm\IdeaProject\junichi\common\sketch\src\main\java\org\apache\park\util\sketch\Murmur3_x86_32.java
Error: (69,22) java: Cannot find symbol
symbols:variable Platform
Location: Class org.apache.park.util.sketch.Murmur3_x86_32

The first error is the following line in the source code (although other errors are the same):

https://github.com/apache/spark/blob/master/common/sketch/src/main/java/org/apache/spark/util/sketch/Murmur3_x86_32.java#L69

Simply put, the source code could not be built (compiled), where Platform is the class Platform in the same package as the class Murmur3_x86_32.A class file or jar file containing it was not found for this class.

It seems that the article you requested is posted on Qiita.It seems that there are many other blogs, but it might be faster to start over by referring to one of them.

Debug Apache Spark source files with IntelliJ
https://qiita.com/iyunoriue/items/66a563dce56afd130dab

How to develop Scala+Apache Spark at IntelliJ
https://qiita.com/yk-tanigawa/items/4fc86c7e0e529c3b2fa0

2022-09-30 19:13

If you have any answers or tips

Popular Tags

python x 4647

android x 1593

java x 1494

javascript x 1427

c x 927

c++ x 878

ruby-on-rails x 696

php x 692

python3 x 685

html x 656