The Perplexity about the combination of JAVA and SPARK

the next is a junior JAVA, who currently takes over a task to build a data processing system. Take a supermarket chain as an example. The owner of this supermarket uploads the receipt file through our system, and then we give the supermarket owner a report according to the store name, sales staff and product information on each ticket in the file. for example, store sales ranking, personnel sales ranking and so on. But the file may be so large that it can"t be read into memory at one time. I hope to realize it with the help of Spark. There is a puzzle here, that is, how to use JAVA to automatically hand over this calculation task to Spark and get the result.

Feb.28,2021

submit the java jar package to spark.
enter the results into the hdfs cluster.

Menu