1. There is no problem with the local test, and there is a problem that you cannot connect to the formal environment when you go online. 2. Now install all the hadoop clusters on one machine and rule out the network reasons. ...
1. In the hadoop environment built, datanode uses private network ip (172.16.1.142172.16.1.148), and only 142nodes activate external network ip . 2. The data on the HDFS is obtained remotely through Java, and the datanode address returned to the Java p...
first of all, to introduce my situation, I have a physical machine as a master node, which is referred to as master later. There are also two additional servers, later known as node1 and node2. Where the docker node slave1-slave10, is configured on node...
I have a batch of data (10 billion) as follows, ID FROM TO 1 A B 2 A C 3 B A 4 C A Delete duplicate two-way relational data as follows ID FROM TO 1 A B 2 A C 1. Because the amount of data is too large, bloomfilter is no...
the write orc file code is: public static <T> void write(Class cls, List<T> datas,Properties props) throws Exception{ String path = props.getProperty("localTempFile"); JobConf conf = new JobConf(); FileSystem fs = F...
problem description hive is stuck or has no speed while executing mapreduce read the log hadoop resourcemanager reported an error 2018-07-17 19br 17br the environmental background of the problems and what methods you have tried set up a hadoop c...
< H1 > hadoop startup error < H1 > configuration files are all written and then started [root@master logs]-sharp start-dfs.sh Starting namenodes on [master] Last login: Wed Jul 4 16:30:19 CST 2018 on pts 0 bash v3.2+ is required. Sorry. Starting da...
< H2 > Business scenario < H2 > A large number of json files need to be read and re-parsed and imported into elasticsearch . Json files are saved in different date folders. The size of a single folder is about 80g. The number of json files under the ...
this is the article I refer to. The program that runs according to this article https: blog.csdn.net wangato., as shown in the picture, always reports the error in the first sentence. I Baidu has the reason, but has not been able to solve it. 1. Py...
introduce the virtual machine environment, which is in the spark cluster. The virtual machine is the master node. After the virtual machine is restarted abnormally, the vnc of other nodes can log in normally, which should have nothing to do with the spa...
main server jps shows DataNodeDataNode jps: dead nodes is displayed as 0 ...
in the hadoop environment built by the company, datanode uses the private network ip,. Only the master node has enabled the external network ip, now the data on the HDFS is remotely obtained through Java, and the datanode address returned by master to t...
use a simple java client to send a message after simulating the configuration of a kafka server, but without message storage, you can only see the error log all the time (the message is really not stored) kafka.common.KafkaException: Wrong request type...
when running hadoop streaming, an error is reported. The exception content is as follows: Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 126 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads...
1. First of all, the zeppelin has been started successfully on the terminal 2. Secondly, the port number of the configuration information is not occupied is 8080 3, but the browser enters http: localhost:8080 , but it cannot be accessed. The error h...
what can I do to get a dialog box as soon as I open the page? Can this be realized? ...
ask for advice, when I use react-id-swiper, I will not show the effect of swiping like the official website demo, but all div will be squeezed into the first block? reference official website codepen code: https: codepen.io stinaq pen.npm react-id...
how to do this ...
problem description wrote a process class application, and the templates are all reused. Suppose there are three types of template switching. If there are two identical templates in a row, the template cache will not be destroyed . related codes th...
https: www.qichacha.com I climbed with a headless browser, simulated search keywords for dynamic ip 5 seconds for a do not log in, you can start to search keywords, but later can not, I do not know through what anti-climbing? ...