1. There is no problem with the local test, and there is a problem that you cannot connect to the formal environment when you go online. 2. Now install all the hadoop clusters on one machine and rule out the network reasons. ...
1. In the hadoop environment built, datanode uses private network ip (172.16.1.142172.16.1.148), and only 142nodes activate external network ip . 2. The data on the HDFS is obtained remotely through Java, and the datanode address returned to the Java p...
first of all, to introduce my situation, I have a physical machine as a master node, which is referred to as master later. There are also two additional servers, later known as node1 and node2. Where the docker node slave1-slave10, is configured on node...
I have a batch of data (10 billion) as follows, ID FROM TO 1 A B 2 A C 3 B A 4 C A Delete duplicate two-way relational data as follows ID FROM TO 1 A B 2 A C 1. Because the amount of data is too large, bloomfilter is no...
the write orc file code is: public static <T> void write(Class cls, List<T> datas,Properties props) throws Exception{ String path = props.getProperty("localTempFile"); JobConf conf = new JobConf(); FileSystem fs = F...
problem description hive is stuck or has no speed while executing mapreduce read the log hadoop resourcemanager reported an error 2018-07-17 19br 17br the environmental background of the problems and what methods you have tried set up a hadoop c...
< H1 > hadoop startup error < H1 > configuration files are all written and then started [root@master logs]-sharp start-dfs.sh Starting namenodes on [master] Last login: Wed Jul 4 16:30:19 CST 2018 on pts 0 bash v3.2+ is required. Sorry. Starting da...
< H2 > Business scenario < H2 > A large number of json files need to be read and re-parsed and imported into elasticsearch . Json files are saved in different date folders. The size of a single folder is about 80g. The number of json files under the ...
this is the article I refer to. The program that runs according to this article https: blog.csdn.net wangato., as shown in the picture, always reports the error in the first sentence. I Baidu has the reason, but has not been able to solve it. 1. Py...
introduce the virtual machine environment, which is in the spark cluster. The virtual machine is the master node. After the virtual machine is restarted abnormally, the vnc of other nodes can log in normally, which should have nothing to do with the spa...
main server jps shows DataNodeDataNode jps: dead nodes is displayed as 0 ...
in the hadoop environment built by the company, datanode uses the private network ip,. Only the master node has enabled the external network ip, now the data on the HDFS is remotely obtained through Java, and the datanode address returned by master to t...
use a simple java client to send a message after simulating the configuration of a kafka server, but without message storage, you can only see the error log all the time (the message is really not stored) kafka.common.KafkaException: Wrong request type...
when running hadoop streaming, an error is reported. The exception content is as follows: Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 126 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads...
1. First of all, the zeppelin has been started successfully on the terminal 2. Secondly, the port number of the configuration information is not occupied is 8080 3, but the browser enters http: localhost:8080 , but it cannot be accessed. The error h...
problem description dynamic nesting routing of nuxt.js does not refresh data 127.0.0.1 cate 2 switching to 127.0.0.1 tag 21 will re-render (cate and tag are dynamic routes) but 127.0.0.1 cate 2 switching to 127.0.0.1 cate 21 routing change page wil...
In most of the examples given by echart, one axis data is fixed, the other axis data is set dynamically, or data is processed through ecStat.regression. wants to make a simple line chart, but both x-axis and y-axis data need to be handled flexibly. ...
recently, I was learning to use create-react-app scaffolding in antd,. According to the configuration of less, on the Internet, but after start, ant reported an error. webpack.configi.dev configuration Code { test: .(css|less)$ , ...
there are many questions about how to solve cross-domain problems, but I have always had a question, such as the figure below. Find a bunch of them on the Internet to explain cross-domain problems web http: store.company.com dir ... according ...
I looked at the code under the on-screen comment and used transition and 3D acceleration, but some doubts are as follows: 1. I don t quite understand how to solve stutter when there are a lot of on-screen comments. 2, transition how the animation ...