Run Impala on Alluxio

because he is a novice, he cannot create a new tag, so the old name tachyon of Alluxio is used as the tag.

< hr >

an attempt to configure Impala to use Alluxio for file operations failed.
here is the configuration of Impala:

/ etc/impala/conf/core-site.xml ( http://www.alluxio.org/docs/1.

<configuration>
    <property>
      <name>fs.alluxio.impl</name>
      <value>alluxio.hadoop.FileSystem</value>
      <description>The Alluxio FileSystem (Hadoop 1.x and 2.x)</description>
    </property>
    <property>
      <name>fs.AbstractFileSystem.alluxio.impl</name>
      <value>alluxio.hadoop.AlluxioFileSystem</value>
      <description>The Alluxio AbstractFileSystem (Hadoop 2.x)</description>
    </property>
</configuration>

/ etc/impala/conf/hive-site.xml ( http://www.alluxio.org/docs/1.

<property>
   <name>fs.defaultFS</name>
   <value>alluxio://192.168.1.10:19998/</value>
</property>

then start impala (impala-server, impala-catalogd, impala-state-store), and the following appears in the impala log. Impalad failed to start:

...impala-server.cc:282] Currently configured default file system: FileSystem. fs.defaultFS (alluxio://192.168.1.10:19998/) is not supported.
...impala-server.cc:285] Aborting Impala Server startup due to improper configuration. Impalad exiting.

impala version in test environment: 2.10.0-cdh5.13.0 RELEASE, Alluxio version: alluxio-1.8.0-hadoop-2.7

has been searching the Internet for a long time, and there is little useful information. Has anyone ever tried to run Impala on alluxio? Thank you for your help!

Jul.17,2021

I have encountered a similar problem. Have you solved your problem



Impala Service Environment Advanced Configuration Snippet (Safety Valve)
add
AUX_CLASSPATH=/path/to/alluxio-2.0.0-client.jar:$ {AUX_CLASSPATH}

in

CDH

however
can only read and create tables, but cannot insert data
data can only be inserted from hive

Menu