Fake HDFS 实例导入方式

在 EasyOps 平台执行 HDFS 的导入操作，需要导入 core-site.xml 和 hdfs-site.xml 配置文件， ranger_hdfs_repo_name，ranger_hdfs_repo_id 信息， 以及相应依赖服务。具体如下：

1. 导入core-site.xml

调整fs.defaultFS为真正的Alluxio Master地址，目前只支持填一个

调整hadoop.security.auth_to_local为适合当前部署环境的Kerberos Realm

<?xml version="1.0"?>
 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
 <configuration>
   <property>
     <name>fs.alluxio.impl</name>
     <value>alluxio.hadoop.FileSystem</value>
     <description>The Alluxio FileSystem (Hadoop 1.x and 2.x)</description>
   </property>

   <property>
     <name>fs.AbstractFileSystem.alluxio.impl</name>
     <value>alluxio.hadoop.AlluxioFileSystem</value>
     <description>The Alluxio AbstractFileSystem (Hadoop 2.x)</description>
   </property>

   <property>
       <name>fs.alluxio-ft.impl</name>
       <value>alluxio.hadoop.FaultTolerantFileSystem</value>
   </property>

   <property>
       <name>fs.defaultFS</name>
       <value>hdfs://alluxio_hdfs</value>
   </property>

   <property>
     <name>io.compression.codecs</name>
     <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.SnappyCodec</value>
   </property>

   <property>
     <name>io.compression.codec.lzo.class</name>
     <value>com.hadoop.compression.lzo.LzoCodec</value>
   </property>

   <property>
       <name>fs.trash.interval</name>
       <value>1440</value>
   </property>

   <property>
       <name>hadoop.security.authentication</name>
       <value>kerberos</value>
   </property>

   <property>
       <name>hadoop.security.authorization</name>
       <value>true</value>
   </property>

   <property>
     <name>hadoop.security.auth_to_local</name>
     <value>
         RULE:[1:$1@$0](.*@BDMS.163.COM)s/@.*//
         RULE:[2:$1@$0](.*@BDMS.163.COM)s/@.*//
         DEFAULT
     </value>
   </property>
</configuration>

2. 导入hdfs-site.xml

调整dfs.nameservices为Alluxio的集群名字

调整dfs.namenode.http-address.alluxio_hdfs.nn[1|2] 至HttpFS的Web地址

若HttpFs为单节点部署，则根据HttpFs部署节点在哪个节点上调整 dfs.namenode.http-address.alluxio_hdfs.nn1 配置

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>

  <property>
      <name>fs.permissions.umask-mode</name>
      <value>027</value>
  </property>

  <property>
      <name>dfs.nameservices</name>
      <value>alluxio_hdfs</value>
  </property>

  <property>
      <name>dfs.ha.namenodes.alluxio_hdfs</name>
      <value>nn1,nn2</value>
  </property>

  <property>
      <name>dfs.namenode.http-address.alluxio_hdfs.nn1</name>
      <value>node17.local:14000</value>
  </property>

  <property>
      <name>dfs.namenode.http-address.alluxio_hdfs.nn2</name>
      <value>node18.local:14000</value>
  </property>

  <property>
      <name>dfs.domain.socket.path</name>
      <value>/var/run/hadoop-hdfs</value>
  </property>
  </configuration>

3. Ranger 相关配置

repo_id 为Ranger创建的alluxio policy repo id

repo_name 为Ranger创建的alluxio policy repo name

4. 依赖选择和导入拓扑

其他按照依赖选择即可

导入一个假的NN拓扑

- external_type: ambari
  componentList:
    - component: namenode
      hostList: ["node10.local"]
      pid_paths:
        - /var/run/sshd.pid
    - component: client
      hostList: ["node[1-10].local"]

调整NN和Client的节点列表