hdfs_client()

Description:

Connect to the HDFS file system and return an HDFS client object.

Syntax:

hdfs_client( xmlfile, …;url )

Note:

This external library function connects to the Hadoop server and returns an HDFS client object.

Parameters:

xmlfile

A Hadoop configuration file, like hdfs-site.xml, core-site.xml, mapred-site.xml and yarn-site.xml. Mulitple parameter files are separated by the comma; the parameter can be omitted. Users can do the configuration according to their needs. Support absolute paths only.

url

The URL format is scheme://authority/path. Scheme is the protocol name, which can be an ordinary file or an HDFS file; use an HDFS file name when accessing the HDFS file system and use an ordinary file name when accessing the local file system; authority is the host name; path is a string of the file (or directory) path.

Return value:

An HDFS client object

Example:

 

A

 

1

=hdfs_client("D:/core-site.xml","D:/hdfs-site.xml";"hdfs://192.168.0.76:9000")

Connect to the HDFS; the Hadoop configuration files are saved in the root directory of D drive on the local machine; can do the configuration based on the actual needs

2

=hdfs_client(;"hdfs://192.168.0.8:9000")

Connect to the HDFS with the parameter xmlfile omitted