Description:
Read a file stored in Spark database.
Syntax:
spark_read(con,sfile,k:v,...)
Note:
The external library function (See External Library Guide) reads content of a file stored in Spark database and returns result as a table sequence.
Parameter:
|
con |
Local or HDFS connection string |
|
sfile |
File name |
|
k:v |
Parameters that can be omitted. When sfile is a text file separated by “；”, set the pair as ”sep”:”;”; and when the text file is separated by “#”, set the pair as ”sep”:”#” |
Option:
|
@c |
Read content of a file and return result as a cursor |
|
@t |
Read the first row of a text file as field names, which is equivalent to "header":"true"; by default, use the automatically generated field names c0, _c1… |
|
@x |
Close Spark database connection |
Return value:
A table sequence or a cursor
Example:
|
|
A |
|
|
1 |
=spark_open() |
|
|
2 |
=spark_read@t(A1,"D:/people.txt","sep":",") |
Read a semicolon-separated txt file and set reading the first row as field names. |
|
3 |
=spark_read(A1,"D:/people.txt","header":"true","sep":",") |
Same as A2. |
|
4 |
=spark_read(A1,"D:/people.json") |
Read content of people.json. |
|
5 |
=spark_read@c(A1,"D:/people.csv") |
Read content of people.csv and return result as a cursor. |
|
6 |
=A5.fetch() |
|
|
7 |
=spark_read@x(A1,"D:/people.csv") |
Close Spark database connection. |