Skip to content

Commit ea4234a

Browse files
[doc][connector][hive] Add hive chinese doc (#8169)
1 parent df2fee0 commit ea4234a

File tree

4 files changed

+949
-37
lines changed

4 files changed

+949
-37
lines changed

docs/en/connector-v2/sink/Hive.md

Lines changed: 50 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Write data to Hive.
88

99
:::tip
1010

11-
In order to use this connector, You must ensure your spark/flink cluster already integrated hive. The tested hive version is 2.3.9.
11+
In order to use this connector, You must ensure your spark/flink cluster already integrated hive. The tested hive version is 2.3.9 and 3.1.3 .
1212

1313
If you use SeaTunnel Engine, You need put seatunnel-hadoop3-3.1.4-uber.jar and hive-exec-3.1.3.jar and libfb303-0.9.3.jar in $SEATUNNEL_HOME/lib/ dir.
1414
:::
@@ -204,6 +204,55 @@ Description:
204204
- `kerberos_keytab_path`: The keytab file path for Kerberos authentication.
205205
- `krb5_path`: The path to the `krb5.conf` file used for Kerberos authentication.
206206

207+
Run the case:
208+
209+
```bash
210+
env {
211+
parallelism = 1
212+
job.mode = "BATCH"
213+
}
214+
215+
source {
216+
FakeSource {
217+
schema = {
218+
fields {
219+
pk_id = bigint
220+
name = string
221+
score = int
222+
}
223+
primaryKey {
224+
name = "pk_id"
225+
columnNames = [pk_id]
226+
}
227+
}
228+
rows = [
229+
{
230+
kind = INSERT
231+
fields = [1, "A", 100]
232+
},
233+
{
234+
kind = INSERT
235+
fields = [2, "B", 100]
236+
},
237+
{
238+
kind = INSERT
239+
fields = [3, "C", 100]
240+
}
241+
]
242+
}
243+
}
244+
245+
sink {
246+
Hive {
247+
table_name = "default.test_hive_sink_on_hdfs_with_kerberos"
248+
metastore_uri = "thrift://metastore:9083"
249+
hive_site_path = "/tmp/hive-site.xml"
250+
kerberos_principal = "hive/[email protected]"
251+
kerberos_keytab_path = "/tmp/hive.keytab"
252+
krb5_path = "/tmp/krb5.conf"
253+
}
254+
}
255+
```
207256

208257
## Hive on s3
209258

@@ -418,26 +467,3 @@ sink {
418467
}
419468
}
420469
```
421-
422-
## Changelog
423-
424-
### 2.2.0-beta 2022-09-26
425-
426-
- Add Hive Sink Connector
427-
428-
### 2.3.0-beta 2022-10-20
429-
430-
- [Improve] Hive Sink supports automatic partition repair ([3133](https://github.com/apache/seatunnel/pull/3133))
431-
432-
### 2.3.0 2022-12-30
433-
434-
- [BugFix] Fixed the following bugs that failed to write data to files ([3258](https://github.com/apache/seatunnel/pull/3258))
435-
- When field from upstream is null it will throw NullPointerException
436-
- Sink columns mapping failed
437-
- When restore writer from states getting transaction directly failed
438-
439-
### Next version
440-
441-
- [Improve] Support kerberos authentication ([3840](https://github.com/apache/seatunnel/pull/3840))
442-
- [Improve] Added partition_dir_expression validation logic ([3886](https://github.com/apache/seatunnel/pull/3886))
443-

docs/en/connector-v2/source/Hive.md

Lines changed: 66 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Read data from Hive.
88

99
:::tip
1010

11-
In order to use this connector, You must ensure your spark/flink cluster already integrated hive. The tested hive version is 2.3.9.
11+
In order to use this connector, You must ensure your spark/flink cluster already integrated hive. The tested hive version is 2.3.9 and 3.1.3 .
1212

1313
If you use SeaTunnel Engine, You need put seatunnel-hadoop3-3.1.4-uber.jar and hive-exec-3.1.3.jar and libfb303-0.9.3.jar in $SEATUNNEL_HOME/lib/ dir.
1414
:::
@@ -180,6 +180,71 @@ Description:
180180
- `kerberos_keytab_path`: The keytab file path for Kerberos authentication.
181181
- `krb5_path`: The path to the `krb5.conf` file used for Kerberos authentication.
182182

183+
Run the case:
184+
185+
```bash
186+
env {
187+
parallelism = 1
188+
job.mode = "BATCH"
189+
}
190+
191+
source {
192+
Hive {
193+
table_name = "default.test_hive_sink_on_hdfs_with_kerberos"
194+
metastore_uri = "thrift://metastore:9083"
195+
hive.hadoop.conf-path = "/tmp/hadoop"
196+
result_table_name = hive_source
197+
hive_site_path = "/tmp/hive-site.xml"
198+
kerberos_principal = "hive/[email protected]"
199+
kerberos_keytab_path = "/tmp/hive.keytab"
200+
krb5_path = "/tmp/krb5.conf"
201+
}
202+
}
203+
204+
sink {
205+
Assert {
206+
source_table_name = hive_source
207+
rules {
208+
row_rules = [
209+
{
210+
rule_type = MAX_ROW
211+
rule_value = 3
212+
}
213+
],
214+
field_rules = [
215+
{
216+
field_name = pk_id
217+
field_type = bigint
218+
field_value = [
219+
{
220+
rule_type = NOT_NULL
221+
}
222+
]
223+
},
224+
{
225+
field_name = name
226+
field_type = string
227+
field_value = [
228+
{
229+
rule_type = NOT_NULL
230+
}
231+
]
232+
},
233+
{
234+
field_name = score
235+
field_type = int
236+
field_value = [
237+
{
238+
rule_type = NOT_NULL
239+
}
240+
]
241+
}
242+
]
243+
}
244+
}
245+
}
246+
```
247+
183248
## Hive on s3
184249

185250
### Step 1
@@ -307,15 +372,3 @@ sink {
307372
}
308373
}
309374
```
310-
311-
## Changelog
312-
313-
### 2.2.0-beta 2022-09-26
314-
315-
- Add Hive Source Connector
316-
317-
### Next version
318-
319-
- [Improve] Support kerberos authentication ([3840](https://github.com/apache/seatunnel/pull/3840))
320-
- Support user-defined partitions ([3842](https://github.com/apache/seatunnel/pull/3842))
321-

0 commit comments

Comments
 (0)