Skip to content

Commit 4c10b3e

Browse files
committed
HADOOP-19696. improve docs.
1 parent 1095f86 commit 4c10b3e

File tree

2 files changed

+24
-10
lines changed

2 files changed

+24
-10
lines changed

BUILDING.txt

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -391,16 +391,18 @@ Note that the site needs to be built in a second pass after other artifacts.
391391
----------------------------------------------------------------------------------
392392
Including Cloud Connector Dependencies in Distributions:
393393

394-
Hadoop distributions include the hadoop modules to work with data and services
394+
Hadoop distributions include the hadoop modules need to work with data and services
395395
on cloud infrastructure
396396

397397
However, dependencies are omitted for all cloud connectors except hadoop-azure
398398
(abfs:// and wasb://) and possibly hadoop-gcp (gs://) and hadoop-tos (tos://).
399399
For the latter two modules, it depends on shading options.
400400

401-
For hadoop-aws the AWS SDK bundle.jar omitted, but everything else is included.
401+
For hadoop-aws the AWS SDK bundle.jar is omitted, but everything else is included.
402402

403-
* This keeps binary release size below the limit of apache distributions
403+
404+
Excluding the extra binaries:
405+
* Keeps release artifact size below the limit of the ASF distribution network
404406
* Reduces download and size overhead in docker usage.
405407
* Reduces the CVE attack surface
406408
* Reduces the risk of classpath conflict.
@@ -425,13 +427,18 @@ Available package profiles:
425427
To build a complete distribution then with all cloud dependencies included:
426428

427429
mvn package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true \
428-
-Dhadoop-aliyun-package -Dhadoop-aws-package -Dhadoop-azure-datalake-package\
429-
-Dhadoop-cos-package -Dhadoop-gcp-package -Dhadoop-tos-package
430+
-Dhadoop-aliyun-package \
431+
-Dhadoop-aws-package \
432+
-Dhadoop-azure-datalake-package \
433+
-Dhadoop-cos-package \
434+
-Dhadoop-gcp-package \
435+
-Dhadoop-huaweicloud-package \
436+
-Dhadoop-tos-package
430437

431438
The resulting tar file will be too large to be distributable through ASF infrastructure.
432439

433-
The hadoop-gcp and hadoop-tos artifacts include their dependencies unless the distribution
434-
is built with -DskipShade.
440+
The hadoop-gcp and hadoop-tos artifacts include their dependencies as shaded
441+
artifacts unless the distribution is built with -DskipShade.
435442

436443
----------------------------------------------------------------------------------
437444
Installing Hadoop

hadoop-cloud-storage-project/hadoop-cloud-storage-dist/pom.xml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -61,9 +61,16 @@
6161
hadoop-huaweicloud-package
6262
hadoop-tos-package
6363
64-
To build a complete distribution then
65-
mvn package -Pdist -DskipTests -Phadoop-aliyun-package,hadoop-aws-package,hadoop-azure-datalake-package\
66-
-Phadoop-cos-package,hadoop-gcp-package,hadoop-tos-package
64+
To build a complete distribution then:
65+
66+
mvn package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true \
67+
-Dhadoop-aliyun-package \
68+
-Dhadoop-aws-package \
69+
-Dhadoop-azure-datalake-package \
70+
-Dhadoop-cos-package \
71+
-Dhadoop-gcp-package \
72+
-Dhadoop-huaweicloud-package \
73+
-Dhadoop-tos-package
6774
6875
-->
6976
<properties>

0 commit comments

Comments
 (0)