# **Disaggregated HDP Spark and Hive with MinIO**

## **1. Cloud-native Architecture**

![cloud-native](https://github.com/minio/minio/blob/master/docs/bigdata/images/image1.png?raw=true "cloud native architecture")

Kubernetes manages stateless Spark and Hive containers elastically on the compute nodes. Spark has native scheduler integration with Kubernetes. Hive, for legacy reasons, uses YARN scheduler on top of Kubernetes.

daysUntilStale: 30

# Number of days of inactivity before an Issue or Pull Request with the stale label is closed.
# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale.
daysUntilClose: 15

# Only issues or pull requests with all of these labels are check if stale. Defaults to `[]` (disabled)
onlyLabels: []

# Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable

install ## ## In addition to default policies [readonly|readwrite|writeonly|consoleAdmin|diagnostics] ## you can define additional policies with custom supported actions and resources policies: [] ## writeexamplepolicy policy grants creation or deletion of buckets with name ## starting with example. In addition, grants objects write permissions on buckets starting with ## example. # - name: writeexamplepolicy # statements: # - resources: # - 'arn:aws:s3:::example*/*' # actions: # - "s3:AbortMultipartUpload"...

install ## ## In addition to default policies [readonly|readwrite|writeonly|consoleAdmin|diagnostics] ## you can define additional policies with custom supported actions and resources policies: [] ## writeexamplepolicy policy grants creation or deletion of buckets with name ## starting with example. In addition, grants objects write permissions on buckets starting with ## example. # - name: writeexamplepolicy # statements: # - resources: # - 'arn:aws:s3:::example*/*' # actions: # - "s3:AbortMultipartUpload"...

MinIO is a high-performance, S3-compatible object storage solution released under the GNU AGPL v3.0 license.
Designed for speed and scalability, it powers AI/ML, analytics, and data-intensive workloads with industry-leading performance.

- S3 API Compatible – Seamless integration with existing S3 tools
- Built for AI & Analytics – Optimized for large-scale data pipelines
- High Performance – Ideal for demanding storage workloads.

}

// sortIPs - sort ips based on higher octets.
// The logic to sort by last octet is implemented to
// prefer CIDRs with higher octets, this in-turn skips the
// localhost/loopback address to be not preferred as the
// first ip on the list. Subsequently this list helps us print
// a user friendly message with appropriate values.
func sortIPs(ipList []string) []string {
	if len(ipList) == 1 {
		return ipList
	}

install ## ## In addition to default policies [readonly|readwrite|writeonly|consoleAdmin|diagnostics] ## you can define additional policies with custom supported actions and resources policies: [] ## writeexamplepolicy policy grants creation or deletion of buckets with name ## starting with example. In addition, grants objects write permissions on buckets starting with ## example. # - name: writeexamplepolicy # statements: # - resources: # - 'arn:aws:s3:::example*/*' # actions: # - "s3:AbortMultipartUpload"...

install ## ## In addition to default policies [readonly|readwrite|writeonly|consoleAdmin|diagnostics] ## you can define additional policies with custom supported actions and resources policies: [] ## writeexamplepolicy policy grants creation or deletion of buckets with name ## starting with example. In addition, grants objects write permissions on buckets starting with ## example. # - name: writeexamplepolicy # statements: # - resources: # - 'arn:aws:s3:::example*/*' # actions: # - "s3:AbortMultipartUpload"...

install ## ## In addition to default policies [readonly|readwrite|writeonly|consoleAdmin|diagnostics] ## you can define additional policies with custom supported actions and resources policies: [] ## writeexamplepolicy policy grants creation or deletion of buckets with name ## starting with example. In addition, grants objects write permissions on buckets starting with ## example. # - name: writeexamplepolicy # statements: # - resources: # - 'arn:aws:s3:::example*/*' # actions: # - "s3:AbortMultipartUpload"...

      the copyright owner that is granting the License.

      "Legal Entity" shall mean the union of the acting entity and all
      other entities that control, are controlled by, or are under common
      control with that entity. For the purposes of this definition,
      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or