# **Disaggregated HDP Spark and Hive with MinIO**

## **1. Cloud-native Architecture**

![cloud-native](https://github.com/minio/minio/blob/master/docs/bigdata/images/image1.png?raw=true "cloud native architecture")

Kubernetes manages stateless Spark and Hive containers elastically on the compute nodes. Spark has native scheduler integration with Kubernetes. Hive, for legacy reasons, uses YARN scheduler on top of Kubernetes.

			if err != nil {
				internalLogIf(ctx, err, logger.WarningKind)
				break
			}

			// Handle if we have some buckets in-memory those are stale.
			// first delete them and then replace the newer state()
			// from disk.
			diskBuckets := set.CreateStringSet()
			for _, bucket := range buckets {
				diskBuckets.Add(bucket.Name)
			}
			sys.RemoveStaleBuckets(diskBuckets)

| `minio_notify_target_queue_length`             | Number of events currently staged in the queue_dir configured for the target.                                                               |

Although in FastAPI it's optional, and is used mainly to set headers, cookies, and alternative status codes.

///

### [Molten](https://moltenframework.com/) { #molten }

I discovered Molten in the first stages of building **FastAPI**. And it has quite similar ideas:

* Based on Python type hints.
* Validation and documentation from these types.
* Dependency Injection system.

        assertEquals(info3.path, decodedInfo3.path);
        assertEquals(info3.comment, decodedInfo3.comment);
        assertEquals(info3.state, decodedInfo3.state);
        assertEquals(info3.num_stores, decodedInfo3.num_stores);
        assertEquals(info3.stores[0].state, decodedInfo3.stores[0].state);
        assertEquals(info3.stores[0].server_name, decodedInfo3.stores[0].server_name);

LICENSE (CDDL)

The code released under the CDDL shall be governed by the laws of the
State of California (excluding conflict-of-law provisions). Any
litigation relating to this License shall be subject to the jurisdiction
of the Federal Courts of the Northern District of California and the
state courts of the State of California, with venue lying in Santa Clara
County, California.

Beachten Sie, dass der Defaultwert weiterhin `None` ist, so dass der Parameter weiterhin optional ist.

Aber jetzt, mit `Query(max_length=50)` innerhalb von `Annotated`, sagen wir FastAPI, dass wir eine **zusätzliche Validierung** für diesen Wert wünschen, wir wollen, dass er maximal 50 Zeichen hat. 😎

/// tip | Tipp

Wenn wir `-> HeroPublic` deklariert hätten, würden Ihr Editor und Linter (zu Recht) reklamieren, dass Sie ein `Hero` anstelle eines `HeroPublic` zurückgeben.

Durch die Deklaration in `response_model` sagen wir **FastAPI**, dass es seine Aufgabe erledigen soll, ohne die Typannotationen und die Hilfe von Ihrem Editor und anderen Tools zu beeinträchtigen.

///

### Helden mit `HeroPublic` lesen { #read-heroes-with-heropublic }

     *   <li>If this field's value == null because it was associated with our thread and was
     *       cleared, we know that we're not executing inline any more
     * </ul>
     *
     * All the states where thread != currentThread are identical for our purposes, and so even
     * though it's racy, we don't care which of those values we get, so no need to synchronize.
     */
    @LazyInit @Nullable Thread thread;

     terms or conditions communicated by You unless expressly agreed.

  b. Any arrangements, understandings, or agreements regarding the
     Licensed Material not stated herein are separate from and
     independent of the terms and conditions of this Public License.


Section 8 -- Interpretation.

  a. For the avoidance of doubt, this Public License does not, and