-
Notifications
You must be signed in to change notification settings - Fork 5.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add documentation for file-based metastore
- Loading branch information
1 parent
a6d82fe
commit a915d6f
Showing
3 changed files
with
76 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
73 changes: 73 additions & 0 deletions
73
presto-docs/src/main/sphinx/connector/file-based-metastore.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
==================== | ||
File-Based Metastore | ||
==================== | ||
|
||
.. contents:: | ||
:local: | ||
:backlinks: none | ||
:depth: 1 | ||
|
||
Overview | ||
^^^^^^^^ | ||
|
||
For testing or developing purposes, Presto can be configured to use a local | ||
filesystem directory as a Hive Metastore. | ||
|
||
The file-based metastore works only with the following connectors: | ||
|
||
* :doc:`/connector/deltalake` | ||
* :doc:`/connector/hive` | ||
* :doc:`/connector/hudi` | ||
* :doc:`/connector/iceberg` | ||
|
||
Partitioned tables are supported when created in the file-based metastore. | ||
|
||
Configuring a File-Based Metastore | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
1. In ``etc/catalog/``, find the catalog properties file for the supported | ||
connector. | ||
|
||
2. In the catalog properties file, set the following properties: | ||
|
||
.. code-block:: none | ||
hive.metastore=file | ||
hive.metastore.catalog.dir=file:///<catalog-dir> | ||
Replace ``<catalog-dir>`` in the example with the path to a directory on an | ||
accessible filesystem. | ||
|
||
Using a File-Based Warehouse | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
For this example, assume the Hive connector is being used, and the properties | ||
in the Hive connector catalog file are set to the following: | ||
|
||
.. code-block:: none | ||
connector.name=hive | ||
hive.metastore=file | ||
hive.metastore.catalog.dir=file:///data/hive_data/ | ||
Create a schema | ||
|
||
.. code-block:: none | ||
CREATE SCHEMA hive.warehouse; | ||
This query creates a directory ``warehouse`` in the directory set for | ||
``hive.metastore.catalog.dir``, so the path to the new directory is | ||
``/data/hive_data/warehouse``. | ||
|
||
Create a table with any connector-supported file formats. For example, if the | ||
Hive connector is being configured: | ||
|
||
.. code-block:: none | ||
CREATE TABLE hive.warehouse.orders_csv("order_name" varchar, "quantity" varchar) WITH (format = 'CSV'); | ||
CREATE TABLE hive.warehouse.orders_parquet("order_name" varchar, "quantity" int) WITH (format = 'PARQUET'); | ||
These queries create folders as ``/data/hive_data/warehouse/orders_csv`` and | ||
``/data/hive_data/warehouse/orders_parquet``. Users can insert and query | ||
from these tables. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters