NOAH: Interactive Spreadsheet Exploration with Dynamic Hierarchical Overviews

NOAH is a dynamic hierarchical overview that can be embedded alongside spreadsheets to explore the data at various granularities by zooming in and out of the spreadsheet. The overview is essentially a histogram with bins and depicts the overall data distribution, enabling users to quickly assimilate the data. Using NOAH users can issue formulae over data subsets without cumbersome scrolling or range selection, enabling users to gain a high or low-level perspective of the spreadsheet. NOAH enables users to customize the overview and also automatically manages user's exploration history to help them retrace previous steps in the analytics session. NOAH is integrated within DataSpread, a spreadsheet-database hybrid system with a spreadsheet frontend and a database backend. Following is a video demo of NOAH:

Key Features

• Navigation: Users can navigate the spreadsheet by interacting with the bins of the overview via clicking and semantic zooming operations. Details here.
• Aggregation: Users can issue spreadsheet formulae on the overview and view the results within an aggregate column. Details here.
• Customization: Users can perform various custimizations such as merging or splitting bins of the overview. Details here.
• Context Management: Users can view their navigation history in a context bar and navigate the hierarchical overview using a breadcrumb. Details here.

Setup Instructions

NOAH can be deployed locally through Docker (recommended) or through Apache Tomcat.

Method 1: Build and Deploy Using Docker

Required Software

• Docker >= 1.13.0

Deploying NOAH locally.

1. Clone the NOAH repository and go the directory in your terminal. Alternatively, you can download the source as a zip or tar.gz.

2. Install Docker. Docker makes it easy to separate applications from underlying infrastructure so setting up and running applications is quick and easy.

3. Start Docker and start the application. It should be accessible at http://localhost:8080/. Stop the application with CTRL+C.

   docker-compose up
   

Method 2: Build and Deploy Using Maven+Tomcat

To host NOAH locally on Tomcat, you can either use one of the pre-built WAR files, available at /build-web/DataSpread.war, or build the WAR file yourself from the source.

Required Software

• Java Platform (JDK) >= 8
• PostgreSQL >= 10.5
• PostgreSQL JDBC driver = 42.1.4
• Apache Tomcat >= 8.5.4
• Apache Maven >= 3.5.0
• NodeJS >= 10.9

Building Instructions (To generate a WAR file)

1. Clone the NOAH repository. Alternatively, you can download the source as a zip or tar.gz.

2. Use maven to build the war file using the following command. After the build completes, the WAR is available at webapp/target/DataSpread.war.

   mvn clean install
   

Deploying NOAH locally

1. Install PostgreSQL database. Postgres.app is a quick way to get PostgreSQL working on Mac. For other operating systems check out the guides here.

2. Create a database and an user who has access to the database. Note the database name, username and password. Typically when you have PostgreSQL installed locally the password is blank.

3. Install Apache Tomcat. You can use the guide here. Make a note of the directory where tomcat is installed. This is known as TOMCAT_HOME in all documentation.

4. Update the Tomcat configuration. You need to update the following file, which is present in conf folder under TOMCAT_HOME folder.

   1. context.xml by adding the following text at the end of the file before the closing XML tag.

   <Resource name="jdbc/ibd" auth="Container"
             type="javax.sql.DataSource" driverClassName="org.postgresql.Driver"
             url="jdbc:postgresql://127.0.0.1:5432/<database_name>"
             username="<username>" password="<password>"
                 maxTotal="20" maxIdle="10" maxWaitMillis="-1" defaultAutoCommit="false" accessToUnderlyingConnectionAllowed="true"/>
   

   Replace <database_name>, <username> and <password> with your PostgreSQL's database name, user name and password respectively.

5. Copy postgresql-42.1.4.jar (Download from here) to lib folder under TOMCAT_HOME. It is crucial to have the exact version of this file.

6. Deploy the WAR file within Tomcat as the root application. This can be done via Tomcat's web interface by undeploying any application located at / and deploying the WAR file with the context path /. To do this manually, delete the webapps/ROOT folder under TOMCAT_HOME while the application is not running, copy the WAR file to the webapps folder, and rename it to ROOT.war.

Launching NOAH

After step 3 of Method 1 or step 6 of method 2, you are ready to run the program. Visit the url which is typically http://localhost:8080/ for a local install. Upload a file and start navigation using the following steps:

Menu Selection

From the menubar select Nav -> Explore.

Attribute Selection

Select the attribute by which you want to explore the data.

Overview Display

The overview is displayed within a Navigation Panel on the left (see a in the following Fgiure). Users can perform various operarion on the overview such as navigation, aggregation, customization, and revisting navigation history.

Technical Details

For more details on NOAH read out technical paper at VLDB 2021. Cite our work as follows:

@article{rahman2021noah,
  title={NOAH: Interactive Spreadsheet Exploration with Dynamic Hierarchical Overviews},
  author={Rahman, Sajjadur and Bendre, Mangesh and Liu, Yuyang and Zhu, Shichu and Su, Zhaoyuan and Karahalios, Karrie and Parameswaran, Aditya},
  journal={Proceedings of the VLDB Endowment},
  volume={14},
  number={6},
  pages={970--983},
  year={2021},
  publisher={VLDB Endowment}
}

License

MIT