Skip to content

Commit

Permalink
Updated documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
Valentin-Metz committed Dec 22, 2023
1 parent 4f89982 commit bd4013d
Show file tree
Hide file tree
Showing 5 changed files with 54 additions and 84 deletions.
104 changes: 53 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,77 +4,79 @@
[![TUM-Live](https://custom-icon-badges.demolab.com/badge/TUM--Live-live-e5312b.svg?logo=tum_live_logo)](https://live.rbg.tum.de/)
[![TUM-Panopto](https://custom-icon-badges.demolab.com/badge/TUM--Panopto-online-11773d.svg?logo=panopto_icon_2015)](https://tum.cloud.panopto.eu/)

# Installation
# Docker

Required system dependencies:
The suggested way to run this project is via Docker:

```
python >= 3.9.9
ffmpeg >= 4.4.1
chromium >= 96.0.4664.93
chromedriver >= 96.0.4664.93
```bash
docker run -it -v config.yml:/app/config.yml -v target_location:/app/output ghcr.io/valentin-metz/tum_video_scraper:master
```

Create a virtual environment (in the project folder) and install project-dependencies into it.
You'll need to link in the configuration file `config.yml`.
You can find an example in the root of this repository under `example_config.yml`.
The output folder you specify in the config file will be the target location *inside* the docker container,
so make sure to mount your target location to `/app/output`.

(This is only required if you run directly from the python source)
# How to find subject identifiers?

```bash
python3 -m venv venv
source ./venv/bin/activate
python3 -m pip install -U pip
python3 -m pip install -U -r requirements.txt
```
Subject identifiers are used to specify the subjects you want to download.

# Running
## TUM-Live:

Activating your venv:
For TUM-Live, you can find them here: https://live.rbg.tum.de/old/.
After you select a lecture series, check the URL for the subject identifier.

(This is only required if you run directly from the python source)
Example:
`https://live.rbg.tum.de/old/course/2023/W/NetSec`

```
source ./venv/bin/activate
```
In this case, the subject identifier is `2023/W/NetSec`.

Usage:
Finally, you can specify the video stream you want to download.
Usually TUM-Live offers three:

```
The first argument supplied must be the output directory.
If you want to supply it at a different position, the argument must be marked with "output_folder".
(Example: output_folder /home/feuermagier/videos/Lectures)
1. The combined view (specified with `:COMB` after the subject identifier)
2. The presentation view (specified with `:PRES` after the subject identifier)
3. The presenter camera view (specified with `:CAM` after the subject identifier)

## Panopto:

For Panopto, you need to supply a `folderID`.
You can find these in the URL of the folder you want to download.

Optional arguments:
Example: `https://tum.cloud.panopto.eu/Panopto/Pages/Sessions/List.aspx#folderID=%22a150c6d5-6cbe-40b0-8dc1-ad0a00967dfb%22`

--help: Prints a help message
In this case, `a150c6d5-6cbe-40b0-8dc1-ad0a00967dfb` would be the `folderID`.

--tum_live: Download a subject from TUM-live (subject_name:subject_identifier:camera_type)
subject_name: Will be used as the folder name. Freely choosable by you.
subject_identifier: Can be found in the URL of your chosen subject. For now please make sure to use https://live.rbg.tum.de/old/
(Example: https://live.rbg.tum.de/old/course/2021/W/it-sec - "2021/W/it-sec" is the subject_identifier for videos of this subject)
camera_type: The camera-view to download
COMB: Presentation slides fused with speaker-camera
PRES: Presentation slides
CAM: Speaker camera
--panopto: Download a folder from TUM-Panopto. As Panopto is login-only you will have to supply your TUM-credentials.
subject_name: Will be used as the folder name. Freely choosable by you.
folder_id: Can be found in the URL of your Panopto folder
(Example: https://tum.cloud.panopto.eu/Panopto/Pages/Sessions/List.aspx#folderID=a150c6d5-6cbe-40b0-8dc1-ad0a00967dfb - "a150c6d5-6cbe-40b0-8dc1-ad0a00967dfb" is the folder_id)
-----

--username: Your TUM-Username (Example: go42tum)
--password: The password for your TUM-Username (Example: "hunter2")
You won't need anything below this line if you are running from Docker.

--keep: Keep the original version of a downloaded video insted of only the jump-cutted ones. Defaults to true.
--jumpcut: Save a jump-cutted version of the video (fast-forwarding frames with silence). Defaults to true.
-----

--temp_dir: Allows you to spcify a custom temp-directory. Usually the system-temp-folder will be used. You probably won't need this.
--maximum_parallel_downloads: Allows you to specify how many videos we download and convert at a time. Conversion uses a lot of RAM, so be careful with this. The default is 3.
# Installation

If you want to run this project directly from the python source,
you'll need to install the following system dependencies:

```
python >= 3.11
ffmpeg >= 6.1
firefox >= 120.0
geckodriver >= 0.33
```

# Automation
In addition to that, you'll need the python dependencies specified in `requirements.txt`.
Create a virtual environment (in the project folder) and install project-dependencies into it:

If you want to automatically download your lectures in the future, all you need to do is run the scraper periodically.
```bash
python3 -m venv venv
source ./venv/bin/activate
python3 -m pip install -U pip
python3 -m pip install -U -r requirements.txt
```

I suggest you use `systemd` for this task. Instructions on how to do this can be found in the `/systemd/README.md` of
this repository.
Run the project with:

```bash
python3 src/main.py -c config.yml
```
2 changes: 1 addition & 1 deletion example_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ Password: "hunter2"
Keep-Original-File: true
Jumpcut: true

Output-Folder: "/mnt/NAS/Video/Lectures/"
Output-Folder: "/app/output"

Maximum-Parallel-Downloads: 3
12 changes: 0 additions & 12 deletions systemd/README.md

This file was deleted.

10 changes: 0 additions & 10 deletions systemd/tum_video_scraper.service

This file was deleted.

10 changes: 0 additions & 10 deletions systemd/tum_video_scraper.timer

This file was deleted.

0 comments on commit bd4013d

Please sign in to comment.