Quick start

Introduction

In the tutorial, I will configure two instances of light sources: a classic LED strip on the back of the TV (highspeed AWA adalight USB) and a remote instance of Philips Hue lamps. Both will be used at the same time to get a deeper immersion in the ambient lighting. Of course, you can only use one instance with LED strip and skip the part for the second instance. HyperHDR is installed on Apple macOS. But the steps are pretty much the same for other operating systems as used LED drivers are independent of the operating system. The Ezcap 321 USB grabber serves as a video source.

1. Must read
2. HyperHDR basics
3. How to create and enable your first LED strip (instance)
4. Manual LED layout editing and testing
5. Smoothing and anti-flickering filter
6. Add your second instance e.g. Philips Hue lamps
7. Configuring and testing your USB video grabber
8. JsonAPI for remotely controlling HyperHDR
9. Configuring and testing audio effects

Performance pro-tips:

Although the use of WiFi LED drivers tempts with simplicity, in fact, for the entire duration of the session that requires constant data transmission, you will be at the mercy of the quality of the radio connection, the reliability and compatibility of your router with your esp8266 / esp32 model, the quality of the latter and the reliability of the given version e.g. Espressif SDK which was used to build the firmware. In addition, LED control can often lead to conflict with WiFi communication (especially "IRQ thunderstorm" if popular RMT method is used). You have been warned ⚠️

- Use the highest framerate for your USB grabber. GENERAL RULE: HIGHEST FRAMERATE, LOWEST RESOLUTION. 
- Choose YUV/NV12/RGB codec for your grabber instead of MJPEG.  
  Note: some low quality USB3.0 OS drivers (Rpi4 is not affected) can cause higher CPU usage for YUV/NV12/RGB.  
  In that case try to switch to MJPEG and monitor the performance.
- Do not set high capturing resolutions like 1080p: they have little effect for your ambient lightning but they
  drain resources. Avoid grabbers without a scaler offering only very high capturing resolution or MJPEG only codec.
- Check your video processing logs updated every minute: if the video processing latency exceeds ~10-20ms
  then you probably want to lower it down. Lower your capturing resolution or use 'Quarter of frame mode'.
- If your CPU usage is near 100% for single core unit (or 400% for 4 cores Rpi 2/3/4) then consider
  reducing the frame-rate. It the grabber doesn't offer lower rate use 'Software frame skipping' option.
- Keep your 'Smoothing' processing settings realistic. Don't use 80Hz Smoothing refresh rate if your 
  old Arduino LED driver can output only 20Hz. 
- If your are not sure about your 'Smoothing' configuration then disable that module. If it helps 
  then turn it on again and start carefully experimenting with your settings starting with
  delay around 50ms and refresh rate ~20-50Hz.

First steps

Ok let's get started. Connect to HyperHDR using the address: http://IP_OF_HYPERHDR:8090

Main menu is located on the left part of the page:

The instance switch button is disabled for now because there is only one instance at the beginning. There is a warning about setting a new password, so it's a good idea to do it now. Follow the link or go to the "Advanced" tab to change the password:

Configuring the LED strip

First give a friendly name for our instance:

The LED strip is controlled by USB high-speed HyperSerialWLED/HyperSerial8266/HyperSerialESP32. So go to the LED devices and select 'LED hardware' tab then adalight from the list.

On Raspberry Pi the COM list could be different. Remember that Rpi 3 & 4 has available Bluetooth device that is enabled on default.

Because we use non-standard highspeed AWA protocol @2Mb we need to configure two more things:

Now we need to set geometry of the LED strip. For example let's assume we have: input in the middle of the bottom, 50 LEDs at the top, 25 LEDs on the left/right. At the bottom we 40 LEDs and a gap of 10 LEDs so you must put 50 LEDs (40 + 10) as bottom and set gap as 10.

Still there is a problem with the input position. Set gap & input position to 95 (top 50 + right 25 + one of bottom part 20)

Manual LED layout and testing

If you are not satisfied with the automatically generated layout, you can correct it manually. Right-click on the selected LED and a context menu will be displayed. You can manually adjust its position (Move) or size (Properties), disable it (it will never be lit) or remove it (Delete) from the system altogether.

Use 'Identify' to test a particular diode or check its physical location. This will cause it to blink for a few seconds.

Smoothing anti-flickering filter

Without it you may experience disturbing subtle flickering on the dark scenes. It's caused by the video source or video player aggressive dithering with a combination of high refresh rate on the LED strip. It's detailed described on the configuration page (Image processing tab).

You can see how the problem manifests on the following clip and how is resolved on the second one when anti-flickering filter is enabled with relaxed settings: threshold = 32 and minimal step = 2 (you may need to increase it on your system) . Capturing video at such conditions is a difficult task for the phone camera to maintain colors but at least luminescence's changes are visible on these samples. New smoothing algorithm 'Alternative Linear' was used. It was introduced in v16 and it is similar to the standard 'Linear' but at the end of transition the decay effect is reduced. Flickering appears at 00:07 and at the 00:18 flickering without the filter and the flickering issue is resolved on the same video source with new anti-flickering filter: no flickering with the filter

Second instance (Philips Hue lamps)

First create new instance for our Philips Hue lamp:

Now switch to the new instance:

Before proceeding you must create an 'Entertainment group' for your lamps in the Philips Android or IOS mobile application. Don't even try without doing it first. Your Hue Bridge should also to be working for at least 2-3 minutes to avoid connection problems during configuration at startup. You should try first to send a color from the Philips mobile app to make sure it's working and your 'Entertainment group' is created.

Then add Philips Hue light source and run the wizard:

HyperHDR should find the Philips Hue bridge in the same sub-network. Some custom router's firewall rules or enabling WiFi isolation can cause the process to fail.

Click the 'Create new user and and clientkey' button. Now it's time to click a large hardware button located on your Hue bridge. It allows to authorize the HyperHDR's access. If everything goes OK you should have the Username and Clientkey filled in automatically. Identificator of 'Entertainment group' should also be found automatically.

Proceed with clicking the 'use group...' button.

Now you can assign area of the TV to single, selected lamp in the entertainment group. Because I have 2 lamps on the floor (one just on the right of the TV and second to the left) I selected following options.

Configuring video & system grabber

Now we configure the video source for HyperHDR: it can be an USB grabber or system screen capture. Go to the 'Video capturing tab' and select your USB grabber.

The yellow 'Info' button will appear after you select the grabber. Click it to open the dialog. Ezcap 321 offers very high capturing modes. I choose the lowest NV12. You should not go above 720p, because it causes drain of CPU resources & introduces video processing lag and gives very little to our ambient effect.

Next I check 'Quarter to frame mode' option. It will reduce the frame dimensions (to 50%) & size (to 25%) but will not reduce color's data due to NV12 codec properties.

Click the 'Video preview & LED visualization' button in the upper left corner:

The video stream colors are washed-out and overall luminescence is low. That's because we have captured a HDR10 video stream and almost none of USB grabbers can process HDR metadata so important part of information about the image is lost. Let's back to the Capturing hardware tab and enable 'HDR to SDR tone mapping'.

Remotely controlling HyperHDR (JsonAPI)

HyperHDR has a built-in tool thanks to which you can quickly generate the necessary commands with a few clicks of the mouse. With it, you can remotely control some HyperHDR functions, turn devices such as USB grabbers or LEDs on or off, etc.

For example:

You can remotely control the HDR tone mapping state using for example Home Automation system. If you are using Home Assistant and Denon amplifier, it's possible to automatically switch it depending on the actual video stream format (example).

Turning HDR tone mapping OFF: http://IP_OF_HYPERHDR:8090/json-rpc?request={%22command%22%3A%22videomodehdr%22%2C%22HDR%22%3A0}

Turning HDR tone mapping ON: http://IP_OF_HYPERHDR:8090/json-rpc?request={%22command%22%3A%22videomodehdr%22%2C%22HDR%22%3A1}

Getting HDR tone mapping state (search for videomodehdr property): http://IP_OF_HYPERHDR:8090/json-rpc?request={%22command%22%3A%22serverinfo%22}

Configuring and enabling sound visualization effects

One of unique HyperHDR feature is sound visualization on your ambient ecosystem. Digital capture devices are preferred as there is no analog sound filter at input available in the current version. To make it work you must make sure that the grabber receive audio stream. Be aware that some amplifiers like Denon can block it in typical configuration (use of ARC/EARC is necessary then).

Let's configure the hardware first in the 'Effect' tab. On the following screen I selected Ezcap 269 device. Do not confuse it with some other system devices.

Next we need to enable 'Activate' option and save our settings.

Navigate to the 'Remote control' tab and turn on video preview to verify result.

For the beginning use 'Equalizer' from the list to test if everything works OK.

If you activated & set up your grabber correctly and it receives a audio stream you should see jumping equalizer's bars.

If they are flat something is wrong. In 99% cases your grabber doesn't receive a sound. Navigate to the 'Logs' tab to confirm it. Otherwise you can have fun with other music effects. See them in the video visualizer to know how they work.

Let's see the logs...

It's a final confirmation: you have enabled some audio device and it works but it provides no sound...only silence. Maybe you've selected wrong device but more probably there is no sound on the grabber's input.