Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Christmas Edition - Audio_service addon to flet_video #4558

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

syleishere
Copy link
Contributor

@syleishere syleishere commented Dec 12, 2024

Description

Christmas Edition, full background play on android/IOS now, full bluetooth support, beautiful configurable notification screen, a function to update flet slider with percentages for position.

Test Code

Nothing changes to normal example except more configurable options.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • I signed the CLA.
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass locally with my changes
  • I have made corresponding changes to the documentation (if applicable)

Additional details

Requires heavy mods to flet-build-template, I will submit a PR to -dev branch there like I have done with client dev area manifest here.

Summary by Sourcery

Integrate audio_service to enable background media playback and Bluetooth control on Android and iOS. Add a configurable notification screen and a function to update the Flet slider with media position percentages. Update AndroidManifest.xml to support new features.

New Features:

  • Add full background play support on Android and iOS using the audio_service package.
  • Implement full Bluetooth support for media playback control.
  • Introduce a configurable notification screen for media playback.
  • Add a function to update the Flet slider with percentage values for media position.

Enhancements:

  • Enhance the AndroidManifest.xml to include necessary permissions and services for media playback and Bluetooth support.

@FeodorFitsner
Copy link
Contributor

I love your mood here! 😅🥂

@ndonkoHenri
Copy link
Contributor

Hey @syleishere, thanks for contributing.
Will you mind providing code for review?

@syleishere
Copy link
Contributor Author

syleishere commented Dec 13, 2024

Sure, just adding on to example on Flet video page:
New configurables in extras section for audio_service, new throttle variable to throttle on_position_changed, new on_position_changed handler returning position, duration and percentage.

import random
import flet as ft


def main(page: ft.Page):
    page.theme_mode = ft.ThemeMode.LIGHT
    page.title = "TheEthicalVideo"
    page.window.always_on_top = True
    page.spacing = 20
    page.horizontal_alignment = ft.CrossAxisAlignment.CENTER

    def handle_pause(e):
        video.pause()
        print("Video.pause()")

    def handle_play_or_pause(e):
        video.play_or_pause()
        print("Video.play_or_pause()")

    def handle_play(e):
        video.play()
        print("Video.play()")

    def handle_stop(e):
        video.stop()
        print("Video.stop()")

    def handle_next(e):
        video.next()
        print("Video.next()")

    def handle_previous(e):
        video.previous()
        print("Video.previous()")

    def handle_volume_change(e):
        video.volume = e.control.value
        page.update()
        print(f"Video.volume = {e.control.value}")

    def handle_playback_rate_change(e):
        video.playback_rate = e.control.value
        page.update()
        print(f"Video.playback_rate = {e.control.value}")

    def handle_seek(e):
        video.seek(10000)
        print(f"Video.seek(10000)")

    def handle_add_media(e):
        video.playlist_add(random.choice(sample_media))
        print(f"Video.playlist_add(random.choice(sample_media))")

    def handle_remove_media(e):
        r = random.randint(0, len(video.playlist) - 1)
        video.playlist_remove(r)
        print(f"Popped Item at index: {r} (position {r+1})")

    def handle_jump(e):
        print(f"Video.jump_to(0)")
        video.jump_to(0)

    def handle_track_changed(e):
        index = int(e.data)
        print(f"Track changed to index: {index}, Currently Playing {SONG_LIST[index]}")

    def handle_position_changed(e):
        print(f"Position is {e.position} seconds, duration is {e.duration} seconds, percent done is {e.percent}%")

    def handle_error(e):
        print(f"Video error: {e}")

    SONG_LIST = [
        "https://user-images.githubusercontent.com/28951144/229373720-14d69157-1a56-4a78-a2f4-d7a134d7c3e9.mp4",
        "https://user-images.githubusercontent.com/28951144/229373718-86ce5e1d-d195-45d5-baa6-ef94041d0b90.mp4",
        "https://user-images.githubusercontent.com/28951144/229373716-76da0a4e-225a-44e4-9ee7-3e9006dbc3e3.mp4",
        "https://user-images.githubusercontent.com/28951144/229373695-22f88f13-d18f-4288-9bf1-c3e078d83722.mp4",
        "https://user-images.githubusercontent.com/28951144/229373709-603a7a89-2105-4e1b-a5a5-a6c3567c9a59.mp4",
    ]
    sample_media = []
    headers = {
        "User-agent": "mobile",
        "Authorization": "Bearer somelongtoken",
        "email": "[email protected]",
    }
    extras = {
        # audio_service configurables: title,artist,artUri,album,genre,displayTitle,displaySubtitle,displayDescription
        "artUri": "https://mp3.sunsaturn.com/crystaltunes.png", # background url for notification bar
    }
    for i in range(0, len(SONG_LIST)):
        if page.web:
            sample_media.append(ft.VideoMedia(SONG_LIST[i], extras=extras))
        else:
            sample_media.append(ft.VideoMedia(SONG_LIST[i], http_headers=headers, extras=extras))

    page.add(
        video := ft.Video(
            expand=True,
            playlist=sample_media,
            playlist_mode=ft.PlaylistMode.LOOP,
            fill_color=ft.Colors.BLUE_400,
            aspect_ratio=16/9,
            volume=100,
            autoplay=False,
            throttle=1000, # throttle on_position_changed this many ms(default 1000ms/1s)
            filter_quality=ft.FilterQuality.HIGH,
            muted=False,
            on_loaded=lambda e: print("Video loaded successfully!"),
            on_enter_fullscreen=lambda e: print("Video entered fullscreen!"),
            on_exit_fullscreen=lambda e: print("Video exited fullscreen!"),
            on_error=handle_error,
            on_track_changed=handle_track_changed,
            on_position_changed=handle_position_changed,

        ),
        ft.Row(
            wrap=True,
            alignment=ft.MainAxisAlignment.CENTER,
            controls=[
                ft.ElevatedButton("Play", on_click=handle_play),
                ft.ElevatedButton("Pause", on_click=handle_pause),
                ft.ElevatedButton("Play Or Pause", on_click=handle_play_or_pause),
                ft.ElevatedButton("Stop", on_click=handle_stop),
                ft.ElevatedButton("Next", on_click=handle_next),
                ft.ElevatedButton("Previous", on_click=handle_previous),
                ft.ElevatedButton("Seek s=10", on_click=handle_seek),
                ft.ElevatedButton("Jump to first Media", on_click=handle_jump),
                ft.ElevatedButton("Add Random Media", on_click=handle_add_media),
                ft.ElevatedButton("Remove Random Media", on_click=handle_remove_media),
            ],
        ),
        ft.Slider(
            min=0,
            value=100,
            max=100,
            label="Volume = {value}%",
            divisions=10,
            width=400,
            on_change=handle_volume_change,
        ),
        ft.Slider(
            min=1,
            value=1,
            max=3,
            label="PlaybackRate = {value}X",
            divisions=6,
            width=400,
            on_change=handle_playback_rate_change,
        ),
    )


ft.app(main)

@syleishere
Copy link
Contributor Author

syleishere commented Dec 14, 2024

Well if that's it for review, then Merry Christmas everyone, now everyone can code a Flet music/video player, install it on their phones, pop in some wireless bluetooth buds and head to gym in New Year playing your own app and music in background :)

Guess have to wait for Feodor sort out AndroidManifest.xml for this as it's a lot of mods, where on IOS it's like 2 lines in info.plist lol

To recap on AndroidManifest.xml for audio_service from PR:

  1. Very first line of AndroidManifest.xml has to be changed to:
    <manifest xmlns:android="http://schemas.android.com/apk/res/android"
        xmlns:tools="http://schemas.android.com/tools">

What this does is includes "tools" we need for audio_service
2) In <activity section:
android:name="com.ryanheise.audioservice.AudioServiceActivity"
Again the <activity android:name not the <application android:name
3) Right after the finishing of </activity> the following needs to be added:

        <!-- ADD THIS "SERVICE" element -->
        <service android:name="com.ryanheise.audioservice.AudioService"
            android:foregroundServiceType="mediaPlayback"
            android:exported="true" tools:ignore="Instantiatable">
            <intent-filter>
                <action android:name="android.media.browse.MediaBrowserService" />
            </intent-filter>
        </service>

        <!-- ADD THIS "RECEIVER" element -->
        <receiver android:name="com.ryanheise.audioservice.MediaButtonReceiver"
            android:exported="true" tools:ignore="Instantiatable">
            <intent-filter>
                <action android:name="android.intent.action.MEDIA_BUTTON" />
            </intent-filter>
        </receiver>

So again a complete example from flet client devel area PR:

    <manifest xmlns:android="http://schemas.android.com/apk/res/android"
        xmlns:tools="http://schemas.android.com/tools">
    <uses-permission android:name="android.permission.INTERNET" />
    <!-- Media access permissions.
    Android 13 or higher.
    https://developer.android.com/about/versions/13/behavior-changes-13#granular-media-permissions -->
    <uses-permission android:name="android.permission.READ_MEDIA_AUDIO" />
    <uses-permission android:name="android.permission.READ_MEDIA_VIDEO" />
    <!-- Storage access permissions. Android 12 or lower. -->
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
    <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE_LOCATION" />
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
    <uses-permission android:name="android.permission.WAKE_LOCK" />
    <uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
    <uses-permission android:name="android.permission.BLUETOOTH" />
    <uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
    <!-- Google TV -->
    <uses-feature android:name="android.software.leanback" android:required="false" />
    <uses-feature android:name="android.hardware.touchscreen" android:required="false" />

    <application
        android:label="Flet"
        android:name="${applicationName}"
        android:enableOnBackInvokedCallback="true"
        android:icon="@mipmap/ic_launcher">
        <meta-data
                android:name="io.flutter.embedding.android.EnableImpeller"
                android:value="false"/>
        <activity
            android:name="com.ryanheise.audioservice.AudioServiceActivity"
            android:exported="true"
            android:launchMode="singleTop"
            android:theme="@style/LaunchTheme"
            android:configChanges="orientation|keyboardHidden|keyboard|screenSize|smallestScreenSize|locale|layoutDirection|fontScale|screenLayout|density|uiMode"
            android:hardwareAccelerated="true"
            android:windowSoftInputMode="adjustResize">
            <!-- Specifies an Android theme to apply to this Activity as soon as
                 the Android process has started. This theme is visible to the user
                 while the Flutter UI initializes. After that, this theme continues
                 to determine the Window background behind the Flutter UI. -->
            <meta-data
                android:name="io.flutter.embedding.android.NormalTheme"
                android:resource="@style/NormalTheme"
            />
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
                <category android:name="android.intent.category.LEANBACK_LAUNCHER"/>   <!-- Google TV -->
            </intent-filter>
        </activity>

        <!-- ADD THIS "SERVICE" element -->
        <service android:name="com.ryanheise.audioservice.AudioService"
            android:foregroundServiceType="mediaPlayback"
            android:exported="true" tools:ignore="Instantiatable">
            <intent-filter>
                <action android:name="android.media.browse.MediaBrowserService" />
            </intent-filter>
        </service>

        <!-- ADD THIS "RECEIVER" element -->
        <receiver android:name="com.ryanheise.audioservice.MediaButtonReceiver"
            android:exported="true" tools:ignore="Instantiatable">
            <intent-filter>
                <action android:name="android.intent.action.MEDIA_BUTTON" />
            </intent-filter>
        </receiver>

        <!-- Below is a test AdMob ID.
        Guide: https://developers.google.com/admob/flutter/quick-start#platform_specific_setup -->
        <meta-data
            android:name="com.google.android.gms.ads.APPLICATION_ID"
            android:value="ca-app-pub-3940256099942544~3347511713"/>
        <!-- Don't delete the meta-data below.
             This is used by the Flutter tool to generate GeneratedPluginRegistrant.java -->
        <meta-data
            android:name="flutterEmbedding"
            android:value="2" />
    </application>
</manifest>

The main required permissions for this:

<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />

and of course regular ones from media_kit people need like:

<uses-permission android:name="android.permission.READ_MEDIA_AUDIO" />
<uses-permission android:name="android.permission.READ_MEDIA_VIDEO" />
etc...

Flet developers will be required to use flet permissions package to enable these after. For testing in devel area while working on this, I just went to App Info on my phone or emulator and just enabled them all manually.

IOS: Info.plist

  <key>UIBackgroundModes</key>
  <array>
    <string>audio</string>
  </array>

Reference:
https://pub.dev/packages/audio_service

@ndonkoHenri
Copy link
Contributor

Commit: Flet Predictive Back Support Google TV

@syleishere please dont mix up stuffs.

@syleishere
Copy link
Contributor Author

Yeah I apologize, I normally wouldn't but Feodor asked me why I closed other PR, so figured he wanted it. Won't happen again.

@ndonkoHenri
Copy link
Contributor

Can you move that into a separate PR, please?

@syleishere
Copy link
Contributor Author

syleishere commented Dec 14, 2024

I am unsure how to do that, I can't seem to fork 2 copies of flet on github to do separate PR requests. Maybe I can create a branch as a workaround.

ndonkoHenri and others added 6 commits December 13, 2024 20:49
* fix center_on

* get default animation duration and curve
…let-dev#4525)

* avoid jsonDecoding `Segment` and `BarChartRod` tooltips

* avoid jsonEncoding `Segment` and `BarChartRod` tooltips

* Unset theme visual density default

* Unset `SegmentedButton` border side default

* `TextField.hint_text` should be displayed if `label` is not specified
…et-dev#4526)

* `BorderSideStrokeAlign` should inherit from float

* properly parse `Chip.border_side`
…et-dev#4556)

* ControlState: rename "" to "default"

* resolve ControlState on user-defined order

* fix failing tests

* remove breaking line
@syleishere
Copy link
Contributor Author

Alright that worked, created a branch called popscope, allowed me do another PR to same repo. Done...

@syleishere
Copy link
Contributor Author

Fixed race condition that happened when tracks switch and duration not updated yet, added buffer in seconds as someone may find it useful. Updating example with buffer, done with this PR, Merry Christmas.

import random
import flet as ft


def main(page: ft.Page):
    page.theme_mode = ft.ThemeMode.LIGHT
    page.title = "TheEthicalVideo"
    page.window.always_on_top = True
    page.spacing = 20
    page.horizontal_alignment = ft.CrossAxisAlignment.CENTER

    def handle_pause(e):
        video.pause()
        print("Video.pause()")

    def handle_play_or_pause(e):
        video.play_or_pause()
        print("Video.play_or_pause()")

    def handle_play(e):
        video.play()
        print("Video.play()")

    def handle_stop(e):
        video.stop()
        print("Video.stop()")

    def handle_next(e):
        video.next()
        print("Video.next()")

    def handle_previous(e):
        video.previous()
        print("Video.previous()")

    def handle_volume_change(e):
        video.volume = e.control.value
        page.update()
        print(f"Video.volume = {e.control.value}")

    def handle_playback_rate_change(e):
        video.playback_rate = e.control.value
        page.update()
        print(f"Video.playback_rate = {e.control.value}")

    def handle_seek(e):
        video.seek(10000)
        print(f"Video.seek(10000)")

    def handle_add_media(e):
        video.playlist_add(random.choice(sample_media))
        print(f"Video.playlist_add(random.choice(sample_media))")

    def handle_remove_media(e):
        r = random.randint(0, len(video.playlist) - 1)
        video.playlist_remove(r)
        print(f"Popped Item at index: {r} (position {r+1})")

    def handle_jump(e):
        print(f"Video.jump_to(0)")
        video.jump_to(0)

    def handle_track_changed(e):
        index = int(e.data)
        print(f"Track changed to index: {index}, Currently Playing {SONG_LIST[index]}")

    def handle_position_changed(e):
        print(f"Position is {e.position} seconds, duration is {e.duration} seconds, "
              f"buffer is {e.buffer} seconds, percent done is {e.percent}%")

    def handle_error(e):
        print(f"Video error: {e}")

    SONG_LIST = [
        "https://user-images.githubusercontent.com/28951144/229373720-14d69157-1a56-4a78-a2f4-d7a134d7c3e9.mp4",
        "https://user-images.githubusercontent.com/28951144/229373718-86ce5e1d-d195-45d5-baa6-ef94041d0b90.mp4",
        "https://user-images.githubusercontent.com/28951144/229373716-76da0a4e-225a-44e4-9ee7-3e9006dbc3e3.mp4",
        "https://user-images.githubusercontent.com/28951144/229373695-22f88f13-d18f-4288-9bf1-c3e078d83722.mp4",
        "https://user-images.githubusercontent.com/28951144/229373709-603a7a89-2105-4e1b-a5a5-a6c3567c9a59.mp4",
    ]
    sample_media = []
    headers = {
        "User-agent": "mobile",
        "Authorization": "Bearer somelongtoken",
        "email": "[email protected]",
    }
    extras = {
        # audio_service configurables: title,artist,artUri,album,genre,displayTitle,displaySubtitle,displayDescription
        "artUri": "https://mp3.sunsaturn.com/crystaltunes.png", # background url for notification bar
    }
    for i in range(0, len(SONG_LIST)):
        if page.web:
            sample_media.append(ft.VideoMedia(SONG_LIST[i], extras=extras))
        else:
            sample_media.append(ft.VideoMedia(SONG_LIST[i], http_headers=headers, extras=extras))

    page.add(
        video := ft.Video(
            expand=True,
            playlist=sample_media,
            playlist_mode=ft.PlaylistMode.LOOP,
            fill_color=ft.Colors.BLUE_400,
            aspect_ratio=16/9,
            volume=100,
            autoplay=False,
            throttle=1000, # throttle on_position_changed this many ms(default 1000ms/1s)
            filter_quality=ft.FilterQuality.HIGH,
            muted=False,
            on_loaded=lambda e: print("Video loaded successfully!"),
            on_enter_fullscreen=lambda e: print("Video entered fullscreen!"),
            on_exit_fullscreen=lambda e: print("Video exited fullscreen!"),
            on_error=handle_error,
            on_track_changed=handle_track_changed,
            on_position_changed=handle_position_changed,

        ),
        ft.Row(
            wrap=True,
            alignment=ft.MainAxisAlignment.CENTER,
            controls=[
                ft.ElevatedButton("Play", on_click=handle_play),
                ft.ElevatedButton("Pause", on_click=handle_pause),
                ft.ElevatedButton("Play Or Pause", on_click=handle_play_or_pause),
                ft.ElevatedButton("Stop", on_click=handle_stop),
                ft.ElevatedButton("Next", on_click=handle_next),
                ft.ElevatedButton("Previous", on_click=handle_previous),
                ft.ElevatedButton("Seek s=10", on_click=handle_seek),
                ft.ElevatedButton("Jump to first Media", on_click=handle_jump),
                ft.ElevatedButton("Add Random Media", on_click=handle_add_media),
                ft.ElevatedButton("Remove Random Media", on_click=handle_remove_media),
            ],
        ),
        ft.Slider(
            min=0,
            value=100,
            max=100,
            label="Volume = {value}%",
            divisions=10,
            width=400,
            on_change=handle_volume_change,
        ),
        ft.Slider(
            min=1,
            value=1,
            max=3,
            label="PlaybackRate = {value}X",
            divisions=6,
            width=400,
            on_change=handle_playback_rate_change,
        ),
    )


ft.app(main)

@FeodorFitsner
Copy link
Contributor

So, this PR implements an ability to add background music for a silent video? Just trying to imagine how all that works without installing your PR on a real device :)

@syleishere
Copy link
Contributor Author

syleishere commented Dec 16, 2024

It adds ability to play audio or video in background, I normally just load mp3s and hide the video player till I want to play a video, but only way to have it keep playing on android with phone screen off or backgrounded, needs a foreground service with a notification to do that which audio_service provides.

You can just run it on an emulator for testing, I just hit the play button on android studio and select an emulator to use. When its installed on emulator, you can long press the app, go to permissions and just manually enable everything, every subsequent run of the app will run with full permissions. Only reason it would not work is if you haven't given it permissions on app itself, or your AndroidManifest.xml is not matching one from PR.

When you can drag your mouse from the top down and see the audio_service notification, you have everything running correctly, i'd edit the SONG_LIST and use normal length mp3's for testing if your using the example.

Here's a few links you can use with audio only:
"https://mp3.sunsaturn.com/soshy.mp3",
"https://mp3.sunsaturn.com/viper.mp3",
"https://mp3.sunsaturn.com/lonely.mp3",
"https://mp3.sunsaturn.com/eclipse.mp3",

Without that audio_service plugin, flet users would not stand a chance of competing against other music players on play store, as they all use it.

@syleishere
Copy link
Contributor Author

syleishere commented Dec 17, 2024

Google_TV
macos_on_android_emulator
Pixel8_real_device
windows_on_android_emulator

1) Google TV
2) MacOS with android emulator
3) My real Pixel 8 Phone
4) Windows with android emulator

I would add IOS emulator to, but media_kit is known to have issues with sound there, so can only be done on real iphone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants