Dictation

This page describes the tentative scheme for implementing dictation and speech input for the desktop using Google voice actions on an android phone.

Connection

The client is the android phone and the server is the desktop client in the TeclaShield repository.

1. The communication will happen over WiFi LAN as that will allow bluetooth to be free to connect to shield on android as well.
2. The Steps for establishing connection are as follows :

• Server will broadcast a packet to notify clients about the it existence.The UDP packets are sent over multicast sockets .Since communication over UDP is unreliable multiple packets will be sent.
• The android client will receive the packets and send an acknowledgement in a similar fashion.The Server's IP address is thus obtained dynamically and can be used to connect over a basic TCP socket.
• When the server receives the acknowledgement packet,it responds by sending a success packet and start a TCP serversocket to listen for clients.
• Android client connects to the server and sends a passkey which the server compares it with its value of passkey. Passkey serves the purpose of resolving between multiple servers in the same network.
• The passkey will have a default value which will be set on a fresh installation to not complicate things for users. The passkey can be set by the user and will be stored in the application preferences thus user will not need to enter the passkey every time.

3. Application will look for server and will attempt reconnection periodically similar to the way bluetooth connections occur now.

Implementation

1. Use a button(mostly on the secondary navigation keyboard) to invoke Google voice actions.A list of possible speech results will be presented to the user to choose from in a dialog which can then be used for selecting an appropriate choice.
2. The choices will be presented in a dialog and can have extra buttons to choose the required window and text field (firing Alt+Tab and Tab key-events respectively) on the desktop for the dictated text to be pasted into.
3. Relay switch events received on android to the desktop client. Some suggestions required here concerning when to relay and when not to relay.
4. TeclaKeyboard to be used for typing on the desktop.
5. Differentiating between dictation and key presses and switch events can be done by adding a known prefix like dictate: for dictation string command: for switch events and keypress: and keyrelease: for Teclakeyboard events.

Important Note:

In order to make the communication over WiFi properly, Allow UDP And TCP Ports form 28195-28197 in your system's firewall.