Add support for 'missing' RPCs

[gavanderhoorn]

Edit: for future readers: please check the main readme for information on which RPCs are already supported.

We're still looking for contributions to add support for the currently unsupported RPCs.

---

comet_rpc currently only supports a subset of the RPCs COMET offers.

Implementing the missing ones should not be too difficult:

1. figure out their names + (required) arguments (using Wireshark fi, or by looking at the Javascript)
2. their behaviour
3. returned response document (JSON)
4. add class to messages.py
5. implement wrapper function in comet.py

[gavanderhoorn]

I'd gladly accept PRs adding support for new RPCs.

[rastreus]

1. figure out their names + (required) arguments (using Wireshark fi, or by looking at the Javascript)

• Can this be accomplished with a Virtual Robot (ROBOGUIDE, OlpcPro), or does it require a physical controller?
• I have little experience with Wireshark. What steps would properly determine an RPC API endpoint name and arguments?

[gavanderhoorn]

1. figure out their names + (required) arguments (using Wireshark fi, or by looking at the Javascript)

• Can this be accomplished with a Virtual Robot (ROBOGUIDE, OlpcPro), or does it require a physical controller?

Yes, you can use Roboguide for this. The FRVC it uses essentially runs the same software.

• I have little experience with Wireshark. What steps would properly determine an RPC API endpoint name and arguments?

All the RPCs I've seen so far use simple GET requests, so all arguments appear in the URL.

Example:

http://<CONTROLLER_IP>:80/COMET/rpc?func=REGVALRD&index=1

There might be some RPCs which take an actual JSON request document via POST, but I haven't encountered them. So it's mostly response documents which are JSON.

As to wireshark itself: you don't absolutely have to use it. You could use any HTTP-level traffic capture tool. Perhaps even your browser's developer tools.

Then browse to the main website of your robot's controller (ie: http://localhost if running a virtual controller in Roboguide), and open iRProgrammer (if you need to, in Roboguide, install J767 on your virtual controller).

At that point start using the interface (ie: iRProgrammer) and keep an eye on your sniffer. You should see the requests as they are generated.

[gavanderhoorn]

I've just added an overview of "all" (?) RPCs and whether they're supported already or not. See Supported RPCs in the readme.

[anandmit93]

Hello @gavanderhoorn ,

I tried inspecting the RPCs via developer tools and found some more information about them. I have tested them for V9.30 and V9.40

IODRYRUN = "71" - Simulate all IOs
IOWETRUN = "72" - Unsimulate all IOs
XMLCOPY = "235" - Copy XML from one location to another 
Arguments:  from_path_name – Source file path; to_path_name – Destination file path
MNCPYPRG = "250" - Copy a TP program 
Arguments: src_name=source TP program name; chg_name=Copy of TP program name
SCDELETE = "15" - Delete lines in a TP program 
Arguments: prog_name= name of TP program; lin_num=Line number; &num=Number of lines to delete
MMDELPOS = "249" - Delete position data in a TP program (works only for Position P and not PR)
Arguments : prog_name= TP program name; pos_num= position index
SCEDIT = "29" - Add or replace line in a TP program 
Arguments: prog_name=TP program name; in_buf = text in url encoding; lin_num=linenumber to insert or replace; opt_sw=0/1/2 where 0 - overwrite 1 - Insert at that line, 2 - Insert after that line
SCSETPOS = "29" - Add or replace position data in a TP program Arguments: prog_name= TP program name; in_buf=position data in ASCII format 
TXLSTPRG_FC = "163" - List all programs 
Arguments: datatype=22 – types of acquired data – program name only/program name with comment..; prg_typ = 1 – tp/karel..; sub_typ=0 (unspecified/macro/condition/collection..)

MMDELPRG = "6" - Delete TP program 
Arguments: prog_name= Name of TP program
MMRENPRG = "116" - Rename a TP program 
Arguments: src_name = Name of TP program; chg_name = New name
MMSETATR = “21” – Set program attributes
Arguments: prog_name = Name of TP program ; attr = Attribute type ( comment /  write protection…); in_buf = value of the attribute
MMGETATR = “20” – Get program attributes
Arguments: prog_name = Name of TP program ; attr = Attribute type ( comment /  write protection…)

MNCHGREP = “243” – Change Position data representation
Arguments: in_buf = position data in ASCII format
MMCHGTYP = “23” – Change program subtype
Arguments: prog_name = Name of TP program ; sub_type =0 (unspecified/macro/condition/collection…)
REMARKLIN = “215” – Remark or unremark line(s)
Arguments: prog_name= Name of TP program; start = line number start; end = line number end; remark = 0/1 (unremark/remark)

[gavanderhoorn]

Hi, thanks.

I should perhaps have been clearer here: I believe we have all those RPCs in the readme:

comet_rpc/README.md

Lines 143 to 229 in dcb6747

	| Name | Description | Supp.? | 8.x | 9.x |
	|:------------------|:----------------------------------|:------:|:-----|:-----|
	| CHGOVRD | Change override | Y | | .40+ |
	| CKTRKPRG | Check linetrack attributes | N | | .40+ |
	| CLLB_CODE_REQ | | N | | .40+ |
	| CLLB_PAYLOAD_CONF | | N | | .40+ |
	| CPKCL | Execute KCL command | Y | | .30+ |
	| DCS_CHECK_APPLY | | N | | .40+ |
	| DCS_CHECK_CODE | | N | | .40+ |
	| DCS_VRFY_REQ | | N | | .40+ |
	| DPEWRITE_STR | Retrieve error code description | Y | | .10+ |
	| DPREAD | Read element from dictionary | Y | .30+ | |
	| ERPOST | Post an error to the log | N | | .10+ |
	| EXEC_TXCMND | | N | .10+ | |
	| GET_FORM | | N | .10+ | |
	| GET_RAW_FILE | Get raw byte contents of file | Y | | .10+ |
	| GETFOCUS | | N | | .10+ |
	| GTFILIST | Get list of files in directory | Y | | .40+ |
	| GTMCRLST | Get list of macros | Y | | .10+ |
	| GTPIDLST | Get list of posregs in TP program | Y | | .40+ |
	| IOASGLOG | Update IO configuration | Y | | .40+ |
	| IOCKSIM | Check simulated status of IO port | Y | .30+ | |
	| IODEFPN | Set/update comment on IO port | Y | | .10+ |
	| IODRYRUN | Treat all IO as-if simulated | N | | .40+ |
	| IOGETASG | Retrieve IO configuration | Y | | .40+ |
	| IOGETHDB | Retrieve the "HW database" | Y | | .40+ |
	| IOGETPN | Retrieve comment on IO port | Y | | .10+ |
	| IOGTALL | Read IO ports, batch-wise | Y | | .30+ |
	| IOSIM | Set IO port to simulated | Y | .30+ | |
	| IOUNSIM | Clear simulated state of IO port | Y | .30+ | |
	| IOVALRD | Read IO port | Y | .30+ | |
	| IOVALSET | Write to IO port | Y | .30+ | |
	| IOWETRUN | Stop treating all IO as simulated | N | | .40+ |
	| LOCAL_PAUSE | | N | | .10+ |
	| LOCAL_START | | Y | | .10+ |
	| MG_RECPOS | | N | | .40+ |
	| MMCHGTYP | Change the type of a program | N | | .10+ |
	| MMCREMN | Create a TP program | N | | .10+ |
	| MMDELPOS | Remove a position from a program | N | | .10+ |
	| MMDELPRG | Delete a program | N | | .10+ |
	| MMGETATR | Read program attribute | N | | .10+ |
	| MMGETTYP | Read 'program type' (TP, PC, etc) | Y | | .10+ |
	| MMRENPRG | Rename a program | N | | .10+ |
	| MMSETATR | Write program attribute | N | | .10+ |
	| MNCHGREP | Convert position representation | N | | .10+ |
	| MNCPYPRG | Copy a program | N | | .10+ |
	| OSSNDPKT_EXT | | N | | .10+ |
	| PASTELIN | Duplicate/move lines in a TP prog | Y | | .40+ |
	| PGABORT | Abort all/a specific program(s) | Y | | .10+ |
	| PMCUPFN | | N | | .10+ |
	| PMCUPRQ | | N | | .10+ |
	| PMCVALRD | | N | | .10+ |
	| PMON_CAN_PKT | | N | .10+ | |
	| PMON_DISCONNECT | | N | .10+ | |
	| PMON_GET_PKT | | N | .10+ | |
	| PMON_START_MON | | N | .10+ | |
	| PMON_STOP_MON | | N | .10+ | |
	| PMON_VERIFY_PKT | | N | .30+ | |
	| POSREGVALRD | Read a position register | Y | | .10+ |
	| RECPOS | Teach position (in program) | N | | .10+ |
	| REGVALRD | Read a register (int/real) | Y | | .10+ |
	| RPRINTF | Print to the controllers conslog | Y | .10+ | |
	| RUN_TASK | Start a program on the controller | N | | .30+ |
	| SCDELETE | Delete line from program | N | | .10+ |
	| SCEDIT | Add/replace line to/in program | N | | .10+ |
	| SCGETPOS | Get position from program | N | | .40+ |
	| SCSETPOS | Update position in program | N | | .10+ |
	| SET_FORM | | N | .10+ | |
	| SKIP_LINE | Change active line in paused prog | N | | .40+ |
	| TPEXTREQ | | N | .10+ | |
	| TPLINK_DISCONNECT | | N | | .10+ |
	| TPLINK_NEW_URL | | N | .10+ | |
	| TPMODE_CHG | | N | | .40+ |
	| TPMULTI_TASKIDX | | N | .30+ | |
	| TPXENSBV_KRL_EXT | | N | | .10+ |
	| TPXENSBV_KRL_TEXT | | N | | .10+ |
	| TPXENSUB_EXT | | N | | .10+ |
	| TPXFILSB_EXT | | N | | .10+ |
	| TPXPRGSB_EXT | | N | | .10+ |
	| TXCHGPRG | Open (and make active) a TP prog | Y | | .10+ |
	| TXLSTPRG_FC | List programs (specific types) | N | | .10+ |
	| TXML_CURANG | Return current joint angles | Y | | .10+ |
	| TXML_CURPOS | Return current TCP pose (XYZWPR) | Y | | .10+ |
	| TXSETLIN | Open TP prog at specific line | Y | | .10+ |
	| VMIP_READVA | Read a (system) variable | Y | | .10+ |
	| VMIP_WRITEVA | Write to a (system) variable | Y | .30+ | |
	| XMLCOPY | Copy an XML file to another | N | | .30+ |

What we need is for them to be supported by the library.

---

Edit: looks like REMARKLIN is actually new.

@anandmit93: if you could compare what you found with the table I linked, could you submit a PR adding the ones that are missing? Thanks.

Could you also mention the system software versions you tested these against?

You already wrote V9.30 and V9.40, but that's not specific enough. You should be able to find the exact version nr on the system summary page of the robot's web server. It should look something like V9.30P/26, or 9.30xxx.

[anandmit93]

Sorry I forgot to mention the complete versions V9.30P/20 and V9.40P/35 & V9.40P/43. I guess only REMARKLIN seems to be missing in the list. Anyways, the support for these RPCs could be extended in this library.

[gavanderhoorn]

Anyways, the support for these RPCs could be extended in this library.

agreed.

Using the existing implementation(s) as a template should make it not too difficult to get something working.

If you'd be up for it, I'd be more than willing to review some PRs getting support for some of those in.

Maybe start with REMARKLIN? That seems simple enough.

[anandmit93]

Sure, I will give it a try! I will update you after some progress has been made.

[gavanderhoorn]

From your description of the arguments, perhaps paste_line could be a good template:

comet_rpc/src/comet_rpc/comet.py

Lines 775 to 817 in 15f3939

	def paste_line(
	server: str,
	prog_name: str,
	select_start: int,
	select_end: int,
	insert_at: int,
	oper: PasteLineOper,
	) -> PasteLinResponse:
	"""Copy or cut lines `[start, end]` in TP program `prog_name` to line `insert_at`.
	NOTE: COMET will insert the copied/cut line(s) *after* the line at `insert_at`. In
	effect, this makes `insert_at` 0-based, whereas `select_start` and `select_end`
	are 1-based.
	:param server: Hostname or IP address of COMET RPC server
	:param prog_name: Name of the program to alter
	:param select_start: Start of region to select for copy/cut operation (1-based)
	:param select_end: End of region to select for copy/cut operation (1-based)
	:param insert_at: Line number to paste to (0-based)
	:param oper: The operation to perform: copy (duplicate) or cut (move)
	:returns: The parsed response document
	:raise InvalidArgumentException: If `select_end < select_start` or if `oper` is
	an invalid value
	:raise NoSuchLineException: If `insert_at` is not a valid line nr in `prog_name`
	:raises UnexpectedRpcStatusException: on any other non-zero RPC status code
	"""
	response = _call(
	server,
	function=RpcId.PASTELIN,
	prog_name=prog_name.upper(),
	start=select_start,
	end=select_end,
	insert=insert_at,
	opt_sw=oper.value,
	)
	ret = response.RPC[0]
	if ret.status == ErrorDictionary.MEMO_027:
	raise NoSuchLineException(f"insert_at: {insert_at}")
	if ret.status == ErrorDictionary.HRTL_022:
	raise InvalidArgumentException()
	if ret.status != 0:
	raise UnexpectedRpcStatusException(ret.status)
	return ret

[anandmit93]

Hello @gavanderhoorn ,

I have just raised a PR for 3 RPC functions. Please have a look at it.

[ghltrdi567]

Hello @gavanderhoorn ,

I tried inspecting the RPCs via developer tools and found some more information about them. I have tested them for V9.30 and V9.40

IODRYRUN = "71" - Simulate all IOs
IOWETRUN = "72" - Unsimulate all IOs
XMLCOPY = "235" - Copy XML from one location to another 
Arguments:  from_path_name – Source file path; to_path_name – Destination file path
MNCPYPRG = "250" - Copy a TP program 
Arguments: src_name=source TP program name; chg_name=Copy of TP program name
SCDELETE = "15" - Delete lines in a TP program 
Arguments: prog_name= name of TP program; lin_num=Line number; &num=Number of lines to delete
MMDELPOS = "249" - Delete position data in a TP program (works only for Position P and not PR)
Arguments : prog_name= TP program name; pos_num= position index
SCEDIT = "29" - Add or replace line in a TP program 
Arguments: prog_name=TP program name; in_buf = text in url encoding; lin_num=linenumber to insert or replace; opt_sw=0/1/2 where 0 - overwrite 1 - Insert at that line, 2 - Insert after that line
SCSETPOS = "29" - Add or replace position data in a TP program Arguments: prog_name= TP program name; in_buf=position data in ASCII format 
TXLSTPRG_FC = "163" - List all programs 
Arguments: datatype=22 – types of acquired data – program name only/program name with comment..; prg_typ = 1 – tp/karel..; sub_typ=0 (unspecified/macro/condition/collection..)

MMDELPRG = "6" - Delete TP program 
Arguments: prog_name= Name of TP program
MMRENPRG = "116" - Rename a TP program 
Arguments: src_name = Name of TP program; chg_name = New name
MMSETATR = “21” – Set program attributes
Arguments: prog_name = Name of TP program ; attr = Attribute type ( comment /  write protection…); in_buf = value of the attribute
MMGETATR = “20” – Get program attributes
Arguments: prog_name = Name of TP program ; attr = Attribute type ( comment /  write protection…)

MNCHGREP = “243” – Change Position data representation
Arguments: in_buf = position data in ASCII format
MMCHGTYP = “23” – Change program subtype
Arguments: prog_name = Name of TP program ; sub_type =0 (unspecified/macro/condition/collection…)
REMARKLIN = “215” – Remark or unremark line(s)
Arguments: prog_name= Name of TP program; start = line number start; end = line number end; remark = 0/1 (unremark/remark)

Hello, @anandmit93 !

This is really great info.
I can repeat all neсessnary RPC`s exept SCSETPOS = "29" and MNCHGREP = “243”.
Could you write valid URL querry examples of this RPC?
Thank you in advance!

[gavanderhoorn]

Would be great to add some of these which are currently unsupported to the library @ghltrdi567.