Make parttool.py / otatool.py more accessible (IDFGH-14594)

[nebkat]

Is your feature request related to a problem?

Virtually every single "ESP32 firmware flashing guide" involves downloading a binary and esptool.py write_flash-ing to an arbitrary address - which as developers we know is typically the location of the factory or ota_0 partition.

Best case scenario factory is at 0x100000 and is the same across all devices, but unfortunately this is not always the case. Aside from being hard to memorize, making a typo in the address can result in data being lost, and extra steps are needed when dealing with devices running ota_1.

The partition table of course provides the necessary information to simplify things but the ability to read it is not included in esptool.py (as it is technically a feature of esp-idf rather than ESP32 in general).

parttool.py and its cousin otatool.py have existed for over 5 years and are the solutions to the above problem, but in my experience they are not very well known and suffer from a number of usability issues that make it quicker and easier to just use esptool.py.

Describe the solution you'd like.

1. Remove dependency on esp-idf and make parttool and otatool available as standalone tools, similar to esp_idf_monitor.
   • This would likely involve incorporating esptool as a Python dependency rather than invoking it as a subprocess
   • Much easier to set up on non-dev machines
   • Ideally provide PyInstaller version

---

2. Use single esptool connection + stub flash
   • Obtaining the partition table and performing an operation are completed using two separate calls to esptool.py
   • Each call repeats the connection process and stub flashing
   • Flashing using otatool involves 4(!) separate operations

---

3. Improve/simplify command names
   • The tool name already implies the operation being performed
   • parttool write_partition should be parttool write, etc
   • parttool get_partition_info should be parttool info
   • otatool write_ota_partition should be otatool write or ota, etc
   • otatool read_otadata should be otatool info (otadata is an implementation detail)

---

4. Improve/simplify arguments
   • --input / --output should be positional arguments
   • parttool --partition-name ota_0 and otatool --name should be a positional argument by default (unless other distinguishing args are specified)
   • otatool should default to the "next update partition" (i.e. esp_ota_get_next_update_partition)

---

5. Support multi-partition operations
   • esptool allows flashing to multiple different addresses at once
   • parttool write factory firmware.bin data data.bin would be quite convenient (though admittedly more confusing since names are used rather than addresses)

---

6. Allow flashing partition table and bootloader
   • Virtual bootloader partition at 0x1000
   • Virtual partition_table partition at 0x8000 (by default, already configurable in parttool)

---

7. Provide a partition table summary command
   • parttool summary could return a CSV of the current partition table

---

8. Introduce a merge_bin equivalent for partitions.
   • Super convenient for bundling multiple binaries together
   • app + preconfigured data
   • otadata empty + ota_0 all in one (though of course otatool is better for this)

---

9. Decouple build system from partition table
   • The build system is highly reliant on the configured partition table
   • The contents of the partition table are not directly used by any other component
   • It is mainly to determine flashing offsets and check for size overflows at CMake level
   • In our application we have multiple devices running the same firmware but different partition layouts
   • This makes it more difficult to manage such setups
   • Ideally there would be an option to flash using parttool and assume a dynamic partition table

---

10. Include similar feature in esptool-js and the Flash Download Tool

Describe alternatives you've considered.

idf.py PART-flash is sufficient for developers in most cases during active development.

Additional context.

• Make it possible to idf.py flash into ota_1/ota_2 partitions after build (IDFGH-11869) #12956