SVD files : missing the core peripherals

[Jegeva]

Hello,

multiple vendors (keil, ST, etc...) refuse to include the core peripherals in theirs SVDs, claiming that it is arm's job (see https://github.com/posborne/cmsis-svd/issues/28) even arm relying on keil's (#48).

The old cmsis's svd had the core peripheral and on CMSIS_5 the commit removing them claims it was "cleaning" (commit 9d12a19), i can understand it for the header files but not really for the SVDs (well being SVDs they should describe the core peripherals for the cores because that's what SVDs are for).

Not having machine parse-able sources for this very (pun intended) core data is extremely annoying.

[JonatanAntoni]

Hi @Jegeva,

the root cause for this forth and back is basically that SVD isn't designed with a modular approach. Today every device has exactly one single SVD file associated that describes all the peripherals. The core peripherals are ignored. Most tools cope with the core stuff in a hard coded way.

It should be possible to come up with SVD files covering the default core peripherals. We need to keep these definitions separate from the SVD templates in Device/ARM/SVD.

Cheers,
Jonatan

[Jegeva]

@JonatanAntoni

Hi Jonatan,

Yes, i understand the non modular nature of the SVDs but :

• the current state of 'things' is that the core peripherals should be in the vendors svds but are not.
• the nature of the SVDs and logic should actually point towards the core periphs. being in the SVDs for the core ( if you look at the content of the core SVDs, without the periphs., they seem to only indicate that 0->0xffffffff is RW... so their usefullness is limited to say the least...)

from my point of view, i see multiple cases where that can be beneficial

• softcores (not my use case really but i can see this helping some people)
• header generation for chips where the svd is incomplete (yes, maybe not the core target of SVDs but at least it is one use for them besides 'yes, the 32bits address space exists')
• code analysis tools
• etc.

If there could be something somewhere where they could be available to soften the 'hard coding' of these in tools that would be great :) (on a side note, 'tools usually hardcode that' looks more for an argument in favor for inclusion than against it to be honest) It is possible to apply some git to get the old ones with the peripherals but for 35P and 55 which is sad.

sincerely

jg

[JonatanAntoni]

Hi @Jegeva,

the situation is a bit more complicated than that.

Silicon vendors use the SVD files (together with SVDConv utility) to generate device header files. The core peripherals must not be (re)defined in device specific header files because these are already part of the generic CMSIS-Core headers.

The SVD files contained in this repo (Device/ARM/SVD) are "empty" templates. As its supposed to be used as a starting point by Silicon vendors we should not add the core peripherals into these files. Otherwise it might happen that we get a redefinition for core peripherals.

The actual register set of core peripherals in Cortex-M controllers is not too big. Hence is should not be a big deal to put together separate SVD files for those (35P and 55 share major parts).

Cheers,
Jonatan

[Jegeva]

@JonatanAntoni

Jonatan, would it be possible to add something like Device/ARM/SVD_full with the complete SVDs in there ?

• That shouldn't mess with vendors toolset while allowing people with less resources at hand to parse them easily (for ex. w/ https://github.com/posborne/cmsis-svd).
• Everything should be available at ARM already no ?

[JonatanAntoni]

Hi @Jegeva,

As discussed earlier, e.g. in cmsis-svd/cmsis-svd-data#20, we don't maintain SVD files for core peripherals (as part of CMSIS or in Arm). And even the template SVD's in earlier version didn't contain all the core peripherals. One would need to spent the time to put this together, e.g. from the TRM's or the core headers.

I agree, we could put the SVDs somewhere in this repo. But perhaps its more reasonable to keep them next to cmsis-svd. I confirm that keeping this separately would not create a mess. But it might be confusing because most tools cannot cope with multiple SVD files at the same time (for a single device). So in order to actually make use of separate core SVDs you need proper support in the tools as well.

Cheers,
Jonatan

[Jegeva]

@JonatanAntoni

Well it's a shame, maybe you have some machine parseable data that describe them in another format that SVD that i could use to publish these svds ? It seems really backwards in 2020, especially in the light of SVD existing as a format, not to have this for a platform as pervasive as as ARM.

[Jegeva]

@JonatanAntoni

hummm... apparently ARM does maintain full fledged SVDs in the CMSDK (they are integrated in keil's BSP packs pointed by reinhard keil in cmsis-svd/cmsis-svd#46 that is available publicly on their repo on Azure and the licence headers in the files are ARM's). Would it be possible to integrate in a SVD_full (or such) directory in the CMSIS ? That'd be great :)

[JonatanAntoni]

Hi @Jegeva,

the files you are talking about are device specific for MPS2. The described peripherals are MPS2 ones, not the core ones. I doubt these files aren't actually what you are looking for. I am sorry, we do not maintain something suitable and there are no plans to do so.

Cheers,
Jonatan

[flit]

@JonatanAntoni

Fwiw and afaik, the cmsis-svd project is no longer being maintained. (I've copied the SVD parsing code into pyocd so there is no dependency on cmsis-svd, which will pull in gigabytes of SVD files. At some point I'd like to improve the parser. It's quite slow, and the resulting objects could be easier to use.)

Storing core peripheral SVDs in the CMSIS_5 repo makes more sense to me. And yes, most tools cannot handle multiple SVD files right now. But to a large extent that is because there is no need to. If standard core peripheral SVD files were made available in CMSIS, then tools would have a reason to support multiple SVD files. We have to start somewhere to make progress.

Also, the problem with not wanting core peripherals in device SVDs because of header generation can be solved relatively easily. We can add an attribute to the top level XML elements for core peripherals that identify them as such. Then SVDConv can avoid generating output for peripherals with that attribute.

[ilg-ul]

Many years ago, when I visited Keil/Arm in Germany, we discussed the issue of system peripherals and the conclusion was that Arm should define separate SVD files with their definitions. As far as I can tell, it never happened.

I'd still prefer the solution with two separate files, one defined by the vendor and one defined by Arm.

I am aware that this solution requires some help from the tools developers, and I'm ready to update the GNU MCU Eclipse debugger to input the names of the two files.

The other solution, to ask all vendors to include the common definitions, is unrealistic, I guess all of them generate the xmls with some automated tools and changing something in the process is kind of mission impossible.

[flit]

@ilg-ul You're right, many silicon partners have their own in-house tools for generating header files from data. Often the same tool generates the SVD as well.

Glad to hear you're ready to update GME to address this. That's highly motivating to really address this, at least for me. 😄

[Jegeva]

i think it is relatively easy keep all sides happy either :
Sol1: by having the cpu elements in the svd definition accepting a "derivedfrom style" pointing to an arm provided set of core peripherals (but the possibility of breaking vendors toolsets)

or

Sol2: by having a separate 'SVD_full' directory in the CMSIS where parsers can find the definition of the core peripherals, delegating the responsability of integrating the peripherals to the tool implementors (based on the series attribute of the cpu element ,winkwinknudgenudge to solution 1), leaving the vendor tooling alone.

To be honest, even tho a lot of the tools are getting around the issue by hardcoding the core peripherals (and we all know how good hardcoding things is), there if a plethora of tool developers (gcc, kvasir, any rtos, etc.) whose life would be really eased by this.

[JonatanAntoni]

@Jegeva, Basically we should already have something like a "derived-from style" in place. Each SVD file has a <cpu> section. The given name is used to include the correct core_<name>.h header file from CMSIS-Core.

@ALL, We could discuss about adding core_<name>.svd files to ./CMSIS/Core/SVD next to the include folder. Ultimately, someone needs to spent the effort putting together those core SVD files. Volunteers welcome.

Unfortunately, there are currently no means to add those to the pack description. Thus tools would need to look up those files either manually or by naming convention (i.e. relative to core_<name>.h). Does this make sense to you?

Cheers,
Jonatan

[ilg-ul]

adding core_<name>.svd files to ./CMSIS/Core/SVD next to the include folder

That would be a first step forward.

there are currently no means to add those to the pack description ... naming convention

There aren't many such files, so a naming convention might be enough for start.

@JonatanAntoni, Can you make a more detailed proposal on how the file name can be derived from the existing definitions, and what would be the actual files in the CMSIS/Core/SVD folder?

[JonatanAntoni]

Hi @ilg-ul,

As an example for the new Cortex-M55, we could add ./CMSIS/Core/SVD/core_cm55.svd. This SVD file can describe all the "core peripherals" encoded in ./CMSIS/Core/Include/core_cm55.h. A tool that is aware of the location of the actually used core header file (core_cm55.h) can deduce the location of the according SVD file (core_cm55.svd).

Same applies to all other core headers we have today.

Cheers,
Jonatan

[ilg-ul]

Sorry if my questions are stupid (I did not use CMSIS recently), but is there any variable in the definition of a Cortex-M55 device that explicitly says 'cm55'?

If so, then I extrapolate that it should be easy to derive the SVD file name for any device.

[ilg-ul]

./CMSIS/Core/SVD/core_cm55.svd. This SVD file can describe all the "core peripherals" encoded in ./CMSIS/Core/Include/core_cm55.h

I did not check that header file, but just to be sure there is no misunderstanding, the use case I see for the new core SVD files is mainly to help debuggers that implement a peripheral registers viewer to show the system peripherals, missing in vendor SVDs.

[Jegeva]

@ilg-ul
normally ? yes, cf. https://github.com/ARM-software/CMSIS_5/blob/develop/CMSIS/Utilities/CMSIS-SVD.xsd => /device/cpu/name is an enumerated value

in practice? vendors not always adhere to the spec. looking at the svd corpus in cmsis svd, roughly 50% have the attribute.

[Jegeva]

@JonatanAntoni
hello jonatan,

is there some data at arm that could be easily transformed ? what are the dgugs generated from ?

[JonatanAntoni]

@Jegeva, no, unfortunately we would need to assemble these SVDs manually.

[ilg-ul]

roughly 50% have the attribute.

Does your SVD validator spot this condition?

[JonatanAntoni]

@ilg-ul, SVDConv creates a warning if cpu section is missing. The cpu section has been added with SVD 1.1. For backward compatibility reasons we made it optional. Hence older SVD files might miss this section. And newer files as well if not properly checked with SVDConv because they were never used for generating the device header file.

[Jegeva]

@ilg-ul

my "highly evolved svd validator" is just a bash loop with xmllint -xpath

j=0;k=0;for i in cmsis-svd/data/*/*;do echo -n $i' ';core=`xmllint -xpath 'string(/device/cpu/name)' $i`;rev=`xmllint -xpath 'string(/device/cpu/revision)' $i`; echo $core $rev $j $k;((j=j+1));[ -z "$core$rev" ] && ((k=k+1)) || echo; done;echo "$k / $j"

-> 268/667

[ilg-ul]

You mean 268 out of 667 include the information that we could use to automatically select the Arm provided SVD with the core peripherals? Not very encouraging...

In practical terms this means that the user must manually enter the path to the core_<name>.svd file.

[Jegeva]

@ilg-ul

no, the reverse , 268 have no device/cpu/name [ -z $val ] evaluate as true if val is empty (replace with -n to see the reverse)

additionally, 23 of the 268 have the CPUID peripheral in there already, meaning you can replace the cpu/name with a derived value of the reset state of CPUID.

for your use-case (debugging, i presume over JTAG) you only need to read SCB->CPUID (AFAIK, *0xE000ED00 on all M cores) of the actual chip to get the partno. Easily deriving the core version and revision (eg. http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0646c/CIHCAGHH.html for the M7 core) and then voila, you know the correct SVD to fuse.

i don't think your need more than knowing that you are talking with a coresight core to do that.

[ilg-ul]

read SCB->CPUID

yes, thank you, just that this is not practical, the debugger must be configured before starting the debug session.

[Jegeva]

making some progress :

https://gist.github.com/Jegeva/cdbca5763a2a424f0fd4d8f37f6ebb8b

m0,m0p,m3,m7,m23,m33 :
-core pripherals
-registers w/ adresses
-bit fields definitions

still missing

• the interrupts
• the coresight/TAP related registers (should that be in there anyways ?)
• M4 (since there is no online dgug, just a pdf 👎 ).
• FPU definitions for M33 (because i can't find a map with the register addresses anywhere)
• CMO definitions for the M7 (they are on the way, they documentation is articulated weirdly for them)

Scraping infocenter.arm.com makes me better understand why @JonatanAntoni was saying "we don't have plans for that", I think the documents are all actually redacted by a human and not generated uniformly, with attributes and ways to describe things varying for no reasons, sometimes between 2 registers of the same peripheral...🤷

[flit]

@ilg-ul I disagree, the debugger should actively perform discovery on the target rather than rely on a fixed configuration. That's kind of the point of CoreSight.

@JonatanAntoni Remember, we do have the data from DS; it only needs to be transformed to SVD format.

[Jegeva]

@flit

what data from DS ? is it shareable (in the end if it is in the svd it should be no ?) cause scraping infocenter is kind of annoying and very brittle...

[ilg-ul]

the debugger should actively perform discovery on the target rather than rely on a fixed configuration. That's kind of the point of CoreSight.

I generally agree, just that implementing this in the Eclipse Debug framework is not trivial.

[flit]

@Jegeva DS is Arm Developer Studio.

[Jegeva]

just FYI i am working on this don't close the issue due to inactivity

[Jegeva]

@JonatanAntoni

hello jonathan,

i progressed quite a bit now and i think i came to a crossroad regarding how we are going to implement the core peripherals in the SVD. This question stems from the fact that the cmsis-svd schema doesn't support peripheral/registers that have no address/offset.

I see multiple possible solutions (in order of cleanliness) :

1. change baseaddress in peripheral and offset in register to have minOccurs="0", implying that this register can only be referenced by instruction if the address/offset is not here
2. define new types (coreperipheral,coreregister) that will basically be clones of the classical type
   without offset and baseaddress.
3. stuff this in vendor extension without changing the schema

from my point of view,

1. makes the most sense and is the cleanest, the change in schema is very minimal, takes care of the core registers and the FPU/Neon cases and the impact on existing tools would be virtually inexistant (only tools interested in generating/parsing a core svd need to take the case into account)
2. it is kind of clunky, impact on existing tools would be higher than 1) (ie. a tool with very hard-coded things but we run the risk of a tool running on an old schema version running into an unknown tag) without any additional value but the adaptation of an existing tool for this would require more work than just reusing the existing code (that 1) helps with)
3. IF the code is bad enough to be broken by 1) or 2) this will also probably break it, it does into the "dirty hack" zone

My current development goes more into the 1) direction (for obvious reasons, its cleaner from the start, requires almost no schema alteration, allows all the existing codebase of other tools to be leveraged and really helps with the fusion of a core and a vendor svd as a complete representation.

what do you think ?

[jkrech]

Hello Jegeva,

I do like the pragmatism in proposal 1. Semantically I think it would be better to add a new section under . In my view the Core registers (r0 - r15, fpu / neon, etc.) must not be mixed with Core Peripherals, like NVIC, SYSTICK, etc. as you rightly state, they are not memory mapped and are accessible by instructions and dedicated accesses via the CoreSight debug logic.
This however also raises the question, what a debugger is going to do with this description listing register names without a "recipe" how to access them.

The name "System View Description" was intentionally chosen as an extension to an existing "Core View".

[Jegeva]

@jkrech

Hello Joachim,

I can see the merit of the new section ( so 2) ) and this direction is completely ok with me (very trivial to implement in the current state of my development)... so how should we name them ?

Regarding the usage interest :

• I think it is important to stress that "core" SVDs can be consumed by a variety of tools, not limited to debuggers (generating headers, supplementing vendor SVDs that lack some core peripherals, vulnerability research, parametric research for core "fitness for purpose", documentation generation, learning, etc...)
• Some core register behaviours vary between core : for ex the definition of the flags in xPSR for an M0 and an M7 is completely different and this should be available in a machine parseable source (honestly i think there could hardly be a better place than the system view description).

Side note:
could you point me to a source to the core view svd is an extension to ?

[jkrech]

The "core view" is traditionally managed by the debug tools in proprietary ways and there was no particular drive to "standardize" this. The reason for inventing SVD was because the device specific details are best known and managed by the device vendors and not the debug tool vendors. The amount of different ARM Cortex-M processors is significantly smaller than devices and therefore much easier to manage.

As to the approach, we should review if we could simply add an optional tag/section
<registers>
as a child for the <cpu> element.
Semantically the <cpu> would be similar to <peripheral> (parent). And we would need to remove the otherwise mandatory offset tags from the types (unless there would be a reason for a certain "order" which we would describe via the offset).

This idea needs further review.

~ Joachim

[Jegeva]

@jkrech
either works for me, i have the generator tool here : https://github.com/Jegeva/ads2svd.
Currently generates for all the cores in ADS for the relevant architectures.

@flit @ilg-ul : your opinion ?

[ilg-ul]

I'm not familiar with ADS, but if you can generate compliant SVDs, and Arm agrees to the content, it should be ok.

[flit]

I'm happy with <registers> under <cpu>.

Some attributes we need to track for each register (aside from the obvious ones):

• Type: general, system, status (e.g. xPSR—should these be separate or system?)
• Group: integer, float, Debug, Virtualization, etc
  • Maybe need hierarchy here, which would be most useful for A-profile.
• Whether it's required or optional, and which extension includes it.
  • This would be useful for core peripherals too.
• Execution states (for S vs NS and future AArch32 vs AArch64)
• Banked?

We can take some direction from the v8-A XML releases in how they specify a means to read and write system registers, e.g. the instructions and required parameters.

Here's an example from the v8-A XML release for the TTBCR register:

<access_mechanisms>
      <access_mechanism accessor="MRC TTBCR">
        <encoding>
          <access_instruction>MRC{&lt;c&gt;}{&lt;q&gt;} &lt;coproc&gt;, {#}&lt;opc1&gt;, &lt;Rt&gt;, &lt;CRn&gt;, &lt;CRm&gt;{, {#}&lt;opc2&gt;}</access_instruction>
            <enc n="coproc" v="0b1111"/>
            <enc n="opc1" v="0b000"/>
            <enc n="CRn" v="0b0010"/>
            <enc n="CRm" v="0b0000"/>
            <enc n="opc2" v="0b010"/>
        </encoding>
          <access_permission>
            <ps name="MRC" sections="1" secttype="access_permission">
              <pstext>
if PSTATE.EL == EL0 then
    UNDEFINED;
elsif PSTATE.EL == EL1 then
    // ...
elsif PSTATE.EL == EL3 then
    if SCR.NS == '0' then
        return TTBCR_S;
    else
        return TTBCR_NS;
              </pstext>
            </ps>
          </access_permission>
      </access_mechanism>
      <access_mechanism accessor="MCR TTBCR">
        <encoding>
          <access_instruction>MCR{&lt;c&gt;}{&lt;q&gt;} &lt;coproc&gt;, {#}&lt;opc1&gt;, &lt;Rt&gt;, &lt;CRn&gt;, &lt;CRm&gt;{, {#}&lt;opc2&gt;}</access_instruction>
            <enc n="coproc" v="0b1111"/>
            <enc n="opc1" v="0b000"/>
            <enc n="CRn" v="0b0010"/>
            <enc n="CRm" v="0b0000"/>
            <enc n="opc2" v="0b010"/>
        </encoding>
          <access_permission>
            <!-- ... pseudocode ... -->
          </access_permission>
      </access_mechanism>
</access_mechanisms>

We almost certainly don't need the complete formal semantics by including the pseudocode. If you needed that then just reference the full XML release (though it's not available for M- or R-profile).

[Jegeva]

gentlemen, do we have a decision on this on arm side ?

[tarek-bochkati]

@jkrech at ST level we are evaluating this proposal by @Jegeva
as it seems promising as solution :)
Just I have one remark to share regarding the Core and FP registers. knowing that there are not memory mapped, I'm wondering if we need to add them to the Core SVD ?

[Jegeva]

@tarek-bochkati

hey tarek,

It depends on your usecase, i have a few examples :

-for debug tools knowing which core peripherals are present is very useful to access them and build the correct jtag payloads to talk to the core (@flit 's usecase)
-for code/firmware analysis tools, this can allow 'cpu attribution' (ie. if the assembly instruction uses this register that is not in cpu y, the code is not for cpu y) (my usecase)
-for compilers, having a solid source about what if available on which target core allows to write more generic code that can generate the correct asm to use the available peripheral in the best way
-jkrech's comment 'The "core view" is traditionally managed by the debug tools in proprietary ways' is also a good incentive for me ;)
-As a matter of principle, if there is a central place to describe a core, the core registers should be described

[flit]

@tarek-bochkati Good to see ST are interested in this!

Adding core register definitions seems like a good idea, including CP15 registers for A/R-profile. However, I'd rather have an early inclusion of only the core peripherals in CMSIS instead of waiting for the core register schema to be defined.

Another question is what SVD files should be made available. Certainly for each core, but we probably need core variants with different extensions (FPU, DSP, TZ). And what about architecture-level files?

It would be nice to have an additional SVD attribute that would allow declaring which extension is required in order to for a definition to be valid (peripheral, register, or even bitfield or enum). @JonatanAntoni Could this be added to the SVD schema?

[Jegeva]

Dears, this is mainly a bump since i am a little big avaiting input from you to know how to make this progress :)

@JonatanAntoni, what is you opinion on @flit's question ?

[Jegeva]

Gentle people ? any news ? decision ? I know this is not a trivial decision but having some info would be nice...

[smurfix]

Gentle people ? any news ? decision ? I know this is not a trivial decision but having some info would be nice...

Same here.

FYI, my usecase is writing an interface generator for a language that probably won't ever be supported by CMSIS (Forth …). I'd really like to offer a complete solution to my users.

[flit]

Specifically for the M-profile cores, I've been applying some fixes to @Jegeva's ads2svd scripts. The results are on the develop branch of my fork: https://github.com/flit/ads2svd/tree/develop. I'm testing with pyocd. There are still some issues, particularly with duplicate register definitions due to S/NS aliasing.

My intention is to finish this up, maybe over my December holiday, and create a PR with proposed SVD files for the M-profile cores. (At least in the first PR) Core register definitions will not be included, nor will A-profile since the system registers require CP15. (And frankly I don't need either, so I can reduce the scope to make completion more feasible.)

[korjaa]

Having the core registers available as SVDs would be nice. Anyone aware of any updates towards this?

[tannewt]

I've switched the ads2svd repo to python only here: https://github.com/tannewt/ads2svd Here is a zip of the output.

[Jegeva]

Not really, I guess this is not a priority for arm tbh. Don’t really get why this can’t be solved in 2 years with most of the stuff available and minimal cleanup on their side to integrate Le mar. 16 août 2022 à 23:19, Je GeVa ***@***.***> a écrit :

…

Can you create a merge request to my repo, getting rid of the Saxon dep is nice actually Le mar. 16 août 2022 à 23:05, Scott Shawcroft ***@***.***> a écrit : > I've switched the ads2svd repo to python only here: > https://github.com/tannewt/ads2svd Here is a zip of the output > <https://github.com/ARM-software/CMSIS_5/files/9354689/out.zip>. > > — > Reply to this email directly, view it on GitHub > <#844 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABFV6WIOSSY6YM3EXJKTVOTVZP7DHANCNFSM4K7T4HQQ> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[Jegeva]

Can you create a merge request to my repo, getting rid of the Saxon dep is nice actually Le mar. 16 août 2022 à 23:05, Scott Shawcroft ***@***.***> a écrit :

…

I've switched the ads2svd repo to python only here: https://github.com/tannewt/ads2svd Here is a zip of the output <https://github.com/ARM-software/CMSIS_5/files/9354689/out.zip>. — Reply to this email directly, view it on GitHub <#844 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFV6WIOSSY6YM3EXJKTVOTVZP7DHANCNFSM4K7T4HQQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[auxym]

Nice work @Jegeva and others involved. I see that the XML files are not licensed for redistribution by ARM. Presumably this would also complicate the redistribution of SVD files generated from them? Is there any chance ARM would consider releasing the ADS xml files under Apache 2.0 or a similar license?