Converting a Buckeburg to ODF

[oleg68]

Hello!

I'm trying to create an odf for the Buckeburg sample set from Sonus Paradisi.

When I run ODFEdit for windows, it seems to work, but when I run ODFEdit fot linux, it sets all samples to None. So I can not open it in GrandOrgue.

[Rank026]
Name=Front (Direct): HW Viola da gamba 8
NumberOfLogicalPipes=58
FirstMidiNoteNumber=36
Percussive=N
AcceptsRetuning=N
WindchestGroup=005
Pipe001Gain=-1.5
Pipe001HarmonicNumber=8
Pipe001LoadRelease=N
Pipe001=None
Pipe001ReleaseCount=3
Pipe001Release001=None
Pipe001Release001MaxKeyPressTime=442
Pipe001Release002=None
Pipe001Release002MaxKeyPressTime=-1
Pipe001Release003=None
Pipe001Release003MaxKeyPressTime=212
Pipe002Gain=-2.7
Pipe002HarmonicNumber=8
Pipe002LoadRelease=N
Pipe002=None
Pipe002ReleaseCount=3
Pipe002Release001=None
Pipe002Release001MaxKeyPressTime=220
Pipe002Release002=None
Pipe002Release002MaxKeyPressTime=449
Pipe002Release003=None
Pipe002Release003MaxKeyPressTime=-1
Pipe003Gain=-0.3
Pipe003HarmonicNumber=8
Pipe003LoadRelease=N
Pipe003=None

The log window shows:

Hauptwerk ODF loaded "/mnt/fast/common/organ/sample_sets/Buckeburg/OrganDefinitions/Buckeburg, Janke Organ, Surround Demo.Organ_Hauptwerk_xml"
1093399 attributes among 137594 objects among 44 object types
WARNING : The package ID 2306 named "Buckeburg, pack. No. 2306 (vol. 1)" provided by "Sonus Paradisi/Hauptwerk.cz"
WARNING : is not present in the folder \mnt\fast\common\organ\sample_sets\Buckeburg\OrganInstallationPackages\002306
WARNING : Some graphical or sound elements of this organ may be not rendered in GrandOrgue
GrandOrgue ODF built and saved in "/mnt/fast/common/organ/sample_sets/Buckeburg/OrganDefinitions/Buckeburg, Janke Organ, Surround Demo.organ"
It can be loaded in GrandOrgue

GrandOrgue ODF loaded "/mnt/fast/common/organ/sample_sets/Buckeburg/OrganDefinitions/Buckeburg, Janke Organ, Surround Demo.organ"
121848 attributes among 507 objects, file encoding utf_8_sig
New panel format

[oleg68]

When I create the odf file on windows and then I'm trying to open it with GrandOrgue for Linux, I receive lots of errors:

'..\OrganInstallationPackages\002306\pipes\Buckeburg_4noc_UW_tr\UW_Regal8\05-reafar_UW_Regal8_tr\093-A.wav' does not exists
Вт 27 дек 2022 21:21:45: 21:21:45: Error: File '..\OrganInstallationPackages\002306\pipes\Buckeburg_4noc_UW_tr\UW_Regal8\05-reafar_UW_Regal8_tr\rel99999\093-A.wav' does not exists
Вт 27 дек 2022 21:21:45: 21:21:45: Error: File '..\OrganInstallationPackages\002306\pipes\Buckeburg_4noc_UW_tr\UW_Regal8\05-reafar_UW_Regal8_tr\rel00220\093-A.wav' does not exists
Вт 27 дек 2022 21:21:45: 21:21:45: Error: File '..\OrganInstallationPackages\002306\pedal\p1_on.bmp' does not exists; FileName=..\OrganInstallationPackages\002306\pedal\p1_on.bmp
Вт 27 дек 2022 21:21:45: 21:21:45: Error: Failed to open the graphic '..\OrganInstallationPackages\002306\pedal\p1_on.bmp'


The reason is the wrong case of the directory names: `Buckeburg_4noc_UW_tr` instead of `Buckeburg_4noc_UW_TR`.

It is not good to rename files and directories in the Hauptwerk sample sets. It would be better ODFEdit to read (recursively) the directory listing of the sample set(s), to match all files mentioned in the .xml and to put filenames with correct case in the .organ file.

[oleg68]

Its a big problem on Linux, as the HW contains invalid case file names. The problem is not necessarily the / or \ as Linux (and Windows) should accept either. The solution I use is to treewalk the filesystem to extract an indexed array of lowercase filename vs actual filename, then lookup lowercase of the HW file.

Traversing the whole filesystem may take very long time. I'd suggest to list files only in the directories that are referenced by the .xml. In case of Buckeburg it is ..\OrganInstallationPackages\002306 (I'd suppose that OrganInstallationPackages is already in the correct case).

[larspalo]

I did look into that sample set and immediately noticed that there are some inconsistencies in cases of the file suffixes among other things - mostly in the image files. This is of course a direct issue coming from the case insensitivity of the Windows OS that in turn leads to sloppiness from its human users... The path separator is not really an issue here but as Oleg correctly points out the issue is caused by differences in case in the path names.

Getting the correct file names with paths from the file system is not the biggest problem with converting HW samplesets, I think, it's (at least for me) decoding what should be used for exactly what purpose among the multitude of files... Interesting project though, I think.

[eturpault]

@eturpault I saw several uses of // or \ to build path names.

You should use os.path.join() to do so. The code will be automatically adapted following the operating system it will be executed on.

You are right @rousseldenis, my first version of OdfEdit was not managing correctly the folders separators vs the OS, I fixed it for the next release, I used os.sep instruction, but indeed os.path.join() is a good method as well, I will think to it, thanks for the suggestion.
In addition for the next release I have implemented what Oleg suggested, that is to put in the GO ODF the files path/name/extension case as it is defined in the package and not as it is defined in the HW ODF, by using Python Path(xxx).resolve() method of pathlib, thus we avoid the warnings ""Filename '%s' not compatible with case sensitive systems" in GO. It works well in Windows, but not in Linux..., and doesn't increase too much the processing time. Does some one know how to get it to work in Linux ?

[eturpault]

I think that if OdfEdit is executed in Linux, it will have to make a quite complex files look-up in order to resolve characters case difference between the Hauptwerk ODF and the real files path/name of the sample set. I will work on it. I found some examples of such processing in stackoverflow.com. At the end the GO ODF generation may take time in Linux and with the sample set which are not mentioning the correct files path/name case in the xml, especially for the ranks which are refering to a lot of files.

[eturpault]

Finally I found a way to retrieve in both Windows and Linux the actual files path/name case as defined in the Hauptwerk sample set
and to put it in the GO ODF, without taking so much additional processing time. It will be present in the next v2.1 release that I hope to deliver in the next days, with one additional feature (noises management).

[oleg68]

This is a working ODF for the Buckeburg sample set.

Buckeburg, Janke Organ, Surround Demo.zip

It was initially made with ODFEdit 2.0 and then the case of directoty and file names were corrected manually.

Some wrong stuff still exists. For example, the windchests of HW and UW are called Pedal that is not correct.

[eturpault]

The wrong windchest names are coming for the original Hautpwerk ODF, OdfEdit simply reuse them. For example in this ODF the samples of the rank "Rear: HW Fagott 16" have their source wind compartment named "Pedal 2.1 R". This source wind compartment has the wind coming from another wind compartment named "SW" ! In the sample sets of Piotr Grabowski which are very well designed I don't remember to have seen such wrong namings.

[larspalo]

@oleg68 Thanks for sharing the ODF. I tried it but sadly ran out of memory! :-) I tried to disable all the rear and direct ranks (to only load the diffuse) but still it was too much for my old computer! I'll try a few other tricks but if they too fail, I'll just have to re-build the ODF to only use smaller parts of the organ until I've upgraded the amount of RAM.

[eturpault]

Here is a new version of the ODF generated with OdfEdit v2.4 beta with the termulants converted from HW. There is one tremulant per manual, and in the "Mixer and Noise" panel it is possible to select between real (wave based, using dedicated ranks) or artificial (synthetized) tremulant. Now only the tracker noise is missing in this converted sample set, but this is a low priority development for me for the moment.
Buckeburg, Janke Organ, Surround Demo.zip

The three synthetized tremulants have the attributes :

StartRate=6
StopRate=6

coming from the HW ODF attributes as suggested by @kerkovits in this comment GrandOrgue/grandorgue#1162 (reply in thread):

StartRatePercent=6
StopRatePercent=6

Is it valid to have the same value in StartRate and StopRate ?

[larspalo]

Is it valid to have the same value in StartRate and StopRate ?

Sure, as long as they are within the allowed range (1 - 100).

[eturpault]

I provide here two new ODF for Buckeburg, generated with OdfEdit v2.4 beta :

• one with the wave based tremulant, RAM used by GO : 49.5 Gb
• one without the wave based tremulant, RAM used by GO : 32.4 Gb

Both have the synthetized tremulant. So for who wants to have less used RAM, the second ODF is a good solution.
I fixed also manually in the ODF errors coming from the HW ODF :

• some wrong windchest names
• some ranks of the pedal which are inside windchest of manuals (so impacted by their synthetized tremulant).

Buckeburg, Janke Organ, Surround Demo - with_without wav trem.zip
The menu of OdfEdit v2.4 will have an option permitting to convert or not from HW ODF the wave based tremulant.

[wsc44]

Could you please provide an updated Bückeburg-ODF (V.11)? Thanks a lot.

[eturpault]

Yes I could, but don't you want to generate it by yourself using OdfEdit and with the conversion options of your choice ?

[wsc44]

I certainly would like to do it myself. But, as mentioned in this thread, the original SP-xml-file contains some errors (wrong windchest names etc.) and I assumed that you have a corrected xml-file, so that it wouldn't take much time to generate the odf-file. But maybe you're right ... I will correct the org-file myself so I needn't ask in the future. [I still wonder why SP hasn't updated that xml-file ... those who have bought the direct samples must have reported the faults.]

[eturpault]

No I don't have a corrected xml file for Buckeburg, my rule is to never modify HW ODF files and make corrections / adjustments / modifications only in the generated GO ODF (which is also simpler to do compared to modifying HW ODF, using OdfEdit which is made for that).
Here is the Buckeburg GO ODF generated by OdfEdit v2.12beta (tremmed samples are not converted, alternate panels layout are converted) in which I have rearranged the ranks in properly named WindchestGroup sections by audio channels / divisions, as I think it should have been done in the original HW ODF.
Buckeburg, Janke Organ, Surround Demo.zip

[wsc44]

O.K., understood - thanks for ODF!