Build STROOP on Linux platforms (with Mono and Docker) #72
Comments
|
From what I’ve heard, STROOP builds fine on wine. I don’t think Pannen has any plans to make it build natively. |
|
@onlymx13 Thanks for your answer. It will run probably fine on Wine, but this was just a question I had looking at the project. Using Mono, a developer using a GNU/Linux distribution would have directly all the tools to develop/debug the application in an IDE (example : VS Code with Mono debug plugin) without using Wine every time to build the software and not be able to debug effectively. In fact, this is also more a question of curiosity, to understand what is the effort to refactor STROOP to not use the WPF APIs (I mean, if there is for example only ten files to change, then it won't probably be too difficult to migrate - I can help on this 😃 - but I may be wrong). Using Mono also opens the door for using technologies like Docker to create reproducible builds and images. And it may attract also GNU Linux/MacOS developers for contributing. |
|
I made further researches, and it looks like hopefully Mono supports WinForms. The 4 missing assemblies used by WPF and missing are the following, as the last warning message suggests in my first message :
I figured out that each one is bound to a particular namespace :
The 2 last namespaces are not used in the entire STROOP project. Imports of Concerned C# files are :
I believe (although not sure) that without these imports, STROOP could be build natively on Linux with Mono. I'll try to see what I can do. |
norbjd
changed the title
|
I removed But I got new build errors now (more) : There are 2 distinct error messages :
So, my last message was wrong, my mistake (not a C# expert) : DLL hides many namespaces. PresentationFramework.dll hides other namespaces, for example There are probably other namespaces hidden behind these DLLs, but having no Windows system, I can't find what's behind all these 4 missing DLLs (maybe a kind Windows user could "decompile" them to help me find all namespaces behind 🙏). Unfortunately, There are many dependencies to these files (except |
|
I was able to make some progress to make STROOP Mono-compatible (thus, natively GNU/Linux compatible). As for the There was also an error with At this step, after fixing the errors, I was finally able to build STROOP with Mono in a Docker container by running : I then tried to run STROOP : Basically here, STROOP is running in a container but the GUI is run on the host by sharing the X11 socket ( But there was an issue with some resources. Indeed, the separator used for directories in --- a/STROOP/Utilities/XmlConfigParser.cs
+++ b/STROOP/Utilities/XmlConfigParser.cs
@@ -255,16 +255,16 @@ namespace STROOP.Utilities
switch(subElement.Name.ToString())
{
case "ImageDirectory":
- imageDir = subElement.Value;
+ imageDir = subElement.Value.Replace('\\', Path.DirectorySeparatorChar);
break;
case "DefaultImage":
defaultImagePath = subElement.Value;
break;
case "MapImageDirectory":
- mapImageDir = subElement.Value;
+ mapImageDir = subElement.Value.Replace('\\', Path.DirectorySeparatorChar);
break;
case "OverlayImageDirectory":
- overlayImageDir = subElement.Value;
+ overlayImageDir = subElement.Value.Replace('\\', Path.DirectorySeparatorChar);
break;
case "EmptyImage":
emptyImagePath = subElement.Value;
@@ -697,15 +697,15 @@ namespace STROOP.Utilities
{
case "ClassicInputImageDirectory":
guiList.Add(CreateInputImageAssoc(
- path, subElement.Value, InputDisplayTypeEnum.Classic));
+ path, subElement.Value.Replace('\\', Path.DirectorySeparatorChar), InputDisplayTypeEnum.Classic));
break;
case "SleekInputImageDirectory":
guiList.Add(CreateInputImageAssoc(
- path, subElement.Value, InputDisplayTypeEnum.Sleek));
+ path, subElement.Value.Replace('\\', Path.DirectorySeparatorChar), InputDisplayTypeEnum.Sleek));
break;
case "VerticalInputImageDirectory":
guiList.Add(CreateInputImageAssoc(
- path, subElement.Value, InputDisplayTypeEnum.Vertical));
+ path, subElement.Value.Replace('\\', Path.DirectorySeparatorChar), InputDisplayTypeEnum.Vertical));
break;
}
}
@@ -934,7 +934,7 @@ namespace STROOP.Utilities
switch (subElement.Name.ToString())
{
case "FileImageDirectory":
- fileImageDir = subElement.Value;
+ fileImageDir = subElement.Value.Replace('\\', Path.DirectorySeparatorChar);
break;
}
}
@@ -1177,7 +1177,7 @@ namespace STROOP.Utilities
assoc.MapImageFolderPath = subElement.Value;
break;
case "BackgroundImageDirectory":
- assoc.BackgroundImageFolderPath = subElement.Value;
+ assoc.BackgroundImageFolderPath = subElement.Value.Replace('\\', Path.DirectorySeparatorChar);
break;
case "DefaultImage":
var defaultMap = new MapLayout() { ImagePath = subElement.Value };
@@ -1290,7 +1290,7 @@ namespace STROOP.Utilities
switch (subElement.Name.ToString())
{
case "ScriptDirectory":
- scriptDir = subElement.Value;
+ scriptDir = subElement.Value.Replace('\\', Path.DirectorySeparatorChar);
break;
case "FreeMemoryArea":
parser.FreeMemoryArea = ParsingUtilities.ParseHex(subElement.Value);
@@ -1336,7 +1336,7 @@ namespace STROOP.Utilities
switch (subElement.Name.ToString())
{
case "HackDirectory":
- hackDir = subElement.Value;
+ hackDir = subElement.Value.Replace('\\', Path.DirectorySeparatorChar);
break;
}
}After this fix, I faced another issue here, the loading screen was launched correctly (until the end) :
But then an exception was thrown. After debugging ( There is a similar issue an other Github project, fixed by this commit. Apparently, Mono does not like uninitialized string. I don't know more about this. So I just replaced (stupidly) in --- a/STROOP/Controls/ObjectSlot.cs
+++ b/STROOP/Controls/ObjectSlot.cs
@@ -36,7 +36,7 @@ namespace STROOP
Image _bufferedObjectImage;
Point _textLocation = new Point();
Point _objectImageLocation = new Point();
- string _text;
+ string _text = "";
int _fontHeight;
#endregionAnd this hopefully solved the error. After the loading screen, I was able to see the connection screen :
By bypassing this step, I was able to have STROOP running under GNU/Linux (in a Docker container) with Mono :
You can compare all the modifications on my fork : Development...norbjd:6b3a768b4c0c9b0315074281ec11ec3a7e56d3b5. |
|
Just a little update, I've made some modifications : Development...norbjd:21c38cf798e402bafac085f32f8268bd17e54f53. STROOP builds fine with Mono. As of now, I cannot connect to an emulator since there is no other process in the container, only STROOP. I'm working on this. When bypassing the connect step, the UI works fine but no values are shown in the different tabs. I think it's the normal behavior because STROOP is not connected. But, sometimes, STROOP also hangs when clicking on tab or buttons with no particular errors. Loading save statesI've tried to install
Also, you can see that objects are not shown in the part below. @scob @onlymx13 : does loading save states is actually working on Windows with the latest version (Development branch)? Connection to
|
|
I don’t know too much about mupen64plus or STROOP. However, what I will ask is whether the mupen64plus .st2 files are the same format as regular mupen64 .st files. If not, they won’t work with STROOP. I think you can get the correct RAM start value using Cheat Engine somehow. Try to find the value of Mario’s coin count or something, and you could convert that into a RAM start. I’m not sure if the issues you’re experiencing with Bypass are normal or not. What kind of buttons and tabs were causing errors? I ask since for Bypass, really M64 is the only useful tab, unless you just want the address of a variable. Not showing variables is normal. I’m pretty sure regular mupen64-rr can be built for Linux; have you looked into that? |
|
@onlymx13 Thanks for your answer. When using Bypass mode, or with
Indeed, it can be a problem with Some errors occurs sometimes when resizing the window, or clicking too fast on some different tabs. I guess building @onlymx13 Before that, could you please share with me a |
|
The M64 tab is supposed to be like that until you press “Open” to open a mupen64-rr TAS file, or .m64 file. Last commit in 2013? Sadly, I’m gonna say that’s the wrong version because that’s too new. I guess it depends on which version of mupen you want. Are you in the SM64 TASing and ABC Discord server? I would advise joining that. Link is on the sidebar of https://ukikipedia.net. I don’t have an st file right now, but you could use Discord’s search to find one in that server. Some of your questions would be better put there. I’m MMMMMMMMMMMMM#2694 and scob is pannenkoek2012. |
|
@onlymx13 Thanks for the links. I joined quickly the SM64 TASing and ABC Discord (I'll create an account later 😄) just to get a valid
So Objects are loaded in the below part, but I don't see values for The Memory tab seems to work correctly, when clicking on an object :
I'll join the Discord as soon as possible and try to get the conversation continued here 😄. Thanks again for your help 👍, that was very appreciated. |
|
Also, I found last version of |
|
No no no, you misunderstand. 2013 is too new. You want one from 2005 or 2006. The 2018 one might still work though. I'm not exactly sure. |
|
Ok, my bad, I misread that. In fact, the code was hosted here before http://code.google.com/p/mupen64-rr, and initial commit is from 2009 (not 2005 or 2006 though 🤔). The migration to Github was done in 2013, hence this date on Github. I'll try to build an old version and see if it works, but I'm very sceptical because dependencies packages have been updated since. |
|
I've tried to build the version hosted on SM64-TAS-ABC Github for Linux (from the initial commit https://github.com/SM64-TAS-ABC/mupen64-rr/tree/d6cffe315be7b4d48abac2285b151cb62df12452), but without success. I've also found the source from v0.5 (2005/08/27) and have not been able to compile it fully. Build may have succeed someday on someone's own machine, but I think there are some missing things (I've tried to tweak the build too). I've finally found the mupen64 v0.5 Linux binary from this site : http://mupen64.emulation64.com/down.htm and was able to run it correctly along with STROOP (after installing some 32 bits dependencies). Unfortunately, the binary I'm not sure this version of |
|
As far as the processes go. The way it works is that STROOP assumes there is a process (executable) section of PC (Windows/Linux) memory that holds the N64 RAM memory. It can either be stored in little or big endian form. So in order to get "RAW N64 memory", the process name, the offset where the N64 memory starts in the process, and the endianess type needs to be known. The offset is assumed changed whenever the emulator is re-compiled. This is is because the compiler can place the n64 memory location to wherever it wants. So because you are recompiling, or using the linux version of mupen, the offset is going to be different regardless. You'll need to find the offset which can be calculated by finding a known address, like Mario's health, y coordinate, etc. using cheat engine. Then subtracting the N64 address of that value from the process address you just found. |
|
@danebou Thanks for your answer. I've made an issue (#73) related to add mupen64plus emulator, which is easier to install on GNU/Linux platforms, and also more recent than the mupen64 v0.5 I talked in my last comment (2005). I've also made a PR to support it (#74), feel free to check it. As I said in the PR, I'm not sure it works on Windows because I don't have a Windows machine and as of now, connecting to a running process on GNU/Linux requires to rewrite some parts of the code as we have discussed on Discord. I have made a branch on my fork : https://github.com/norbjd/STROOP/tree/linux-build to rewrite functions requiring Windows only DLL to native functions ( |
|
Just to keep this issue up-to-date, values are now shown in STROOP on GNU/Linux :
The hidden values was caused by the |
Next step : rewrite extern functions relying on some DLLsThe next step is now to rewrite all methods defined as |
|
I believe that on In addition, Mupen64Plus uses a heap allocation for its memory, so that's somewhat annoying. |
|
Hi! I've been working on BizHawk's Linux port for a few years now and I'd like to offer my help. I'm in the Discord (same username) if you'd prefer to discuss there. Some things that were helpful for BizHawk:
Some specific concerns from this thread:
|
|
I know its been a while. As far as this goes, if you still want to do the development work, then we can add linux support. For that just make many gradual PRs and we can go back and forth. |
|
Is that directed at me or OP? I can rebase my PR if you like. |
|
Either lol. That's sounds good. I guess what I'm saying is I'll actually monitor PRs that improve STROOP, but and I don't want a massive PR to review (not your PR, just a giant linux one). |
|
Hello 👋 I've completely given up this to be honest, sorry 😅 I only remembered it was too difficult (at least for me) to port to Linux because there were some references to Windows specific stuff, e.g. kernel32.dll. I even asked on StackOverflow to get some help:
But it seemed impossible at the time, and I really lacked C# and Windows expertise 😕 Nevertheless, the last version I've made (dev...norbjd:STROOP:linux-build) was correctly loading If @YoshiRulz wants to work on this, I'll closely follow his work, but I won't have neither the time nor the expertise required to help him. But glad to see this issue revived! |
|
You're good. Hey I'm responding to this after four years lol. Compatibility would be nice, but yeah sounded like to much of a pain to get working. |
|
I'll close this out unless Yoshi says otherwise |
|
I'll do what I can. I rebased my PR already. edit: merged 2024-04-30 |
|
I drafted a similar migration for the test project, but the main project doesn't compile on Linux so I can't test it. Not that there'd be much point to it since there are only 4 test methods. I'll work on case-sensitivity next, |
|
I haven't seen any issues so far. |
|
Figured I should give an update on this: |
|
Nice, sounds good. Yeah modernizing is an understatement. I know pannen might be doing work here and there, but my current plan for stroop is just to let it coast (aside from community updates). Maybe AI will eventually make pull requests lol |
Hello ✋,
I'd like to help on this project but I am not working on a Windows platform. I'm not an expert in C# too. I'm just a regular developer impressed by the work made on this project to find new strategies for the different challenges (ABC and others).
So, I'm trying to build the application under Linux environment using Mono, an open-source implementation of Microsoft's .NET Framework. I'm using a Docker image to create reproducible builds, using an official base Mono image based on Debian 9 (Stretch).
You can see the different updates I've made here on my fork of STROOP and the branch (
linux-build) I've created : Development...norbjd:6f219e3c6efa509a045504f4163db9c8c43ceb70.To summarize quickly, here are the steps I have made :
STROOP/STROOP.csprojtoSTROOP/STROOP_linux.csprojto avoid side effects with current builds. I'll use and modify thisSTROOP/STROOP_linux.csprojfor the Linux build.STROOP/TTCfolder (apparently, Windows does not matter about case but Linux does), precisely replacingTtctoTTCPreBuildEventandPostBuildEventto match Linux commands (cpinstead ofxcopyandrminstead ofdel). Not sure that thePostBuildEventmakes sense, but I did it just for the sake of consistencyThe difference between
STROOP/STROOP.csprojandSTROOP/STROOP_linux.csprojcan be seen usingdifftool :When building, I've got some errors :
I have fixed some of them by removing some unused (maybe?)
usingimports, more precisely inSTROOP/Controls/DecompilerView.xaml.cs(see this commit : norbjd@c217ea5). This may be wrong (it may add some side effects?), but it removed some errors :I have also noticed that some warnings appeared before the errors (which may explain the errors) :
The build could not locate the assemblies
PresentationCore,PresentationFramework,UIAutomationProviderandWindowsFormsIntegration. Indeed, WPF (Windows Presentation Foundation) APIs are not implemented byMono. That explains whyPresentationCoreis not available for example.By not using WPF APIs, it will be possible to build STROOP for Linux (and probably MacOS). It may (or not) attract some developers like me to contribute to this project. The problem is : I don't know how many efforts one have to make to not use WPF APIs. @scob will probably be able to answer this question. Do you think it's worth it? Maybe I'm chasing ghosts here.
If you have any questions, feel free to ask.
Thank you very much and great job on this tool once again. 👍
The text was updated successfully, but these errors were encountered: