UE4 Development

[tidus9000]

Hey I'm working on Integrating PopH264 into Unreal Engine 4, (I've forked my work on https://github.com/tidus9000/PopH264) and thought it might be good to share what I'm doing so far to make sure I'm on the right track along with confirming some things.

I've taken the released libraries on all platforms and have added them as a third party library to a new blank Unreal project that at the moment I have saved on the PopH264 repo in it's own folder similar to the folder structure of the Unity example.

In terms of the project layout. I'm thinking the best way to approach class design is to have a c++ Actor component that essentially acts like poph264.cs in the unity project i.e. making the relevant calls to PopH264. As a starting point, I'm thinking I'll have a c++ Actor that has the poph264 actor component and decodes only raw h264 data from a local file. Again, similar to how the Unity example does it.

One question I had when looking at the Unity examples, the RawH264 decoder doesn't seem to actually show an image though the debug log does show that it is decoding the data, Is that how it's supposed to work in that example?

[SoylentGraham]

Yeah, that example just feeds in data, and gets frames out, but doesnt do anything with them
https://github.com/NewChromantics/PopH264/blob/master/Unity/PopH264/Assets/RawH264Decoder.cs#L27

if it prints out

Decoded frame xx
then it's working

PopH264.cs in unity is the bridge/interface to the plugin, rather than itself an "actor". All the examples in the Unity/ project are essentially the actors.
In unreal, it might be better if the equivilent PopH264Actor.cpp (Or the more correct UE naming scheme :) was an individual actor that just calls PopH264_CreateInstance() itself. It doesn't really need a seperate bridge like unity does as it's all CAPI calls. (Can just include PopH264.h - although you may want to do dynamic library loading... I'm not sure what unreal's support for that sort of thing is like)

Given the way unreal plugins work, it might be easiest to start in a seperate project and make the c++actor there and get it going in a project, (using the built DLL's and PopH264.h) and then get it back into this repository as a proper set of files that can be integrated into UE as a plugin.

Typically THEN, it would probably be setup with a root directory like PopH264.UnrealEngine with build files inside there (which is what a user would want to include in their project)

[SoylentGraham]

Basically I'm hoping for unreal, you can basically just use the .h and C calls, and then the unreal-specific actor/blueprint/texture stuff doesn't need to be in the PopH264 DLL.

If there was a way to include the actor classes etc in the Poph264 DLL, without being dependent on all the UE core libs, that would be great though. Last time I made plugins in unreal, I couldn't really just drop-in a DLL to get new blueprints, but would be great if we could!

[tidus9000]

Hey so an update on where I'm at:
What I've created so far is a blank Unreal Project with a few c++ classes. First is a class that loads in the PopH264 dll and essentially acts as the 'bridge' between Unreal and PopH264, this is the class that gets used when any of the functions of PopH264 are used.

Then there is the class that utilizes this, which essentially does the same functions at the Unity PopH264.cs called UE4PopH264. This is an Actor Component c++ class with the idea that this can be attached to any actor in an Unreal scene like a Unity script I've done by best to replicate all the functions used in the c# version.

Lastly is a small C++ actor class that uses the UE4PopH624 actor component to decode raw H264 that is is pointed at via an http Request.

Currently, this seems to work fine(?) but only for the initial frame that this is loaded. Once I try to proceed to the next in game frame by stepping through code, Unreal throws up an access violation:

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0xffffffffffffffff

UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_OutputLog
UE4Editor_OutputLog
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_Slate
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_Slate
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_Slate
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_Slate
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_Slate
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_SlateCore
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor_Slate
UE4Editor
UE4Editor
UE4Editor
UE4Editor
UE4Editor
kernel32
ntdll

It's also a bit slow at the moment as this is all running on the main game thread at the moment.

[tidus9000]

So a lot of stepping through has lead me to believe that it might possibly be my calling of the PeekFrame function in the PopH264 dll, though it seems the same happens when I call EnumDecoders and GetMeta as well. The function itself seems to run fine and I'm able to deserialize the json blob that comes out of it but it looks like when Unreal finishes the update cycle for that frame, it decides to throw up that exception. The commonalities I see with these 3 functions is that they're all void functions and they make use of arrays of values. I'll give an overview of how I load in the dll and functions and maybe you can see if I'm going wrong somewhere?

I Used this guide as a template for how to link a dll in Unreal: https://unreal.gg-labs.com/wiki-archives/devops/linking-dlls

All the code I've written here is part of the same ImportPopH264dll class
First I import the dll like so:

void* v_dllHandle
bool ImportPopH264dll::importDLL()
{
 FString filePath = "<DLL Directory>";

 if (FPaths::FileExists(filePath))
 {
	 v_dllHandle = FPlatformProcess::GetDllHandle(*filePath);
	 if (v_dllHandle != NULL)
	 {
		 return true;
	 }
 }
 return false;
}

Then I import the function:

bool ImportPopH264dll::Import_PopH264_PeekFrame()
{
    if (v_dllHandle != NULL)
    {
    	    m_PopH264_PeekFrameFromDLL = NULL;
	    FString procName = "PopH264_PeekFrame";
	    m_PopH264_PeekFrameFromDLL = (_PopH264_PeekFrame)FPlatformProcess::GetDllExport(v_dllHandle, *procName);
	    if (m_PopH264_PeekFrameFromDLL != NULL)
	    {
		    return true;
	    }
    }

    return false;
}

Then I actually call the function like so:

void ImportPopH264dll::PopH264_PeekFrame(int32_t Instance, char* JsonBuffer, int32_t JsonBufferSize)
{
    if (m_PopH264_PeekFrameFromDLL != NULL)
    {
	    m_PopH264_PeekFrameFromDLL(Instance, JsonBuffer, JsonBufferSize);
	    return;
    }
    else
    {
	    UE_LOG(LogTemp, Error, TEXT("Function PopH264_PeekFrame being called when not imported"));
	    return;
    }
}

This is what I'm actually calling when I'm trying to access PopH264 functions, I'm guessing that the issue is how I'm calling the function from the dll? there weren't any void functions or functions that take arrays or pointers in the guide I linked above so this was a bit of guesswork!

Apologies for the wall of text!

[SoylentGraham]

That looks okay, but missing the crucial information... where does in the code does it throw the exception?
The exception is that something is reading 0xffffffffffffffff, but this could be a misnomer. (this can sometimes be how it causes an exception, when throwing an exception)
But it could also be assigning -1 returned from GetDllExport() (not nullptr) to the function pointer.

When you debug, is m_PopH264_PeekFrameFromDLL a valid number (ie. not ffff and not zero)

You said before you had the first frame before? was that not quite right afterall?

You might find also that to avoid other possible problems (like bad pointers being passed into functions), is to link to PopH264_GetVersion() first and get an int out of that :)

The easier approach with all this, might be to include PopH264.h, then it will fail to link because of missing symbols.
You can link to PopH264.lib (in the windows release dir) which will then try and load PopH264.dll when it starts, or lazy-load the DLL (this is a setting in the visual studio project normally, I'll have to check if UE has a setup for this), so it doesn't resolve the symbols at link time, but will resolve when you manually load the DLL with GetDllHandle()

Then you don't have to resolve symbols yourself at all.

Maybe not crucial for now, but you could make the C-function assignment a bit more modern by converting them to std::function's, like here; https://github.com/SoylentGraham/SoyLib/blob/master/src/SoyRuntimeLibrary.h#L50

[SoylentGraham]

When it crashes, the debugger should have a stack trace (kinda like you originally posted, but that was missing all the important info), and point right at where the exception is being thrown

[tidus9000]

The exception itself seems to get thrown deep in Unreal dlls that I'm not familliar with. Don't know if it helps but I took a shot of the call stack

The debugger shows m_PopH264_PeekFrameFromDLL as what I assume is a valid memory address, certainly not ffff or zero (0x00007ff81daa7510 as an example)

Yeah, I assumed that I was getting the first frame as I was able to step through and PopH264_PeekFrame went through without throwing an exception. Looks like that isn't the case as I think the exception is thrown at the end of the update cycle of the game frame that gets the first video frame.

I did manage to get a valid output from PopH2564_GetVersion() without any issue, I should have mentioned that a lot of the other functions seem to work fine. I haven't tried every single one out but before calling PopH264_PeekFrame() I manage to create a decoder with PopH264_CreateDecoder without any issue.

I'll have a try at linking poph264.lib

[SoylentGraham]

If some calls are succeeding, then leave the DLL loading as you have it, rather than change stuff :)

I think the exception is thrown at the end of the update cycle of the game frame that gets the first video frame.

I'm not really seeing any evidence of that, but then, still can't really see where in the code its crashing.
Your call stack has no symbols, you should be able to rectify this;

• Either a debug build of the engine
• Maybe UE has a PDB somewhere to go with UE4Editor-core.dll UE4Editor-Renderer.dll

You say you're rendering the first frame... can you show me how you do that? Have you tried NOT rendering the first frame? you can just grab frame meta (peek) and frame buffers (pop frame), and throw them away. Maybe the renderer is the problem, and not fetching frames?
Where are you getting the memory to put into Peek() Pop() etc?

Basically, what stuff do you remove until it stops crashing?

But really, we still need to know where it crashes, all I can tell at the moment is that it crashes in unreal code. (But I don't know what the crash is, other than reading memory at 0xffffffff)
Nothing in the output with a description of the exception?

[SoylentGraham]

As for the speed, the plugin is synchronous, but in reality (or at least, most platforms) once the data is copied in, it just returns.
Same with reading the next frame out, it's synchronous (although doesn't block if no frame is ready), so when it comes to speeding up, having a thread with data queued up to push will be a speed up (the Unity version does this), and you can have a read-thread, which can block until a new-frame-callback is called.
But deal with that later :)