Reverse Engineered The Dreamcast Emulator

I did some reverse engineering on the DEMUL emulator to accomplish something I am not sure anyone else on the internet is doing.I’ll explain how here, but first let me explain my overall goal.

https://i.redd.it/fx9wg0lcojt31.png
Virtual On set

I bought 4 arcade sticks, typically used in flight sims, for about 30$ each on amazon. I attached them to my standing bar that I had, and I put virtual on for Sega Model 2 emulator.The emulator emulates the game including the networking, the only real difference here for me is I am doing all of this on 1 computer. The sega model 2 emulator was no problem, 2 windows open with the game, inputs worked fine.

Now, the goal here was to do the same thing for Virtual on 2 (*googles name) ‘Virtual On Oratorio Tangram’.

The issue here is that DEMUL seems to be the only working option to not only emulate this game but also have it networking as well. DEMUL is coded in a way that says ‘only when a window has focus can we receive input from the users hardware’.

So, to understand the work that needs to be done here we have to understand- Windows API / Messaging / input- What is DEMUL doing to handle input, so that we can try and change it.

So in summary I want to play multiplayer arcade Virtual on 2 on a single PC and in order to do this I need to modify the only emulator out there that can handle this.

Now I will explain in simple terms what I did, so enjoy the ride.

Windows API –https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-postmessageahttps://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput

Windows has 2 main ways that you can send input to a window. You can use the SendInput function, which only works when a window has focus, and pretty much always works, or you could try to post a message to the messaging handler of your application with PostMessage and pray to god that it works. You can google the caveats here, but if you have ever tried to bot in a game with, say, autohotkey and it doesn’t work, this is why.

You can download a tool right now called SPY++ that lets you see system calls of any running process.Every ‘thing’ that runs on your computer is a process that has threads. Threads are tasks being performed for that process

https://i.redd.it/oxsm4vkkojt31.png

In the case of DEMUL when you press a key you can see in the background its firing off a POSTED event (notice how it says posted), and there are two events that occur.WM_KEYDOWN and WM_KEYUP
You can go to town on what those actually mean if you want.https://docs.microsoft.com/en-us/windows/win32/inputdev/wm-keydown

Now if you are familiar with Autohotkey (now AHK) then you might have a script like this
ControlSend,, d, ahk_exe demul.exe

Basically going through the WINAPI POSTMESSAGE that I showed above to send messages to a window.
Additionally you can chose a language of your choice and code it yourself, like I did in C# + powershell *CODE OMITTED*

Using SPY++ We can see that there is no difference between the data sent from AHK, my actual keyboard or the C# code.

Now, when sending input from a ‘location’ the windows OS knows the difference between a hardware call from a USB device such as a keyboard, or a virtual keypress from software. You can get the same messages posted to a thread, but ultimately if the application is coded a certain way, it could just say
‘if the input is not from hardware device A, throw it out the window’

What I theorize is happening with DEMUL emulator is two things:
1.) If its a virtual device, NO THANKS
2.) If the application isn’t the main window in focus, NO THANKS

Windows is decided so that one window is your main window, it might not be the highest Z order window on your screen, but its the one that has focus and where the hardware inputs naturally go.

Now the game of reversing DEMUL to try and eliminate #1 or #2 begins.

First things first when you have DLL code or an EXE it depends on what it was written in. If you code it in C#, C, or C++ its different compilers, thus different machine instruction sets. So you need to reverse it with a tool that can understand those instructions..

In comes PEID.Install PEID and run it.

Post image
PEID in action

It’s easy as pie. Basically point it to something. In my case I’m looking at the PADDEMUL.dll, as i’m assuming pad means game pad, and that the code that handles input is here (turns out I am correct).This shows me that i’m dealing with a Win32 application. Written in C.

Here I will mention, some work was done with the DEMUL GIT code:https://github.com/p1pkin/demul
This is a git repo of, what I think is alpha 0.001 of DEMUL. It is so old I am not sure it can run more than 1 game if that. I haven’t gotten it to work at all but I did use it as a starting point to try and see how the code works. And pray to god its underlying input system remained similar in later revisions as its closed source.

Now that we’ve identified the instruction set, we download a tool to reverse it. Googling ‘reverse C DLL code’ will get you some stuff. I ended up with IDA.

Now, going back to the GIT code of alpha 0.1, long story short, it turns out the code uses Direct Input.https://docs.microsoft.com/en-us/windows/win32/xinput/xinput-and-directinput

And checking out this code here..https://github.com/p1pkin/demul/blob/master/padDemul/device.c

I saw this line of code.

hr = IDirectInputDevice8_SetCooperativeLevel(lpDIKeyboard, gDemulInfo->hGpuWnd, DISCL_NONEXCLUSIVE | DISCL_FOREGROUND);

With some googling..

https://docs.microsoft.com/en-us/previous-versions/windows/desktop/ee417921(v%3Dvs.85)
DISCL_FOREGROUNDThe application requires foreground access. If foreground access is granted, the device is automatically unacquired when the associated window moves to the background

DEMUL coded it as point #2 mentioned above. I see a possible parameter is

DISCL_BACKGROUND

Granted I really have no idea at this point, this is all speculation.I sure do want the application to take input while its a background window, and this is dealing with input. so, lets see.

Now, the next step here is I compiled the source code which took a few days because of the way VS2017 updated a project from prior to VS2010, linking and dependencies were broken.

I compile the code 3 times. I compile the code with no change twice, this way I can eliminate any ‘bytes’ that are just versioning. When you build an EXE or DLL or other, the code will change stuff behind the scenes per build even if you don’t change it. For example, what version the code is. Incrementing each time.

Using our next tool ‘COMPARE IT’ we compare the HEX data of the DLL files.I first compared the two versions with no changes, and marked down the address locations that changed.These will be immediately ignored when I compare the change of the flag
DISCL_FOREGROUND
to
DISCL_BACKGROUND

Now when you change 1 thing in code it could echo throughout many different parts of the hex data.What I saw was that the byte 06 became byte 0A.

Post image
HEX view of the code change

Now what I don’t show is I use IDA to look at the instruction set of the unmodified old DLL file, as this compes with PBD debugging files since I compiled it in debugging mode. This means when I read the assembly, it knows what function names are and its partially human readable code due to the debugging information.I see that the assembly instruction is
Push 6

6 being the 06 byte we saw in the hex code.

I then went to the obfuscated assembly of the newest DLL of demul, and searched it for all occurrences of Push 6.

Post image

Second to last one I finally see code that looks similar, and even mentions COOPERATION. So I change the hex values here.

Post image

For some reason the HEX view on all these things are different memory locations. But I use IDA to generate a DIF file, which shows me the differences , or changes I made. DIF files can be used to patch EXEs and DLLs but I ended up changing a single byte manually.

Dif file contents
This difference file has been created by IDA padDemul2.dll 00000000000282E3: 06 0A

So the address 282E3 changes from 06 to 0A.Just one byte!!

Now using a hex editor I modify that address location and….

tldr
I reversed the code of DEMUL using the alpha 0.1 git version to deduce things, guess a lot, and end up finding the single byte responsible for blocking keyboard inputs to non-window focused application so that I can play multiplayer NAOMI games with DEMUL, specifically, virtual on 2.
tools required
PEID – identify source code
IDA – view assembly of source code
WINCompare – compare the hex data of the source files before/after compile changes.

knowledge required
C
WInAPI
Windows Messaging / Direct Input
Assembly

Final result 1PC, 2 emulators, always accepting input.

2 Emulators, 1 PC, 1 input stream.


Leave a Reply

Your email address will not be published. Required fields are marked *