City of Heroes Local Map and Sandbox Mode

[dwturducken]

Source code for the Issue 1 client would be a huge bonus! Of course, I downloaded the source code for Arena (the first Morrowind) a couple of years ago, and now I don't know what to do with it. I thought it would be informative to see how some of the pieces interact, but I can't even manage to open the pieces. :)

[Joshex]

The authorization server's IP can be supplied through a command line argument when starting the client: -auth X.X.X.X

wonderful to know! I will definitely use that.

[Joshex]

If you had used a packet sniffer while it was running, you would have gotten a file filled with unreadable semi-random garbage.

The game protocol is encrypted, and the encryption key is unique to each mapserver connection. The key is agreed on by means of a Diffie-Hellman Key Exchange and never sent in the clear.

In order to decrypt the COH protocol and begin analyzing it, you need one of the following pieces of information in addition to the packet capture:

• The DH private key generated by the client.
• The DH private key generated by the server.
• The blowfish key that was mutually agreed upon and is used to encrypt the traffic.

(1) exists in client memory only briefly (less than a few microseconds) and is wiped and discarded once the DH exchange is over. (2) is the same, but exists only on the server and is impossible to get without having access to the server. (3) exists in client memory during the same small window as (1) and is discarded once the blowfish S-boxes are populated.

If you have detailed knowledge of how the client works, the approach that I would use is to use debugging tools to set breakpoints in the client and capture (1) and (3) during the connection setup phase, then save them in a file to go alongside the packet capture so that it can be decrypted later.

An alternative approach is to man-in-the-middle the connection so that you can be in control of (2) and (3), at least for half the connection. The MITM program would need to establish a separate connection to the real server and relay the traffic, but would have access to the unencrypted data as it passed through. The COH protocol does no endpoint verification so this is trivial *IF* you already know how the outer UDP protocol works.

Both of these of course require a server to connect to.

interesting, seems like it will take quite a bit of work for me to crack it, though now I have an Idea, i'll have to study the way this particular encryption is scripted, aka if I can write my fake server to propose it's own code which makes everything stay normal "A=A a=a B=B ETC." then it would technicaly decrypt itself, from where a packet sniffer will work wonders.

 

once you have enough of the game on servers, now it's time to go back and set-up IP addresses instead of ethernet addresses, IP addresses will allow others to connect from any computer accessing the internet.

Simply using IP from the start would be approximately 73% less work than implementing a new layer 3 protocol just to replace it with UDP later.

Quote

    if anyone is interested in going through with this, then post here, cause I wont be doing it allone lol.

I hear that the SEGS crew is working on reproducing a network server and has made some very good progress. Rumor has it there's also a second group that's being unnecessarily secretive about their work, but gathered a large number of packet captures before the game shut down. That's completely unsubstantiated though.

hmmm ok, seems I'll contact Segs and see what kinda a start they've got, I wish I knew who the other secretive members were though so I could contact them lol.

Anyways myabe I can help out somewhere in that category.

sorry if this meesage is in a jumble, I am at the moment nfit to do anything, I have my head wrapped on the wrong side of my head lol aka very dizzzy.

[srmalloy]

I can't help thinking that one of the former workers for the now-closed Paragon Studios might 'annonymously' slip a source code.

It's not like NCsoft treated them particularly well with the closing, amiright?

It would still likely be a violation of their terms of employment. However, even given that, I would not be surprised if a full set of client and server sources existed offsite from the Paragon Studios offices, and survived the shutdown of the company. Whether -- or how soon -- they would surface, and in what manner, is pure speculation, though.

[Joshex]

It would still likely be a violation of their terms of employment. However, even given that, I would not be surprised if a full set of client and server sources existed offsite from the Paragon Studios offices, and survived the shutdown of the company. Whether -- or how soon -- they would surface, and in what manner, is pure speculation, though.

maybe we should take the devs out to dinner again, lol *wink wink*

[Codewalker]

Success! (kinda)

It ended up being slightly more complicated than I thought. Loading the map was easy enough, as was pulling in a character.

And why not start with a random costume, just for fun:

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox1_zpsd9ae54d8.jpg

You can't see it there because I forgot to turn on screenshotui for that one, but it has the same situation as the opening post: LOST CONNECTION TO MAPSERVER as well as "You have been defeated".

So, first order of business, I changed the hitpoints to 5 to get rid of the hospital box. Out of a max HP of 0, but whatever. Then I modified some of the functions that check for server connectivity and short-circuited them to lie about being connected, which got rid of the annoying lost connection to mapserver text.

Only one problem: Still can't move, even with server sync disabled. Oh, there's no key bindings at all. Set those up, but still unable to move. Making progress, though -- the character ACTS like they're trying to move. Animations play, but they can't get off that spot.

So that's where the majority of tonight's digging was. First avenue was movement speed -- it was 0 like all other attributes. So I changed that, but still couldn't move. Until I tried to jump. I went up. And up. And up. And couldn't stop. Finally hit the zone ceiling after a long, slow rise in zero gravity.

Huh. Weird. I didn't think to grab a screenshot as I was more concerned with figuring out what went wrong. Did some more digging and discovered that properties like friction, gravity, etc. are also sent by the server. But the client keeps them on-hand in order to do motion prediction. With no server, they all defaulted to 0.

Okay, so, using 1.0 for all those for now. May not be completely correct, but RUNNING WORKS!

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox2_zps9afb459b.jpg https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox3_zpsf7684f9b.jpg

As expected, some things are missing:

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox4_zpsa43ec0fe.jpg

Trying to click on one of those locks the character in place waiting for a server response that never comes.

Poking around with the control state in memory allows access to some of the various movement modes that are simulated by the client -- ninja run, walk (though the speed wasn't slowed so it looked really strange), staggering around stunned, and the one that most people I'm sure want to know about: Flight!

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox5_zpsc7736fce.jpg

Well, sort of flight. The animations play, but I was unable to actually get OFF the ground. So there's another piece that's missing there. Vertical camera rotation wasn't working either, which was kind of weird. The key bindings for it worked, but mouselook could only swing horizontally.

Now, half of this was done manually with a debugger, so it'll still be a bit of time before I can toss something together that you guys can run, but it definitely looks workable. Not sure if it'll have the desired effect though -- it feels really empty and lonely.

[downix]

Could one hack demo mode to accept a steady stream of external data, in effect turning the client into a graphical front end?

[The Fifth Horseman]

That was something already mentioned before: The entire demo is loaded at start time.

[Tiger]

That was something already mentioned before: The entire demo is loaded at start time.

(Note to derail CodeWalker's excellent discovery.)

Well while the demo data is loaded at start time some of it is stored in non volatile plain text Exactly the same as the Demo file itself. For example I could use Cheat Engine to change the ASCII of wolf into lion, and after the demo finishes the current cycle the wolf tail would change into a lion tail.

However most of the demo data is loaded into Binary in volatile locations

[Codewalker]

Could one hack demo mode to accept a steady stream of external data, in effect turning the client into a graphical front end?

It would probably be more trouble than it's worth. The demo file gets read into a binary structure at startup. While that structure could conceivably be changed once it's loaded, you'd have to deal with syncing up an ever increasing time code with the delay values for each event, and rolling that back around at the end of the demo file.

On top of that, demoplay mode uses a simplified game loop that skips a lot of stuff. No UI, no real input processing, and no taking advantage of motion prediction to run player physics. Also you'd need to feed in MOVs directly to do animations, since the sequencers aren't used in in demo playback. The client is more than capable of running the sequencers itself, as evidenced by the power customization menu, as well as the above experiment of running around in Atlas.

So it's possible, but wouldn't be as effective as just injecting the events into the standard game loop. And at that point, for anything more complex than a simple hack, instead of mucking around in memory it would probably be better to pass the events over something like say, a network socket.

[Ironwolf]

Codewalker - can you point us to any tools we could use to learn these techniques? I am not stupid - just that I have experience in a completely different field.

[Codewalker]

Codewalker - can you point us to any tools we could use to learn these techniques? I am not stupid - just that I have experience in a completely different field.

Well, my preferred tool for 32-bit programs is OllyDbg. It's an analyzing debugger with a clean interface, support for all the usual debugging tools (hardware breakpoints, et al), the ability to save custom labels for addresses, and a very good analyzer that identifies common constructs like switch() statements and adds some comments, as well as showing things like jump targets and loops.

Since this is all at a machine code level, you need a solid understanding of x86 machine code. I like the reference here, because it includes the encodings for the various instructions in an easy to read format. If you've worked with assembly language before it should make a lot of sense, if not I'd recommend googling some tutorials on it.

If you have a C compiler available, a very good way to get in the right mindset for reverse engineering is to write some programs doing various things, then compile them to assembly (use "gcc -S" to output assembly instead of object code), and read through and see what the compiler decided to generate. Understanding how and why compilers work, especially the optimizations they do, is key to successful reverse engineering. Maybe even open up your test program in a debugger like Olly and step through one instruction at a time. In a reverse engineering project you'll be doing a lot of single stepping.

I'd definitely recommend starting with something simpler than the COH client before jumping into that. It's a complex beast and some of us have been at it for years to get to the point where we are.

[Joshex]

Success! (kinda)

It ended up being slightly more complicated than I thought. Loading the map was easy enough, as was pulling in a character.

And why not start with a random costume, just for fun:

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox1_zpsd9ae54d8.jpg

You can't see it there because I forgot to turn on screenshotui for that one, but it has the same situation as the opening post: LOST CONNECTION TO MAPSERVER as well as "You have been defeated".

So, first order of business, I changed the hitpoints to 5 to get rid of the hospital box. Out of a max HP of 0, but whatever. Then I modified some of the functions that check for server connectivity and short-circuited them to lie about being connected, which got rid of the annoying lost connection to mapserver text.

Only one problem: Still can't move, even with server sync disabled. Oh, there's no key bindings at all. Set those up, but still unable to move. Making progress, though -- the character ACTS like they're trying to move. Animations play, but they can't get off that spot.

So that's where the majority of tonight's digging was. First avenue was movement speed -- it was 0 like all other attributes. So I changed that, but still couldn't move. Until I tried to jump. I went up. And up. And up. And couldn't stop. Finally hit the zone ceiling after a long, slow rise in zero gravity.

Huh. Weird. I didn't think to grab a screenshot as I was more concerned with figuring out what went wrong. Did some more digging and discovered that properties like friction, gravity, etc. are also sent by the server. But the client keeps them on-hand in order to do motion prediction. With no server, they all defaulted to 0.

Okay, so, using 1.0 for all those for now. May not be completely correct, but RUNNING WORKS!

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox2_zps9afb459b.jpg https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox3_zpsf7684f9b.jpg

As expected, some things are missing:

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox4_zpsa43ec0fe.jpg

Trying to click on one of those locks the character in place waiting for a server response that never comes.

Poking around with the control state in memory allows access to some of the various movement modes that are simulated by the client -- ninja run, walk (though the speed wasn't slowed so it looked really strange), staggering around stunned, and the one that most people I'm sure want to know about: Flight!

https://i727.photobucket.com/albums/ww272/ft5512/sandbox/th_sbox5_zpsc7736fce.jpg

Well, sort of flight. The animations play, but I was unable to actually get OFF the ground. So there's another piece that's missing there. Vertical camera rotation wasn't working either, which was kind of weird. The key bindings for it worked, but mouselook could only swing horizontally.

Now, half of this was done manually with a debugger, so it'll still be a bit of time before I can toss something together that you guys can run, but it definitely looks workable. Not sure if it'll have the desired effect though -- it feels really empty and lonely.

very nice, however I can tell you whats wrong with the flight,

Ehem, inorder for flight to work in a GE with a constant gravity calculation; there needs to be an invisible cube (usually at the characters feet) Set to 'static' motion meaning it's not effected by gravity or collisions, it is the parent of the main ground based controll cube (also invisible) when flight is toggled on the Flight Cube is then Toggled on via a bool opperation where as the normal movement cube is toggled off via a bool operation.

the integers are True and Flase.

at least thats how I'd do flight in blender.

Also, movement should be a calculation, not a specific number, especially for walk, other things like ninja run and such add +# to the Local speed so they will still work to some extent. infact if I remember correctly ninja run negates your original run values and jump values and replaces them.

The Doors, I never liked that band.. ok on a note of your problem; you will need to set the value to a textual value verses a number value, if the value includes the location of the map that needs to be loaded the game should be able to load said map, however there is still the problem of the server response, I'm talkign to a guy right now who has actually developed a cross platform server application (sadly it can only handle 20 clients) but the performance is through the roof. anyways that will be released on the 23rd of may, yep only a few days away.

It is written in python, but according to his knowledge it can definitely communicate with outside programs no matter what they are written in. especially seeing as C++ comes natively with python pickle handles and python object code processing.

so that might be extremely handy in writing ones own server file.

[Codewalker]

The issue with doors is that they're not actually part of the map. Technically, they're entities spawned by the server, just like Ms. Liberty and your average Hellion. You can see this in demorecords, with all the "Dr" NPCs.

[Joshex]

The issue with doors is that they're not actually part of the map. Technically, they're entities spawned by the server, just like Ms. Liberty and your average Hellion. You can see this in demorecords, with all the "Dr" NPCs.

I see, a tougher problem. I really think we could make a server program the broadcasts these messages, we just need to know what to broadcast, a door will be broad cast with 2 pieces of information, Which Number of a door it is and where it leads (what map to load If mouseover and left click = True)

so say the door is G1, the server file needs to send Icon.exe a constant message

"G1=C:\Program Files\NCSoft\City of heroes\door.geom (I pulled the name out of my ass sorry if it's incorrect)
Main()
If mouseover + leftclick = True
run C:\Program Files\NCSoft\City of heroes\dooranimfile.anim (again pulling the file name out of my ass as a place keeper)
loadfile C:\Program Files\NCsoft\City of heroes\Cityhall.map (again pulling the file name out of my ass as a place keeper)

Else
goto line 1"

you get me some door numbers and I'll sort out which files do what and script it all together in blender and set it to communicate with Icon.exe

then it will be as simple as starting a blender game before opening Icon.

[Safehouse]

I had a dream last night that I'd figured out how to get this all working. It was bittersweet. Sweet, in that I was playing the game (it's shocking how perfectly my mind recalls gameplay and the storylines), bitter in that I woke up...

[Ice Trix]

But if you guys are really just frothing at the mouth, I could probably get you running around Atlas Park with a few hours of solid code time... I mean really barebones -- no powers, no NPCs, no working doors, etc. Maybe a couple days instead to do something more flexibile to load different maps rather than hardcoding it.

My goodness yes. And being able to mess around in the city with my daughter would brighten my year.

[Shenku]

I think in the short term, most people would be happy with just being able to run around in Atlas with their friends and being able to chat with folks. Powers, NPCs, Missions, inventions, stores/influence, and mapserver loading of other zones can all come later.

But that's just my opinion on the matter...

[Rain Maker]

I for one would be thrilled to simply run around Atlas again. With my very limited understanding of coding, this progress is indistinguishable from magic. Please keep up the good work.

[Triplash]

I think in the short term, most people would be happy with just being able to run around in Atlas with their friends and being able to chat with folks. Powers, NPCs, Missions, inventions, stores/influence, and mapserver loading of other zones can all come later.

But that's just my opinion on the matter...

I think that would do for now, yeah. Plus to the average player it would feel like a major step forward, which wouldn't exactly hurt people's optimism.