Titan Network

Paragon Chat => Development => Topic started by: Mister Hassenpheffer on May 26, 2016, 12:04:04 AM

Title: Technical discussion of bases
Post by: Mister Hassenpheffer on May 26, 2016, 12:04:04 AM
I hope one day we will see bases in PC.

At one time on the live forums I remember someone mentioning that bases were stored on a separate server.

Might be safe to assume (hate that word) that since it would require a server that it simply wouldn't be able to happen. <---tech limitations?

But if there was a way around that...

Just think of the base contests we could hold, Imagine gamezines getting involved with the other standard media already in place.

I can dream. I can hope.
Title: Re: Technical discussion of bases
Post by: Arcana on May 26, 2016, 12:51:31 AM
I hope one day we will see bases in PC.

At one time on the live forums I remember someone mentioning that bases were stored on a separate server.

Might be safe to assume (hate that word) that since it would require a server that it simply wouldn't be able to happen. <---tech limitations?

But if there was a way around that...

Hypothetically speaking, there's a way around that.  We can extrapolate that there's a way around that because you could demorecord a base.  That means it is possible to, in effect, tell the game client "please load this geometry, it is a base" without the game servers existing.  The free-camera mode even allows you to walk around a recorded base.

Very hypothetically speaking it should be possible for Paragon Chat to send base geometry to the CoH client, and in effect teleport you into a base.  It is then just a hop, skip, jump, rocket launch, warp drive, wormhole, temporal displacement, and dimensional shift away from allowing a Paragon Chat user to create a private room that is a base, and have every user that joins that room load the base geometry like a mega costume stanza and tell the CoH client to render that.

I don't know where this is on Codewalker's roadmap, if it is on it at all, and there are likely implementation details I'm glossing over or completely unaware of that could make this far more difficult than I perceive it to be, but I think this is a stretch goal for Paragon Chat that falls within the realm of possibility moreso than most things people want Paragon Chat to do.  If I had to guess, I would guess this is the kind of thing Codewalker might take a swing at after he does the work he's mentioned about administrative access private rooms.
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 26, 2016, 02:40:45 AM
Might be safe to assume (hate that word) that since it would require a server that it simply wouldn't be able to happen. <---tech limitations?

It's just another map.  Should be possible to run it off of any server handling private maps iow missions.  The trick is where to store the base when not needed due to no players on that map.

I don't see any reason the client can't render the map by being told where things are.  The question is how to generate the map so the client has something to render.  of course what would be nice is if the map could be shared among clients which is where the server is likely needed.

Thing to keep in mind is an impressive amount of work has gone into getting the client to allow chatting and getting costumes working and being able to walk around the pre-defined zones.
/e clap

I can dream. I can hope.

We all are doing that.
Title: Re: Technical discussion of bases
Post by: Arcana on May 26, 2016, 03:00:26 AM
It's just another map.  Should be possible to run it off of any server handling private maps iow missions.  The trick is where to store the base when not needed due to no players on that map.

http://www.xmpp.org/extensions/xep-0060.html
Title: Re: Technical discussion of bases
Post by: Codewalker on May 26, 2016, 08:30:45 AM
Hypothetically speaking

Very hypothetically speaking

(https://i1359.photobucket.com/albums/q795/campweezer/forum/hypothetical_zpsbr424g4e.jpg~original)

(https://i1359.photobucket.com/albums/q795/campweezer/pc/base_zpsb813qpld.jpg~original)
Title: Re: Technical discussion of bases
Post by: LateNights on May 26, 2016, 10:50:55 AM
"Upgrade Plot"...

Oh yeah  8)

 ;D
Title: Re: Technical discussion of bases
Post by: Mister Hassenpheffer on May 26, 2016, 11:43:05 AM
Wait... what?!

Huh?

What was that?
Title: Re: Technical discussion of bases
Post by: Codewalker on May 26, 2016, 12:52:34 PM
Wait... what?!

Huh?

What was that?

Evidence that Arcana's hypothesis, and what I already knew to be true, is correct. It's possible to extract a base from a demorecord and send it to the client to load.

Even that took some doing, because the format it gets sent over the wire is not the same as what's in a demorecord. It's a compressed text format in the demo, but to get the client to load it, it needs to be sent in the binary equivalent (which the client will dutifully convert back to text to save in the demo file if recording is on). Going between the two formats requires more or less perfectly duplicating what the game calls the textparser. But that was already mostly done because it's similar to loading .bin files.

The incremental update protocol is even wackier.

It's still quite a way from being usable though. There's a few minor issues like:

Title: Re: Technical discussion of bases
Post by: Codewalker on May 26, 2016, 01:04:43 PM
I don't know where this is on Codewalker's roadmap, if it is on it at all

Same place it's always been on the roadmap (https://www.cohtitan.com/forum/index.php?topic=11126.0). :P

after he does the work he's mentioned about administrative access private rooms.

Yes, it does depend on the private instance functionality being done first. If somehow the work on the editor gets finished first I'll probably roll it into the development build for prerelease testers to play with as a local-only feature, but for people to be able to coexist in a base, both the private instance and pubsub modules have to be done.

http://www.xmpp.org/extensions/xep-0060.html

There are a number of features in Paragon Chat that are planned to be built on top of pubsub and are waiting for work on the module implementing it to be done. The simplified security model from PEP (http://xmpp.org/extensions/xep-0163.html) makes several of them more convenient to write, as pubsub is a quite complex protocol by XMPP standards, though some of the serverwide stuff will use baseline pubsub since it's not owned by specific users.

Openfire also doesn't yet support setting a default access model for new node creations (required by XEP-0163 and generally just really convenient to have). I'll probably end up having to implement that as well, and submit a patch for it back to Openfire.
Title: Re: Technical discussion of bases
Post by: Tahquitz on May 26, 2016, 01:40:23 PM
(https://i1359.photobucket.com/albums/q795/campweezer/forum/hypothetical_zpsbr424g4e.jpg~original)

(https://i1359.photobucket.com/albums/q795/campweezer/pc/base_zpsb813qpld.jpg~original)

You tease.
Title: Re: Technical discussion of bases
Post by: Codewalker on May 26, 2016, 03:24:04 PM
It's just another map.

It really isn't. There are considerable differences between bases and regular maps.

When connecting to a regular map, whether it's an open zone or an instanced mission (there is very little difference between the two), the server just sends a filename. Say, maps/City_Zones/City_01_01/City_01_01.txt. That's Atlas Park.

The client sees that filename and opens the appropriate geobin, inside of which are a bunch of GroupDefs and Refs that, handwaving away some complexity with the object library, tells the client how to build the geometry for the zone. The server has its own copy that it loads, and -- assuming the two of them are identical -- both client and server have an independent copy of the geometry and the player can move around without the client and server desyncing.

Base maps on the other hand branch off very early in the scene negotiation protocol. Instead of a filename, they send a compressed binary representation of the base itself. That format is completely different than the on-disk format for regular maps. Instead of groupdefs and refs, it contains high-level information about room layout, color schemes, base items that have been placed, even the contents of storage bins (ever wondered why the base would flicker and redraw itself when putting stuff in storage?).

When the client receives that information, it uses it to construct an in-memory map out of predefined groupdefs that contain pieces of rooms. There's a special base details file for base items as well that gets referenced. It's assumed that the server does the same thing in order to build up its representation of the scene; this is necessary to have the geometry in order to run physics and collision detection, which in COH is done serverside. This has to happen in exactly the same way on the client and the server for things to sync up; otherwise you get rubber banding.

With some dirty debugger tricks, it's actually possible to convince the client to write out the temporary map it constructed for the base in geobin (groupdef) format. That gives valuable clues about how the world is constructed from the base map and what needs to be done to replicate that process.
Title: Re: Technical discussion of bases
Post by: Codewalker on May 26, 2016, 03:38:36 PM
I see that you suggest PvP arena matches might be possible in the future - so that means you "think" combat is possible...

There's some rather huge caveats with that. The reason I specifically said arena matches (and left it in the 'maybe' category) is that:

1. It's PVP, so it can be done without having a working AI and things like NPC pathing. Sorry masterminds and controllers.

2. It's a small, controlled instance, so it can be handed off to another protocol rather than trying to sync it over XMPP. The most likely situation would be one of the players is selected as 'server', and everybody else connects to that player's Paragon Chat (or an arena bolt-on package) using either the good old fashioned mapserver protocol or something new and different. Navigating firewalls and such can be assisted by information exchanged over XMPP.

In that case, it works a lot like some first person shooter games do, with XMPP as the matchmaking server and one player being chosen as host. Or SIP where the initial contact can be through a central server, but the actual call is made direct from peer to peer. Pick your protocol, there are a dozen examples.

Would that mean AE is a possibility?

Now, I recall you mentioning AE before - specifically that it was actually more difficult than combat itself - but would AE functionality not alleviate lag that might be more prevalent in true zones?

There's nothing special about AE that would make it useful for that. AE is just a paint-by-numbers template for assembling missions out of canned pieces that sits on top of the everything else. In order for AE to exist, the ability to create missions by writing them outside of AE would have to exist first, as well as a power system, full NPC AI, a mission engine, and a system to handle dynamic spawning of enemies. If hypothetically all that stuff exists someday, there are far more interesting things it can be used for than AE.
Title: Re: Technical discussion of bases
Post by: nicoliy on May 26, 2016, 07:05:11 PM
For features on the roadmap, bases has been my most wanted item. I'll be watching this closely.

I wish I could help with figuring out how to make it work, but this is not my expertise :(. I'll be glad to test it when that day comes though.
Title: Re: Technical discussion of bases
Post by: Arcana on May 26, 2016, 07:26:29 PM
I don't see any reason the client can't render the map by being told where things are.  The question is how to generate the map so the client has something to render.  of course what would be nice is if the map could be shared among clients which is where the server is likely needed.

Paragon Chat's development prime directive seems to be "keep it compatible with any reasonable XMPP server that supports core extensions."  To that extent, I think Codewalker is loathe to implement any feature that requires a central server beyond what an XMPP server can do with core extensions.  Custom modules, for example, are currently not appealing.  That's why I mentioned, and Codewalker confirmed, XEP-0060 aka PubSub.  PubSub allows an XMPP client to "publish" data to the XMPP server that other users can "subscribe" to and fetch.  A base would then be something that only the base owner "possesses" the geometry for, but Paragon Chat could be extended to allow a player to "instance" a base by creating a new room (in the discussion channel sense) on the XMPP server and then publishing the base geometry.  When you wanted to enter someone else's base, Paragon Chat would add you to the room associated with that base (in the same way that zoning works) and also fetch the base geometry by essentially subscribing to the right stream that contains that data, and then feed it to your own local game client.  There would be no central server in the sense of a single thing "running" the base, instead bases would function logically like zones, but with the extra machinery necessary to fetch the base geometry when needed.  The only central server bit you'd need is something that a base owner could send base geometry to and other users could fetch it from, and PubSub allows a generic XMPP server that supports the appropriate (and reasonably common) XMPP extension.

Keeping everything in sync would then happen via extensions to the current XMPP encapsulated Paragon Chat messages that keep everything else currently in sync.

TL;DR: anything that can work but requires a central server falls into the category of "theoretically possible if you want to write it, but not likely to be something I think Codewalker is working to release at the current stage of Paragon Chat development."  For the feature to be on this side of "eventually" there needs to be a way to do it with basically Openfire and its currently supported extensions, plus custom (but standards-supporting) XMPP stanzas.  Bases appear to fall into that category of possible.
Title: Re: Technical discussion of bases
Post by: Cremzen on May 26, 2016, 08:25:16 PM
I understand so very little discussed, but that pic....

(https://i1359.photobucket.com/albums/q795/campweezer/pc/base_zpsb813qpld.jpg~original)

I think I just soiled myself
Title: Re: Technical discussion of bases
Post by: Tahquitz on May 26, 2016, 10:56:25 PM
Idea... maybe not a good one, but what the hell? 

Possibility of a separate editor like Titan Icon for bases?  (Naming isn't my strongsuit... Titan Hardhat? Sure there's something better.)  Something for people to play with the Base Editor off XMPP, and to make bases offline and import into Paragon Chat someday?  The app doesn't have to be released before the feature is ready necessarily.

And like Titan Icon, if there's a mode to shut off validation offline to make some truly weird stuff that wouldn't be allowed in the game (like placement limits, deleting the Base Portal, or other ideas.)  Bases that don't validate can't be uploaded to Paragon Chat, of course... just as a online toy, or to do custom demorecording with.

Or scrap the separate app idea, perhaps this could be an extension to Titan Icon later?
Title: Re: Technical discussion of bases
Post by: Arcana on May 26, 2016, 11:16:58 PM
Idea... maybe not a good one, but what the hell? 

Possibility of a separate editor like Titan Icon for bases?  (Naming isn't my strongsuit... Titan Hardhat? Sure there's something better.)  Something for people to play with the Base Editor off XMPP, and to make bases offline and import into Paragon Chat someday?  The app doesn't have to be released before the feature is ready necessarily.

And like Titan Icon, if there's a mode to shut off validation offline to make some truly weird stuff that wouldn't be allowed in the game (like placement limits, deleting the Base Portal, or other ideas.)  Bases that don't validate can't be uploaded to Paragon Chat, of course... just as a online toy, or to do custom demorecording with.

Or scrap the separate app idea, perhaps this could be an extension to Titan Icon later?

I'm not sure what the benefit of making a "stand alone" base editor would be, in the sense that implementing a method to activate the editor in Paragon Chat would allow for a stand alone editor in the degenerate case of not logging Paragon Chat into an XMPP server.  Moreover, doing so in Paragon Chat creates the obvious opportunity to allow Paragon Chat to "see" and thus save the base in a form that the larger Paragon Chat environment could use.  Even if there was a quickie way to hack the client into base editor mode, unlike with Icon and costumes the game client itself has no way to save bases explicitly (at least as far as I'm aware).  "Base Icon" might be able to shove you into the base editor, but it couldn't save your work without even more hacktitude. 

Just to make sure this is clear: Paragon Chat can already hypothetically work without connecting to anything.  You can connect to your own personal XMPP server that no one else has access to, and you can even disconnect it from everything and in effect run Paragon Chat without the need for anything else anywhere else.  If and when base editing in Paragon Chat is implemented, you will automatically have an offline editor if you want one.  As in: you could be on your laptop, in an airplane, with no wifi, connected to nothing, editing your base.

Now, editing a base while people are standing in it might require more work if you want the edits to happen dynamically.  But that could always be a 1.01 base feature: it shouldn't hold up base functionality if Paragon Chat implemented base editing, bases as rooms, and dynamic editing of base geometry as three separate and independent features (i.e. base editing can work with or without the ability to share them as room geometry, instancing bases can work without without the real time editor if there was a way for Paragon Chat to read and convert base geometry extracted from demorecords, and real time geometry editing can be a pasted-on feature on top of the other two).
Title: Re: Technical discussion of bases
Post by: Codewalker on May 27, 2016, 12:35:11 AM
Possibility of a separate editor like Titan Icon for bases?  (Naming isn't my strongsuit... Titan Hardhat? Sure there's something better.)  Something for people to play with the Base Editor off XMPP, and to make bases offline and import into Paragon Chat someday?

You want the -offline command line parameter. Once I have some time to work on the options panel UI (or find a helper who knows C/Win32), it'll also be a checkbox you can tick.
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 27, 2016, 02:25:07 AM
It really isn't. There are considerable differences between bases and regular maps.

I stand corrected.  Though I was thinking about the "running" of the map, collision detection for example, rather than how the client gets the data to construct it as someone enters.
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 27, 2016, 02:46:21 AM
Paragon Chat's development prime directive seems to be "keep it compatible with any reasonable XMPP server that supports core extensions."  To that extent, I think Codewalker is loathe to implement any feature that requires a central server beyond what an XMPP server can do with core extensions. 

Actually, I'm quite impressed with Codewalker's efforts.  Though I must admit I don't partake in the fruits of his labor.  While there are many parts of the game I love dearly, and getting bases working would certain be near the top of my list, the one thing I would miss most is combat.  And, I believe Codewalker has already stated combat will never be available.  Feel free to tell me I'm wrong about combat, Codewalker.  And for that reason alone, I don't avail myself to Paragon Chat.

I am very happy to see we have such talented people here.  Codewalker, your efforts are most appreciated.  If your efforts bring relief to even one person, then it was worth it.  I cannot be that person today.  But perhaps one day I will be able to be one of many.

Good luck and as my mice would say, "No Surrender!"
Title: Re: Technical discussion of bases
Post by: Arcana on May 27, 2016, 04:08:10 AM
Actually, I'm quite impressed with Codewalker's efforts.  Though I must admit I don't partake in the fruits of his labor.  While there are many parts of the game I love dearly, and getting bases working would certain be near the top of my list, the one thing I would miss most is combat.  And, I believe Codewalker has already stated combat will never be available.  Feel free to tell me I'm wrong about combat, Codewalker.  And for that reason alone, I don't avail myself to Paragon Chat.

Depends on what you mean by "combat."  Combat as in a reasonable replica of the way combat worked in City of Heroes?  I don't think that is doable even if Codewalker wanted to do that within the implementation limits of Paragon Chat.  I mean, technically speaking there's always a way: you could implement a mapserver as an XMPP server module that Paragon Chat could connect to, but that would be doing it just to prove you could do it: it would be intrinsically harder to make Paragon Chat support combat that way than it would be to recreate functioning game servers.

I think - but I have to point out I don't think this is a specific priority of Codewalker's - it is possible to create a semblance of combat in PC.  Something that, say, a PnP GM could use to visualize combat in a manner congruent with using Paragon Chat as a graphical replacement for tabletop wargaming.  I think one day we could "see" combat, but I don't think we will "do" combat in Paragon Chat.

Ironically, given the subject of this thread, if there is anything related to combat I think might be doable in Paragon Chat, it might be some simplified version of base raids.  Simplified people vs geometry combat in small instance room just *might* be within the limits of what Paragon Chat could do, for a number of reasons.  For example, message frequency might be tunable faster in base rooms with small numbers of people.  For another thing, geometry doesn't move.  Moving thing vs stationary thing combat is a somewhat less intractable problem than moving thing vs moving thing.

It goes without saying (but I feel compelled to keep saying) this is just informed speculation.
Title: Re: Technical discussion of bases
Post by: Codewalker on May 27, 2016, 04:24:17 AM
There are different levels of combat as well. Combat that only affects things locally on a solo instance is an entirely different beast than combat that other people can join in or witness via XMPP. And that might be enough for some people, so who knows. There are benefits to having a real powers system even if the numbers don't work over the XMPP portion, such as being able to toggle on defensive powers to activate their FX without me having to hardcode every one of them.

So who knows. But it's a long way off and there are several things that are more easily reachable and have more tangible benefits that have a higher priority. Base building is of course a huge one, not only because a number of people enjoy it, but because it allows the creation of new locations for hangout / RP / pen and paper gaming purposes.
Title: Re: Technical discussion of bases
Post by: Tahquitz on May 27, 2016, 05:25:25 AM
You want the -offline command line parameter. Once I have some time to work on the options panel UI (or find a helper who knows C/Win32), it'll also be a checkbox you can tick.

That would do the trick.  :D
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 27, 2016, 12:19:38 PM
I wasn't actually expecting an answer but thanks.  Combat for me would be both private mission and street fighting like we used to do in coh.  XMPP could handle messages about how much dmg was dealt in an attack just as it could handle messages about what attack was executed and how defense and dmg resist affected it.  The clients involved would be responsible for rendering the graphics and playing the sounds.

We used to do basically that all the time in AD&D in college over aim with our friends from high school.  So it is doable but how much work would be involved is another matter.  And, your right, Codewalker.  There is easier, low hanging fruit to go after that were also very important parts of the game itself.

There's also message brokering services available today that could handle passing messages too.  I wonder how well one of those, say rabbittmq for example, might do keeping the clients together.  It's just a thought.  Not a suggestion.  I'm sure Arcana will jump in with an answer shortly.

Anyways, good work, Codewalker.  Keep up the good work.  Who knows, maybe you'll be the one to bring back our beloved game.  If nothing else, I wouldn't be surprised if some gaming company started talking to you about building their next mmo.  That, sir, is likely the price you'll pay for succeeding here.

lol  :D
Title: Re: Technical discussion of bases
Post by: Arcana on May 27, 2016, 06:59:57 PM
There's also message brokering services available today that could handle passing messages too.  I wonder how well one of those, say rabbittmq for example, might do keeping the clients together.  It's just a thought.  Not a suggestion.  I'm sure Arcana will jump in with an answer shortly.

Ask and ye shall receive.

One of the bigger fundamental problems with trying to make for-realsies combat work in Paragon Chat is that Paragon Chat is designed to send messages only every second or so, to prevent flooding the XMPP server with a lot of messages.  Particularly because a lot of PC's messages are broadcast messages, meaning when one message is sent the XMPP server has to send a lot of copies of that message to everyone in that channel (metadata, for example, which tells PC servers where things are and what is going on goes to everyone in the same zone).

The problem is that CoH combat really requires much faster updates to work right: at least eight per second minimum would be what I would consider the floor.  And the faster things move, the more you'd need even faster updates for positioning to prevent weird ranging issues: fifteen to thirty per second.  That's a huge flood of XMPP messages being sent to XMPP servers.

Still, in very small rooms (small = few people) the message amplification might not get too bad. But the XMPP system itself would introduce significant lag overhead, which creates other potential problems.  You could decide to bypass XMPP.  Theoretically speaking you could have PC servers negotiate a "combat-enabled instance" and then initiate direct communications between them.  Since there's no central server, you'd have each PC server talking to all the others in the zone.  That's a lot more work than a CoH client does, but it might be doable.  But at this point, you've left the realm of Paragon Chat as an XMPP client.

To make *this* work, in effect every Paragon Chat has to implement a subset of mapserver capabilities: in effect each Paragon Chat server has to be able to become a single player mapserver that can then exchange data from other PC servers so everyone sees the same thing.  You'd need some method of synchronizing all the PC servers together, plus some means of generating reproducible combat.  Remember prediction effects?  Suppose your client thinks you're in range and can attack and your target's client thinks you're out of range and can't?  How do you arbitrate that?  You need a system to authoritatively resolve conflicts like that, because you don't have a central server.

You could design the system so that one player's Paragon Chat becomes a de facto server and everyone else's becomes clients to that server.  But at this point, you've created City of Heroes game servers that happen to also speak XMPP.  We're far away into the realm of "cheating" in the sense that while this works, it isn't really Paragon Chat doing combat, it is Paragon Chat growing a server appendage when no one was looking.  Which is work comparable to, and probably greater than, recreating the game servers in the first place.

Short answer is that how Paragon Chat communicates is part of the problem, but not the only problem.  And the more you drift away from using XMPP, the more you drift away from what Paragon Chat is in the first place.
Title: Re: Technical discussion of bases
Post by: Azrael on May 27, 2016, 11:08:54 PM
There are different levels of combat as well. Combat that only affects things locally on a solo instance is an entirely different beast than combat that other people can join in or witness via XMPP. And that might be enough for some people, so who knows. There are benefits to having a real powers system even if the numbers don't work over the XMPP portion, such as being able to toggle on defensive powers to activate their FX without me having to hardcode every one of them.

So who knows. But it's a long way off and there are several things that are more easily reachable and have more tangible benefits that have a higher priority. Base building is of course a huge one, not only because a number of people enjoy it, but because it allows the creation of new locations for hangout / RP / pen and paper gaming purposes.

So, an 'offline' version of 'CoH' might be possible..?  Easier because it's on a local level without have to have a 'server' co-ordinate other players interactions?  I'd be quite happy being able to activate powers locally...and fight mobs 'locally.'  But I'd like to able to have some 'peer to peer' functionality with it so you could at least team up with a combat partner.

As per your timeline in the PC thread.  It makes sense to encroach on all the low hanging fruit 1st.  It's probably easier and quicker to do this relative to combat, AI, mobs etc.  It also builds out basic functionality of the game 1st.  Eg.  Chat.  Trains.  Moving around the zones.  Travel powers.  Costume creator.  Message of the day.  Just being able to invite somebody to a team.  Or having Super Groups.  TP to a base would be a huge...being able to edit the base.  It's a big chunk of the game to many people.  I never went for it a lot.  But I know it meant a lot to many others and some bases were huge and took some...wandering through...

I was thinking of the interview you guys did in massively.  I seem to recall that AI pathfinding was consigned the status of most difficult thing to do.  Part of the charm of CoH for me...was the way the mobs animated and behaved along their 'paths' in the zones.  It never got old playing cat and mouse with them.  I have vivid and fond memories of street sweeping in Steel Canyon. 

The private map feature and being able to summon NPCs is something I'm looking forward to.  Patiently.

Azrael.

Quote
You could design the system so that one player's Paragon Chat becomes a de facto server and everyone else's becomes clients to that server.  But at this point, you've created City of Heroes game servers that happen to also speak XMPP.

Micro server?  I'd be happy if there was no 'central' server.  But 'one' just hosted locally and the 'rest of the team' was peer to peer functionality.  Ie.  To get an instance.  And a team of '8' is 'all' you need to replicate CoH combat experience on a map or for street sweeping.  Would that being easier in terms of combat traffic and 'server' stress?  (Ie.  without having to deal with thousands of others the same server?)

Well.  Who knows what form the pass off to real time combat server functionality might take.  But I guess that 'real time' combat won't be in Paragon Chat? 

I guess we're quite a while off this but Paragon Chat has given us quite a lot so far.  And as I look at the timeline for its development, it's going to restore a lot more 'CoH' functionality in the interim as I've mentioned above.  The timeline gives us things to look forward to before we get to the business end of SCORE's plan which seems to point to something 'after' Paragon Chat to give us 'what we really want.'
Title: Re: Technical discussion of bases
Post by: Arcana on May 27, 2016, 11:56:42 PM
I'd be quite happy being able to activate powers locally...and fight mobs 'locally.'

When Codewalker talks about having "a real powers system" I believe he means something that can read the embedded powers data within the client and use that to fill power trays, allow people to activate powers, and have power activation "do" what the powers database says to do.  But I think most of that is to give him a general framework to allow users to instantiate the effects associated with those powers.  In other words, the animations associated with the powers, the visual FX auras, and perhaps even the projectiles.  But to fight things you need more than "a real powers system" in the sense Codewalker mentions: you also need a combat engine that can track, evaluate, and apply power effects.  In some ways that's actually easier than the general powers engine.  But in two distinct ways it is potentially a lot more difficult.  Unless you are shooting at target dummies, combat all but requires that you either shoot at another player who is themselves moving and acting and that requires distributed synchronization, or you are shooting at an NPC which requires implementing AI to make it anything other than a destructible mayhem mission mailbox.

You could say that when it comes to everything except what Paragon Chat already does - seeing, chatting, zoning - City of Heroes is fundamentally two big things glued together.  There's a component I'll call the reality engine, and a component I'll call the action engine.  The reality engine in a sense executes the rules of the world and updates the world's state moment by moment.  It is the software that applies gravity to objects that should be affected by gravity (i.e. things without the boolean FLY set).  It is the thing that updates all attributes of all entities based on all other attributes.  In other words it is what updates your health because of your regeneration.  It updates your position based on your velocity and the surrounding geometry.  It enforces geometry: it prevents you from walking through walls.  Some of this stuff is implemented partially in the City of Heroes game client itself, but all of it would have been implemented in the CoH mapservers.  Paragon Chat doesn't have this engine, of course, so Paragon Chat recreates bits and pieces of this functionality on a case by case basis.  That's how movement powers work, for example.  How movement works at all, really.  Paragon Chat also has a sense of gravity.  It doesn't do a lot else at the moment.

The action engine is conceptually the thing that takes user inputs and ultimately resolves them into things the reality engine can handle.  It converts keypresses into commands.  It converts power tray clicks into power executions.  It also does things like convert "execute power Blaster.Ranged_Attack.Power_Bolt" into a set of distinct actions.  It tells the animation engine to execute an animation, it spawns sound and visual effects and projectiles.  It tells the combat part of the reality engine to roll some dice and perform a set of attribmod actions if those random rolls succeed.  The attribmods modify the attributes of the target of the power as they are defined to.  All of this converts a mouse click into numbers the reality engine can crunch by figuring out which power is being executed, reading and interpreting its definition, and performing the required actions.  At the moment Paragon Chat pretends to do this on a case by case basis for things like movement powers, but it is all a fake.

In the real game clicking on fly in your power tray causes the game to look up what Inherent.Flight.Fly does, and it sets your FLY boolean to true, and that's it.  Fly doesn't "do" much besides that, because the reality engine can take over: the reality engine sees your entity is flagged with FLY and doesn't apply gravity to you (technically, Fly also sets certain inertia attributes so you decelerate to a stop in a specific way if you aren't actively sending movement commands).  WASD sends acceleration inputs into the reality engine and that causes the reality engine to adjust your velocity and position in specific ways.  All of this is sort of "automatic" behavior built into the engines.  Whoever made the "Fly" power didn't have to write any code to make Fly do anything.  The game engines just processed the bits of data inside the Fly power, and flight emerges as a behavior.

In Paragon Chat, because these general purpose engines don't exist everything has to be done ad hoc.  Within the game client itself implicitly is a complete description of how every single power, including travel powers, should work but Codewalker has to basically hard code their behavior for each one he implements by hand, because Paragon Chat can't use any of that information.  *If* Codewalker makes more generic processing engines that can understand the powers database, and builds enough of the "reality engine" to understand that information, he can start leveraging the powers database itself rather than having to replicate each power's behavior manually.  Instead of implementing each and every possible toggle aura, he can implement a generic "how to turn on auras" behavior and let Paragon Chat's action engine figure out on its own how to tell Paragon Chat's reality engine which aura to turn on and how.

That is of course much harder than implementing a single power by hand, but collectively it is easier to implement generic engines than implementing every power by hand.  Codewalker is implementing things like travel powers by hand for two reasons I presume: first its better in the short term to get these critical powers working as fast as possible with short cuts, and save building more generic engines for less high priority powers.  And second, doing the work of implementing these powers by hand can greatly inform how you will ultimately want to create generic engines rather than attempting to do so completely from scratch.

All of this is the long way around to saying that when Codewalker talks about implementing "a real powers system" that doesn't necessarily mean getting combat of any kind to work.  Its really more of a statement about creating something that can understand the language of the powers database, so Paragon Chat can leverage that information to understand what powers are intended to do and translate that into things it knows how to do.  Even if it knows nothing about how to make combat work, it can still do all the other things the powers database tells it to do: play animations, spawn visual effects, etc.
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 28, 2016, 11:53:23 PM
Ask and ye shall receive.
I figured you would respond whether I said so or not.  :)
One of the bigger fundamental problems with trying to make for-realsies combat work in Paragon Chat is that Paragon Chat is designed to send messages only every second or so, to prevent flooding the XMPP server with a lot of messages.  Particularly because a lot of PC's messages are broadcast messages, meaning when one message is sent the XMPP server has to send a lot of copies of that message to everyone in that channel (metadata, for example, which tells PC servers where things are and what is going on goes to everyone in the same zone).
Now that would be a problem.  When I was watching the traffic with wireshark, I noticed a lot of extra traffic going on between some server at ncsoft and the game client during combat. Too bad I didn't save the dumps.  They might have been helpful here.  Especially if I had recorded what activities were going on at certain times in the streams.
Title: Re: Technical discussion of bases
Post by: Arcana on May 29, 2016, 12:12:13 AM
Now that would be a problem.  When I was watching the traffic with wireshark, I noticed a lot of extra traffic going on between some server at ncsoft and the game client during combat. Too bad I didn't save the dumps.  They might have been helpful here.  Especially if I had recorded what activities were going on at certain times in the streams.

It is safe to assume that a lot of that was happening in the last couple of months of the game's existence.  A lot of what we know emerges from a lot of break-neck activity then.  A lot of demorecord stuff was happening.   I was the one telling people to demorecord their bases, for example.  Free camera mode emerges as part of that (I recall it was Black Pebble that clued us into that one).  I'm aware of a couple efforts to look closely at the mechanics of how the game worked right up to shutdown, which I may have had a small hand in.
Title: Re: Technical discussion of bases
Post by: Codewalker on May 29, 2016, 02:50:46 AM
Now that would be a problem.  When I was watching the traffic with wireshark, I noticed a lot of extra traffic going on between some server at ncsoft and the game client during combat. Too bad I didn't save the dumps.  They might have been helpful here.  Especially if I had recorded what activities were going on at certain times in the streams.

Saving them would have just been a waste of disk space. The mapserver protocol is encrypted, and the encryption key is negotiated dynamically -- a unique key is used for each session. The negotiation occurs using Diffie-Hellman key exchange, in which the keys are not sent directly but rather mutually agreed upon in a way that both ends can mathematically arrive at the same answer, but an eavesdropper cannot.

Since the keys are different each time and can't be recovered from the trace, Wireshark dumps are useless. Unless you have a huge distributed network at your disposal, then you could probably bruteforce one key every 3-4 years and decrypt part of a single session (each time you zone a new connection is established and new key negotiated), if you already know what the protocol looks like and can recognize that you've guessed the correct key.
Title: Re: Technical discussion of bases
Post by: Azrael on May 29, 2016, 02:48:36 PM
Quote
When Codewalker talks about having "a real powers system" I believe he means something that can read the embedded powers data within the client and use that to fill power trays, allow people to activate powers, and have power activation "do" what the powers database says to do.  But I think most of that is to give him a general framework to allow users to instantiate the effects associated with those powers.  In other words, the animations associated with the powers, the visual FX auras, and perhaps even the projectiles. 

Well, I'm looking forward to having the 'powers' in the trays again.  Just having the 'hard coded'(?) travel powers in the trays is exciting. 

Quote
Codewalker is implementing things like travel powers by hand for two reasons I presume: first its better in the short term to get these critical powers working as fast as possible with short cuts, and save building more generic engines for less high priority powers.  And second, doing the work of implementing these powers by hand can greatly inform how you will ultimately want to create generic engines rather than attempting to do so completely from scratch.

I see.  Paragon Chat is a learning process of restoring 'CoH' functionality in a 'server' context stopping just shy of 'combat.'  It will restore 'most' functionality upto that point.  But no further.  Codewalker must know the client inside out by now.  But the server aspects.  Ie mechanisms and systems to activate the functionality of the client is another matter.  In that regard, Paragon Chat's timeline sets expectations (tempered to a degree...) and seems like it can be a graduated learning process of adding peripheral and basic functionality while in the context of running 'a' server.  While PC won't do combat...all those steps leading up to the 'passover' which surely be invaluable.  Ie.  Getting super bases back will be massive.  Just being able to 'join team' or have a 'Supergroup' are the things we used to do - those were still part of the 'game.'  Having a 'message of the day.'  To me, it's all part of the jigsaw.  Each piece is important.  I wasn't big into Supergroups...but from reading Codewalker's technical response...you can read a map but there's a list of caveats to actually wandering around one at the moment.  But it's doable?  In time?  And it will give the Paragon Chat community 'something more' to do.


Quote
Even if it knows nothing about how to make combat work, it can still do all the other things the powers database tells it to do: play animations, spawn visual effects, etc.

To be able to play animations and spawn visual effects would be nice. 

From my time of doing 3D I recall 'nulls' as being objects that can be used as 'reference' points to enable certain functionality.  Almost like a 'target' object. 

What I'm getting at.  Could 'nulls' or invisible target objects be put e.g. as a 'zero' reference point say...10-100 yards of a player character alt in Paragon Chat.  ie.  It would give you something ie.  AN invisible target 'dummy' to aim your projectile powers at.  Or even, on private maps where 'soon' you will be able to 'summon NPCs' could you use them as a 'target dummy' for a 'play animations, spawn visual effects etc.' thing?  The null or NPC summon would be fixed reference point relative to you?

Quote
Simulated attack powers, possibly usable on target dummies

From the Paragon Chat timeline.

Though it's someway off!  But it would be nice to have target dummies in the Super bases.  Like a 'danger room...'

Quote
Handoff to realtime communication service for more demanding activities, such as PVP Arena matches

Which seems to indicate PVP arena matches...that some kind of 'realtime' com' service is needed so there isn't spaghetti rubber banding during combat.  What I like about this part of the timeline is that it gives some hope of 'combat' in the future.  For PVP first.  You could team up with someone and have them be the baddy in combat while you play the hero and swap.  *Imagines dressing an alt like a Troll... 

If there will be 'summon' NPC on these private maps, a side thought, will there be 'Be_NPC' also?  Will the Superbases be like these private maps also with summon NPC functionality?

Azrael.

Quote
The action engine is conceptually the thing that takes user inputs and ultimately resolves them into things the reality engine can handle.  It converts keypresses into commands.  It converts power tray clicks into power executions.  It also does things like convert "execute power Blaster.Ranged_Attack.Power_Bolt" into a set of distinct actions.  It tells the animation engine to execute an animation, it spawns sound and visual effects and projectiles.  It tells the combat part of the reality engine to roll some dice and perform a set of attribmod actions if those random rolls succeed.  The attribmods modify the attributes of the target of the power as they are defined to.  All of this converts a mouse click into numbers the reality engine can crunch by figuring out which power is being executed, reading and interpreting its definition, and performing the required actions.  At the moment Paragon Chat pretends to do this on a case by case basis for things like movement powers, but it is all a fake.

It's a convincing fake so far. :)  There's only one Mona Lisa but that doesn't stop people having prints on their wall...

All that in a 'second,' eh? 

The breakdown of the 'reality' engine and the 'action' engine helped me to understand a little bit more of the functionality of what the server does.  And what anyone trying to 'simulate' it is up against.  A decent post that.
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 29, 2016, 07:53:41 PM
It is safe to assume that a lot of that was happening in the last couple of months of the game's existence.  A lot of what we know emerges from a lot of break-neck activity then.  A lot of demorecord stuff was happening.   I was the one telling people to demorecord their bases, for example.  Free camera mode emerges as part of that (I recall it was Black Pebble that clued us into that one).  I'm aware of a couple efforts to look closely at the mechanics of how the game worked right up to shutdown, which I may have had a small hand in.

Took many pictures of my base just in case I had a chance to rebuild it some day.  too bad it's in my personal wiki and cannot be shared with anyone without a bit of work on my part.
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 29, 2016, 08:23:58 PM
Saving them would have just been a waste of disk space. The mapserver protocol is encrypted, and the encryption key is negotiated dynamically -- a unique key is used for each session. The negotiation occurs using Diffie-Hellman key exchange, in which the keys are not sent directly but rather mutually agreed upon in a way that both ends can mathematically arrive at the same answer, but an eavesdropper cannot.

Since the keys are different each time and can't be recovered from the trace, Wireshark dumps are useless. Unless you have a huge distributed network at your disposal, then you could probably bruteforce one key every 3-4 years and decrypt part of a single session (each time you zone a new connection is established and new key negotiated), if you already know what the protocol looks like and can recognize that you've guessed the correct key.

I figured they might try something like that but encryption over udp can be a bit rough.  Most of the advanced modes like CBC over udp could easily get out of sync real fast.  Drop a single packet and all future packets are garbage.  The channel would be worthless until a rekey was done.  They can't be decrypted without the missing dropped packet.  And udp can't do anything about missing packets beyond detect them.  The higher level protocols would have to handle that.  The other part is how often they rekeyed.  Changing from zone to zone would be the obvious place, but I'm willing to bet rekeying happened inside single zones as well.  I'd be willing to bet rekeying happened with every dopped packet in fact.  Encryption happens to be one of my hobbies from college.

It was probably overkill for them to even do encryption like that for game data.  The login info certainly should have been.  But the rest was probably more expensive than what they gained from it.  I know a few years ago, they got really upset when I told them I was watching game traffic with wireshark.  Wanted me to prove I held valid licenses.  They got real quiet about it when I gave them the 12 codes I had dating all the way back to issue 2 plus all the add-ons.  And even the 1st code which was bad and the one they sent me to replace it.

But you're probably right.  Keeping the dumps probably wouldn't yield anything useful.  It was just a thought.  Even if the data wasn't encrypted, unless you knew what to look for, it would all be meaningless, random (likely binary) data anyway.

Title: Re: Technical discussion of bases
Post by: Codewalker on May 30, 2016, 02:47:53 AM
It's ECB mode, which doesn't require the packets to be received in order.

That is normally vulnerable to replay attacks, and repeated patterns can sometimes be detected, but it's less of an issue with the mapserver protocol than most. The UDP layer does its own sequence tracking and will ignore duplicate packets or packets with wildly wrong sequence numbers. Because the data is tightly packed as variable-length values into a bit stream (and not byte aligned), the exact position relative to byte boundaries can and does vary based on header information, which is constantly changing. That makes repeated patterns harder to detect as there are 8 different possible variations depending on how they straddle the byte boundaries.

It's not quite as bulletproof as block chaining, but for the type of data being sent it is good enough and holds up surprisingly well.

I've not found any evidence of key renegotiation within a stream. Given that the game is heavily instanced, the boundary of changing between maps (which initiates a new connection) was probably seen as a good enough interval for rekeying.
Title: Re: Technical discussion of bases
Post by: Arcana on May 31, 2016, 08:59:01 PM
Saving them would have just been a waste of disk space. The mapserver protocol is encrypted, and the encryption key is negotiated dynamically -- a unique key is used for each session. The negotiation occurs using Diffie-Hellman key exchange, in which the keys are not sent directly but rather mutually agreed upon in a way that both ends can mathematically arrive at the same answer, but an eavesdropper cannot.

I suppose someone could have been hopeful that the DH prime leaked one day, or that Cryptic was dumb lazy foolish enough to use 512 bit DH in their implementation.

Incidentally, Diffie-Hellman exchange is on my top ten list of most beautiful mathematical algorithms.
Title: Re: Technical discussion of bases
Post by: Azrael on May 31, 2016, 10:27:21 PM
I suppose someone could have been hopeful that the DH prime leaked one day, or that Cryptic was dumb lazy foolish enough to use 512 bit DH in their implementation.

Incidentally, Diffie-Hellman exchange is on my top ten list of most beautiful mathematical algorithms.

A shame no server source code 'leak...'

Azrael.
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 31, 2016, 11:19:12 PM
It's ECB mode, which doesn't require the packets to be received in order.

That is normally vulnerable to replay attacks, and repeated patterns can sometimes be detected, but it's less of an issue with the mapserver protocol than most. The UDP layer does its own sequence tracking and will ignore duplicate packets or packets with wildly wrong sequence numbers. Because the data is tightly packed as variable-length values into a bit stream (and not byte aligned), the exact position relative to byte boundaries can and does vary based on header information, which is constantly changing. That makes repeated patterns harder to detect as there are 8 different possible variations depending on how they straddle the byte boundaries.

It's not quite as bulletproof as block chaining, but for the type of data being sent it is good enough and holds up surprisingly well.

I've not found any evidence of key renegotiation within a stream. Given that the game is heavily instanced, the boundary of changing between maps (which initiates a new connection) was probably seen as a good enough interval for rekeying.

Very interesting.  ECB is usually not used due to being weak compared to other encoding schemes yet strong enough to survive what most companies could throw at it.  One of the spy agencies could break it but unless it contains terrorist data (perhaps 2 terrorists met on a private map to discuss operations) unlikely they'd be interested in taking a shot at breaking it.  And if it did, they'd not share their findings (too bad) though I don't know how useful it would be here either.

It wouldn't have the problem of missing data packets as each packet stands on its own.  Dupes can be safely ignored too.  Quite smart on their part.  I doubt they had any key leaks either and probably used a 1024 dh key since that was the standard for a long time.

Would be interested to know the key space and algorithm used
Title: Re: Technical discussion of bases
Post by: LadyVamp on May 31, 2016, 11:31:31 PM
I suppose someone could have been hopeful that the DH prime leaked one day, or that Cryptic was dumb lazy foolish enough to use 512 bit DH in their implementation.

Incidentally, Diffie-Hellman exchange is on my top ten list of most beautiful mathematical algorithms.

I would bet the library they used to do the encryption wouldn't have that problem.  And if they put that much thought into securing their communications, they likely used 1024.  Computers of 10 years ago could handle that and still haul pretty good.

Oh and thank you for stating mathematics and not computer science.  I hate it when programmers think encryption, keys, and digital signatures are computer science related items.  Never can get them to understand computers just make the math go faster.
Title: Re: Technical discussion of bases
Post by: Arcana on June 01, 2016, 12:35:17 AM
I would bet the library they used to do the encryption wouldn't have that problem.  And if they put that much thought into securing their communications, they likely used 1024.  Computers of 10 years ago could handle that and still haul pretty good.

Well, CoH development was started back in 2001 (and possibly back to 1999).  In 2001, there were people implementing 512 bit DH.  Heck, I saw people still implementing Cisco VPNs with DH group 1 just ten years ago, which uses 768 bit DH mod.  That's easily crackable today.

Quote
Oh and thank you for stating mathematics and not computer science.  I hate it when programmers think encryption, keys, and digital signatures are computer science related items.  Never can get them to understand computers just make the math go faster.

Well, technically speaking no algorithm is specific only to digital computing but there are algorithms I consider to be "computer" algorithms contextually rather than "mathematical" algorithms even if they aren't unique to computing.  For example, I call Diffie-Hellman a mathematical algorithm because I conceptualize the algorithm in terms of pure mathematics.  But I would be comfortable calling, say, LFSR a "computer algorithm" because its really only encountered as a digital circuit implementation.

I'm not fond of the term "computer science" in general.  First of all, there's no computer non-science: there's no generally accepted computer art major or computer fiction major per se.  Also, computers are a technology: it is almost nonsensical to talk about computer science as a science: we don't talk about "bridge science" or "skyscraper science" or "hammer and chisel science."  50% of computer science is mathematics, and 99.9% of the rest is computer engineering.  The 0.1% that is actual computer "science"?  Things almost no one learns or is taught.  Like scalability issues due to memory error rates due to cosmic ray bit flipping.  That's arguably computer science.  Quicksort is not science, it is math.  There are no scientific principles involved in devising, analyzing, or implementing sorting algorithms.

There are times I wonder if I chose EE as a major instead of computer science in part because I hated the nomenclature.
Title: Re: Technical discussion of bases
Post by: LadyVamp on June 01, 2016, 03:16:21 AM
Well, CoH development was started back in 2001 (and possibly back to 1999).  In 2001, there were people implementing 512 bit DH.  Heck, I saw people still implementing Cisco VPNs with DH group 1 just ten years ago, which uses 768 bit DH mod.  That's easily crackable today.

Granted but if their security group was worth a darn, they'd upgrade the software to take advantage of new capabilities or at least plug the holes.  But then I seem to remember some turd brain taking over their servers as admin and rebooting the one I was on as we were completing the 8 mission posi tf on the 8th mission right before you kill the bad guy...urm I mean arrest the bad guy.

Wonder whatever happened to that turkey.

I digress.  Let's put this discussion back on the bases.  That too was one of the more interesting things, and I never got to build out the rodent consortium base completely to my satisfaction.  The ultra tech equipment with the stone walls and floors and wood or stone ceilings and dark lighting was nice.  Gave the base that, "we mice carry our ultra tech but never let go of our rodent, dark hole loving roots," feeling I was trying to create.
Title: Re: Technical discussion of bases
Post by: Tahquitz on June 01, 2016, 03:38:28 AM
Codewalker, your efforts are most appreciated.  If your efforts bring relief to even one person, then it was worth it.

This.  Just this.
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on June 15, 2016, 04:48:39 PM
Quick, and hopefully not to stupid if a question. What is this powers database being spoken of? Is this a thing, or a concept? If it's real where can. I find it? Would love to start digging through it...
Title: Re: Technical discussion of bases
Post by: Arcana on June 15, 2016, 06:14:11 PM
Quick, and hopefully not to stupid if a question. What is this powers database being spoken of? Is this a thing, or a concept? If it's real where can. I find it? Would love to start digging through it...

Short version.  City of Heroes (the game) like most MMOs didn't hard code game behaviors.  Fundamentally the game was a generic engine that just processed data.  One part of that data set was a database that included a definition for every power in the game used by every player, NPC, and mission.  That data was included in the game servers and told them how the game should process game powers.  For technical reasons it was also included in the game client in the pigg files.  Although the game client itself had no ability to process most of that information (because it did not include any code for processing game powers) the data was/is still there.  Enterprising players managed to decode that data and determine the precise definition of every power and how they worked and deliver that data to the players.

Paragon Chat has no ability to perform combat, but it could one day enable players to display the visual effects (and other things) of various powers.  The powers database would tell Paragon Chat precisely what visual effects each power was supposed to display.

Two ways to peruse it.  The hard way: learn to pigg dive the game client.  Easy way: read City of Data.  City of Data translates a subset of that data into human readable form.
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on June 18, 2016, 02:28:04 AM
Thanks Arcana!

I'll look into City Of Data. I take it looking into the piggs file is not an easy thing to decipher? If it not in the realm of insane are there resources that discuss how to read the powers data?
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on June 18, 2016, 02:37:46 AM
Hey, I tried to get into city of data @ https://tomax.cohtitan.com/ but it says site is under maintanance. Any one have access to data that was stored there or no how to get the site back online?

Title: Re: Technical discussion of bases
Post by: Arcana on June 18, 2016, 02:49:56 AM
Hey, I tried to get into city of data @ https://tomax.cohtitan.com/ but it says site is under maintanance. Any one have access to data that was stored there or no how to get the site back online?

Hmm, first I've seen that.  Not sure what happened.  I'll look into it.


Just looked into it.  Apparently its Tony's fault: https://www.cohtitan.com/forum/index.php?topic=12011.0 .  I'm sure they'll have the data back up in a perusable manner eventually. 

Title: Re: Technical discussion of bases
Post by: Jeff Anderson on June 18, 2016, 04:38:36 AM
Thanks for checking!

If anyone else access to the data and can share Please let me know thanks!
Title: Re: Technical discussion of bases
Post by: Lycantropus on June 21, 2016, 01:13:30 AM
This.  Just this.
Tossing in my thanks as well for all you program savvy folks do to keep the City going!

I pop into Paragon from time to time when I'm feeling down and missing my CoH, but I have a hard time getting anyone else from my little group to come visit, so inevitably we do other things.

Without getting into all the hopeful powers talk, if the functionality of bases returns somehow, I may be able to guilt them into it, if just to see what I made  ;D

Now to remember how to stack the desks just... so.

Lyc~
Title: Re: Technical discussion of bases
Post by: Stitchified on July 01, 2016, 05:30:03 AM
Hey, I tried to get into city of data @ https://tomax.cohtitan.com/ but it says site is under maintanance. Any one have access to data that was stored there or no how to get the site back online?
What exactly is the City of Data? I thought there wasn't any code from CoH/CoV that anyone had after it shut down, is it the Paragon Chat code?
Title: Re: Technical discussion of bases
Post by: eabrace on July 01, 2016, 06:37:55 PM
What exactly is the City of Data?
City of Data was the Red Tomax site that most people would go to for information about the real numbers behind the powers in the game.  There was also contact/story arc information, but most of the time you saw people referencing City of Data, it was to discuss powers.
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on July 10, 2016, 12:42:35 AM
Arcana / or anyone else, it doesn't look like city of data is coming up anytime soon, and would love to be able to get at the powers information for an application I am writing on top of icon that turns COH into an RPG desktop.

I'm making awesome progress, getting into the pigg files to see the relationship of MOVS, FX, and sound files for a specific power would be huge, as right now I am creating powers configurations for my application by guessing. If anyone can point me to how to dig into these files to look at powers data that would be great. I know how to open them up with pigg viewer, but half time what to do with the content is completely meaningless to me.
Title: Re: Technical discussion of bases
Post by: Arcana on July 10, 2016, 04:11:01 AM
Unfortunately, the data in the pigg files is a) variable length record, b) encoded in often unusual ways, and c) often meaningless outside the context of other data (i.e. pointers, indexes, and the like).

There's an easy way to figure out what's in there and a hard way.  The hard way is the way I did it starting back in I9, which is to stare at the data in binary form and decode it like a science project.  The easy way requires some knowledge of debugging the game client: it of course knows how to read the pigg files and if you can figure out where in the game client's code the data file templates are, then you can use them yourself to read the pigg files.  This would be the way the smart people at City of Data (eventually) did it.

I can answer specific questions about specific pieces of data, maybe, if I still remember or can find some notes (its been years since I've looked at them, of course).  But I can't really answer the general question of "how do I dump all that data" because that's a non-trivial question to answer completely.

One of these days I might try to do something there, but right now I don't have the time to pick up that project again.
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on July 10, 2016, 06:05:06 PM
Thanks Arcana that is super helpful. I am looking for pretty specific information.

I am trying to find the relationship between specific powers and the FX, sounds, and MOVS that they run, and what order they run. Timing and duration would be helpful, but not absolutely essential.

I have been looking at powerset.bin and powers.bin to see if I can understand and trace
-> powers to powers sets
-> FX , MOVS and sounds that run when a power is invoked, and when a target is hit with those powers.

I *think* I am seeing a relationship between powers, power sets, and fx, but don't seem to see MOVS anywhere.

If you have any knowledge of the existing code  or the file structure to help me in my analysis that would be awesome !
Title: Re: Technical discussion of bases
Post by: Arcana on July 11, 2016, 02:51:22 AM
In the power structure there are a set of sequencer bits that define the animations the power plays on the caster.  There's a set of them corresponding to the animation sequence that is played if you aren't yet in combat mode, if you already are in combat mode, if there is an interruptible preamble, among other things.  City of Heroes players would recognize these as weapon draw, interrruptible animation, and power animation.

The power doesn't specifiy what animation is played directly.  Instead it specifies a set of animation "bits".  The way CoH animation works, entities "play bits" and something called an animation sequencer translates those bits into a specific animation.  There are technical reasons both good and (imo) incomprehensible for why this indirection happens.  Bottom line is a power plays a set of bits, that set of bits is translated by the animation sequencer into an actual animation, and that's where the MOV comes from.

Somewhere on these forums are threads that dealt with customizing Icon with a way to hack the player to play certain animations, and one of those ways was to set the specific bits the animation required.  Those threads will contain more information on the basics of how this works.  I believe they also contained a link to a decoded version of the sequence tables.  They are also in the pigg files in a sequencer.pigg or something like that.

Edit: I should add that anyone interested in understanding how the game worked to provide context to what kinds of information are contained in the game client pigg files should really be reading as much as possible all of the threads concerning Icon customization and Paragon Chat development.  Here and there is a lot of useful information about how very technical things worked, necessary to have any chance at figuring out what any of the data in the game client files means and how it is used.
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on July 13, 2016, 01:08:01 AM
Ok, so I did some digging, I managed to find a cool thread on how to call seqbits instead of MOVS by codewalker. Which I tried out. this also had a thread call playerseq.txt which maps movs to text. But nothing in there was obvious to me concerning how to relate back to powers.bin.

I also looked through Powers.bin and sequencer.bin and was at a loss as when it comes to relating the two together, mappping to FX looks pretty straightforward, and the relationship between MOVS and Seqbits also was apparent, but between powers are seqbits not so much. If you have an idea what the original post concerning how to reverse engineer some of that mapping was named Id be super grateful. Ill keep digging in the mean time

Title: Re: Technical discussion of bases
Post by: Arcana on July 13, 2016, 01:19:55 AM
Unfortunately, the bits have names in the sequence files but are identified by id number in the powers database.  So (just making up off the top of my head) a power might call COMBAT RIGHT SHOOT but in the power's definition that might be in the binary data as "10 30 42".  It was challenging to figure out the correspondence.  For myself, the rosetta stone was actually knowing what animations a power actually played, by reading demorecord files.  Then I worked backward figuring out how the sequencer worked to call that animation, and continued backward until I found the bits corresponding to the bit names.  It was ... tedious.  For extra giggles, the devs added bits all the time during development, and that sometimes caused the bits to get renumbered.  Because just appending to the end would be too easy.

Honestly, if you're going to do the "the hard way" I should point out it took me about six month of concerted effort to get most of it figured out, and I was continuing to put effort into it from I9 to practically shutdown.  There were still a couple of fields here and there I wasn't sure about at the end.  It is not for the timid. I don't want to discourage you, but I do want to prepare you for the task of trying to figure out all that data.

I'm curious, why would you need all of this data for the particular project you were describing? 
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on July 13, 2016, 02:35:12 AM
owch, that does sound like a ton of work  :o

The reason I wanted to know the relationship between a power and its dependant resources (including MOVs) is that I have a pretty ambitous project where I am extending Icon to turn into a virtual RPG tabletop, (see https://www.cohtitan.com/forum/index.php?action=profile;area=showposts;u=24847)

Features include
- persistent collections of characters and crowd
- associate character with abilities(ie sequence of MOVS, FX, sound) that can be played at the push of a button
- attack sequences and knockback
- move external characters and crowds
- full integration with Pen and Paper Champions ruleset with character creator and turn based combat

Right now I can fully associate characters with abilities, but the process of creating good abilities is tedious, as I am combing through 1000s of Movs, FX, and sound files and connecting them together from scratch. If i knew the relationships between a power and its dependent resources it would be a lot quicker for me to assemble a good repository of abilities that I could associate from scratch. Still that sounds easier than what you are describing :)

Guess I can always hope city of data comes up.

Either way Arcana really appreciate all the help you have given so far. The app I am building should be pretty cool for COH and Champions fans alike and ill be sure to share it here once I have my first MVP coded up.
Title: Re: Technical discussion of bases
Post by: Arcana on July 13, 2016, 03:10:04 AM
I know that at one time a while ago people were discussing making Icon popmenus that allowed you to trigger the animations for various powers, and I think I contributed a fair number of seqbits to that, but I can't remember now where any of that was posted.  Also, I don't recall Icon being able to spawn non-player FX, so a lot of power animations are kind of boring like say energy blast just looks like a mime playing patty-cake with an invisible friend.  Those popmenus would be a good place to start, insofar as they represent what Icon can actually do.  Maybe combinations of sequence bits don't do anything in Icon or don't do what you expect because of technical issues.  Since a lot of that work has already been done by other players, it might be easier to fill in the gaps than starting from scratch.  If you have like a specific question about a specific thing, I can try to answer that.
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on July 13, 2016, 01:26:58 PM
I'll try taking a look for any mappings done.

So far I have had pretty good results inserting FX Into costume files and loading the costumes over the toon. It doesn't work on NPCs but ok with that.

I was even able to get someone to extend icon so you can fire powers at any direction as opposed to 0,0,0 https://www.cohtitan.com/forum/index.php?topic=12106.msg209005#msg209005

I'll look for those threads you mentioned, if remember where they are please do share :)
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on July 18, 2016, 10:25:15 PM
I think I figured it out!

1)Launch Icon in NPC mode, choose the powersets you want to learn mapping of (eg: Ice Blast, Ice Control)
2)type /accesslevel 9 and ~ to activate the console, and type debugPowerCustAnim 1, this puts the power customization screen in debug mode
3) type/ tailor to get at the costume editor with your character, click next to get to the power customization screen
4) all powers you chose at launch will be available in the customizer, launch  a power and seqbits will display on the screen
5) download seqbits.ods, I found this on the forums here -> https://docs.google.com/file/d/0B5FhLhcBjWhvMEhXeDh6RUhiNFU/edit
6)search the ods for the seqbits displayed in the customizer, and you should find the matching mov!

Title: Re: Technical discussion of bases
Post by: Arcana on July 18, 2016, 11:59:50 PM
(https://images.weserv.nl/?url=cdnstatic.visualizeus.com%2Fthumbs%2F19%2Fb0%2Ftypography%2Cawesome%2Ccolor%2Cdatavisualization%2Cdesign%2Cinfographic-19b0bf4b27e145a59d72de0ca5fc35d8_h.jpg)
Title: Re: Technical discussion of bases
Post by: Jeff Anderson on July 22, 2016, 01:29:59 AM
Yo COH!!
Title: Re: Technical discussion of bases
Post by: Codewalker on August 02, 2016, 04:51:23 PM
Since all but one of my prerelease testers were asleep last night*, I'll just leave these here.

(https://i.imgur.com/5m2zfi6l.jpg) (http://imgur.com/5m2zfi6)

(https://i.imgur.com/c0lol9tl.jpg) (http://imgur.com/c0lol9t)

(https://i.imgur.com/7O7M30gl.jpg) (http://imgur.com/7O7M30g)


* Not that I blame them. What I wouldn't give for a solid 8 hours...
Title: Re: Technical discussion of bases
Post by: Ulysses Dare on August 02, 2016, 05:46:06 PM
Drool.  :o
Title: Re: Technical discussion of bases
Post by: War-Base on August 06, 2016, 08:42:45 PM
OMG!!!  Is it really possible the base editor might be functional and useable again someday soon?!?!?

That would be a dream come true for me personally!

I used to spend close to 95% of all my logged in game time making bases. The rest of the time I was touring other people's bases or finding clients to make bases for...

Sigh... the good old days...

I want to thank Codewalker and the rest of the crew for all they have done to bring back a huge and important part of many people's life.

If you can make the editor live again, you all won't just be heroes... you will be LEGENDS!
Title: Re: Technical discussion of bases
Post by: MaidMercury on August 07, 2016, 12:44:59 AM
I use to enjoy being invited to see other SG's.  :roll:
( wasn't always that way in earlier issues.)
Title: Re: Technical discussion of bases
Post by: Nyx Nought Nothing on August 07, 2016, 03:47:27 AM
Since all but one of my prerelease testers were asleep last night*, I'll just leave these here.* Not that I blame them. What I wouldn't give for a solid 8 hours...
(https://c4.staticflickr.com/6/5595/14672547915_c373c81891_o.jpg) (https://flic.kr/p/omyDj4)
Title: Re: Technical discussion of bases
Post by: dphender on August 07, 2016, 11:50:24 PM
Looks Great! It's amazing work your doing!
Title: Re: Technical discussion of bases
Post by: saipaman on August 10, 2016, 04:18:18 AM
Here is another teaser:

(https://c5.staticflickr.com/9/8828/28599418740_8ec0983a8c_c.jpg)
Title: Re: Technical discussion of bases
Post by: War-Base on August 14, 2016, 06:58:59 PM
*drool*

Now you guys have my full attention...

Will there be a way to import demorecords into the editor through PC?  I think I have demos about 30 of my bases I would love to be able to work on again... 

Are these bases in the pictures edited conversions from demos, or are they all built new from scratch?

The progress and potential of what you guys are doing is simply amazing!




(By the way, for anyone else that that wants to know or remembers me... Yes, I am the same War Base from the BaseBuilders Inc. chat group, winner (and co-winner) of the PERC base-building contests - I am occasionally sort of bad with replies to private messages, so just throwing it out there in case anyone else was curious...)
Title: Re: Technical discussion of bases
Post by: Ulysses Dare on August 14, 2016, 08:16:30 PM
(By the way, for anyone else that that wants to know or remembers me... Yes, I am the same War Base from the BaseBuilders Inc. chat group, winner (and co-winner) of the PERC base-building contests - I am occasionally sort of bad with replies to private messages, so just throwing it out there in case anyone else was curious...)

Hey WB! Long time no see.
Title: Re: Technical discussion of bases
Post by: saipaman on August 14, 2016, 09:40:58 PM

Are these bases in the pictures edited conversions from demos, or are they all built new from scratch?


The base in the picture above was loaded from a demo.
Title: Re: Technical discussion of bases
Post by: Pengy on August 15, 2016, 12:14:58 AM
Paragon Chat can't provide Personal Items yet, and standard items you place don't produce or consume resources. But it can load bases with all that stuff:

(https://i555.photobucket.com/albums/jj477/fuzzygnome01/OMGexistingbase_zpsryviezhz.jpg)

That base is loaded from a demo. There is a crafted generator and teleporters, and they even give the appearance of producing and consuming power. There's a Longbow Flier in there somewhere, too.
Title: Re: Technical discussion of bases
Post by: Codewalker on August 15, 2016, 02:04:29 PM
Will there be a way to import demorecords into the editor through PC?  I think I have demos about 30 of my bases I would love to be able to work on again... 

There already is. In the current (unfinished) prototype that people are testing, demorecords are actually the only way to save a base between sessions.

Even once PC is able to remember bases on its own, demos will of course still be supported as a means of importing ones that were saved before the shutdown.
Title: Re: Technical discussion of bases
Post by: Codewalker on August 15, 2016, 07:40:44 PM
Arcana, if you're still reading this thread, I saw something that reminded me of an earlier discussion.

In another thread there was discussion was about re-engineering things and how closely you had to get to the original implementation versus designing something that works completely differently and making it protocol-compatible. I grumbled something about the protocols in COH using a lot of implementation details in them that makes it difficult to do the latter. SG bases turn out to be a good example of this.

Bit of necessary backstory: The COH world graph consists of two things: Groupdefs - or defs for short, and Refs. Defs at the lowest level represent a single mesh, but more often reference other defs in a hierarchical manner, building up a complex object from smaller pieces, which is where the 'group' part of the name comes from. Defs are defined both in map files as well as the object library - a shared repository that contains every static 3D object in game. The defs from map files are typically more complex and might be things like a big group that instances many buildings from other defs that come from the object library.

But defs only define what groups look like, they don't actually place them anywhere. Map files also contain a list of Refs - references to specific defs with instructions on where to place them. In zone maps there are only a handful of top-level refs that point at the defs defined in that same file. One common thing to see is a ref to "grp_Geometry", the customary name of the Groupdef that all of the zone geometry is under.

When I say things like "the server just sends a map filename for the client to load", that's actually a gross simplification to keep conversations manageable. What actually happens is that the server first sends a list of files to load defs from - first all of the various object library files it depends on*, then the map itself. Then it sends the list of top-level refs that point to some of the defs just loaded.

And that's just to get a normal map loaded, that's not even the point to my story.

As I mentioned, bases work quite differently. They use a similar mechanism, but instead of the final filename in the list being the zone map, it's an empty filename followed by a compressed base structure. The client uses that structure and builds a number of defs in memory, parented by "grpbase_0" -- the 0 is presumably to identify the base so that multiple bases can be loaded during a base raid. It also adds a grpbase_0 ref to the global list of refs in memory. In theory, that means all you have to do is send a compressed base (in binary format) and the client will do all of the work of generating the geometry for it, just like when playing back a demorecord.

When I first started working on bases in Paragon Chat, I wasted a good day's worth of time on a real headscratcher of a problem. The base was being sent correctly, I verified that it was being received and the geometry was being generated in client memory, but when I actually zoned in it was an empty map. I tried all sorts of different things to try to make it work, or even do something different like crash just to see if I was on the right track.

Finally, I managed to piece together what was happening. After sending the file names (or base), it still expects a list of refs. When an empty list didn't work, I had previously tried changing that to send simply "Base" (the name I was using internally in Paragon Chat to represent the base) with no success. Only once I started watching the ref list memory area did I realize that when I was sending a "Base" ref, or no ref at all, the client was checking it against the list it already had. It then deleted the "grpbase_0" ref it had created earlier because it wasn't on the list of refs sent by the 'server'. In the end I had to modify Paragon Chat to nearly perfectly duplicate how the client was generating groupdefs from the base data in order to make it successfully load.

In hindsight I should have realized that sooner, since the normal map load procedure involves sending all of the refs as well. However, in the early days I was just duplicating the packet captures I was going by and since the map file already had the list in it, I never thought to try something different. Had I done so I would have realized just how tightly coupled the protocol is to the implementation behind it.

That's just one example, but I've run into the same thing numerous times. Even sending costumes correctly requires building up a single-instance list of costume parts in a particular order and then transmitting numeric indices into that list.
Title: Re: Technical discussion of bases
Post by: Arcana on August 15, 2016, 10:36:11 PM
Well, I was always told that the base code was something really ugly that the devs always wanted to refactor but never had the resources (or the priority) to tackle.

To me, the interesting part of learning bits of implementation was trying to figure out just exactly what the devs were thinking when they did something.  Like some of the attribute packing can be attributed to memory compression from the early days.  Some not.  Sequence bits kept getting redefined for what appeared to be no good reason, probably because of the way they decided to handle def files associated with animation.  I have no idea what the devs were thinking here, except it is possible the base implementer said "I'm not 100% sure how the object defs work precisely, so I'm going to make my own geometry system and tack it onto the end of it."

Its the only explanation for the reward system random number generator fiasco, so I would have to go with that by default.


You sure rewriting the combat engine wouldn't be easier at this point?  :p
Title: Re: Technical discussion of bases
Post by: Codewalker on August 16, 2016, 05:05:40 AM
I have no idea what the devs were thinking here, except it is possible the base implementer said "I'm not 100% sure how the object defs work precisely, so I'm going to make my own geometry system and tack it onto the end of it."

Most of the editor UI itself is clientside and needs to know the structure of the base, so it makes some sense to only send the "source" of the base rather than the geometry generated from it.

It would mean that the client and server would need to independently generate the same geometry (for collisions). However, as there is strong evidence from debug information left in the client exe that the client and server were built from the same source tree with a number of files shared in a 'Common' folder tree, it wouldn't have been difficult for them to keep those in sync.

From what I've seen of the base infrastructure, it certainly is a little odd in places and definitely looks like a rush job to get it working before the CoV release. Once you're past the initial shock of those differences from how everything else works, I'd say it's not nearly as scary as we were led to believe. No more so than anything else in the game, anyway.

You sure rewriting the combat engine wouldn't be easier at this point?  :p

Possibly. A combat engine itself wouldn't be too hard -- it's mostly just an attribute state machine. The hardest part would be implementing all of the 'special' attributes that do things like spawn pets. I consider AI pathing to be an order of magnitude harder than a combat engine.

However, working on Paragon Chat has given me some very interesting ideas and changed my perception of exactly what needs to be part of the core mapserver. While the mapserver itself I'd say absolutely has to run synchronously on a fixed 30hz clock - there's just too much that assumes that clientside to make it practical to do anything else - most things other than physics can get away with being 'close enough'. I used to think that combat absolutely had to be part of that loop, but now I'm not so sure. Since the combat clock is much slower, a separate process running on that clock, or even (gasp!) asynchronously and feeding event data to a mapserver running in a separate thread, process, or even machine on a low-latency network would probably be indistinguishable to anyone not doing demorecord analysis, and maybe even stand up to that as well.

Combine that with some ideas I have for mapserver to mapserver synchronization - also inspired by this XMPP experiment and much easier if all the mapserver is doing is entity tracking, sequencers, physics and client communication - and you have some very interesting scalability potential.
Title: Re: Technical discussion of bases
Post by: Arcana on August 16, 2016, 08:27:46 PM
However, working on Paragon Chat has given me some very interesting ideas and changed my perception of exactly what needs to be part of the core mapserver. While the mapserver itself I'd say absolutely has to run synchronously on a fixed 30hz clock - there's just too much that assumes that clientside to make it practical to do anything else - most things other than physics can get away with being 'close enough'. I used to think that combat absolutely had to be part of that loop, but now I'm not so sure. Since the combat clock is much slower, a separate process running on that clock, or even (gasp!) asynchronously and feeding event data to a mapserver running in a separate thread, process, or even machine on a low-latency network would probably be indistinguishable to anyone not doing demorecord analysis, and maybe even stand up to that as well.

Combat itself doesn't in general have to be either fast or synchronous in my opinion, but there are a lot of weird corner cases that might arise in certain places.  Range effects in particular would get hinky (i.e. in-range/out-of-range detection would require some more sophisticated kind of interpolative handlng).  From my experience, most players can't tell the difference in how combat clocking works down to about 500 milliseconds.  They are more sensitive to things happening in the right order than at the right time.
Title: Re: Technical discussion of bases
Post by: Nyghtshade on August 16, 2016, 09:19:36 PM
If we are at some point in the future able to load our bases into Paragon Chat from a demo-record file, then will other players be able to enter the base and walk around in it as well, or will the base(s) be just a sort of solo-instance?
Title: Re: Technical discussion of bases
Post by: Arcana on August 17, 2016, 12:07:54 AM
If we are at some point in the future able to load our bases into Paragon Chat from a demo-record file, then will other players be able to enter the base and walk around in it as well, or will the base(s) be just a sort of solo-instance?

I can't speak for Codewalker but I believe it is fair to say that any feature Codewalker adds to Paragon Chat will always have the specific intent of functioning within the parameters of Paragon Chat.  Which is another way of saying I don't believe Codewalker would add a feature to Paragon Chat that wouldn't work as a multiuser experience.  If it could only work single user and there was no likely path to making it work in a shared chat context, I don't think Codewalker would focus on that feature, or if he absolutely had to scratch that itch he'd more likely backport it to Icon.

Short version: if Codewalker adds it to Paragon Chat, and you can see it, everyone else will (eventually) be able to see it also.
Title: Re: Technical discussion of bases
Post by: Codewalker on August 17, 2016, 01:13:57 PM
If we are at some point in the future able to load our bases into Paragon Chat from a demo-record file, then will other players be able to enter the base and walk around in it as well, or will the base(s) be just a sort of solo-instance?

The plan is for other people to be able walk around in it (there is a sticky in the Development section with a feature roadmap).

Right now the work in progress prototype is solo only. What's undecided is if a refined version of that will go into a release build at some point, or if we'll wait until the tech to synchronize the base with other players is finished. I can see a case for it - to give people a chance to work on building their bases even before other people can visit them.
Title: Re: Technical discussion of bases
Post by: FloatingFatMan on August 17, 2016, 03:47:19 PM
What about manual sharing of the base datafile in the mean time?
Title: Re: Technical discussion of bases
Post by: saipaman on August 17, 2016, 05:00:19 PM
You could easily record a demo of your new base and then share it with someone else.   That person would then use the demo load command to pull into into PC.
Title: Re: Technical discussion of bases
Post by: War-Base on August 21, 2016, 05:30:14 PM
The plan is for other people to be able walk around in it (there is a sticky in the Development section with a feature roadmap).

Right now the work in progress prototype is solo only. What's undecided is if a refined version of that will go into a release build at some point, or if we'll wait until the tech to synchronize the base with other players is finished. I can see a case for it - to give people a chance to work on building their bases even before other people can visit them.

Some of us actually worked on bases for several months before they were "ready" to show others, so a little lead time to work on them before we started tours might be a really nice thing...   The only thing I loved more than editing my bases was to visit other people's creations (ironically, I almost never gave tours of my own stuff - I guess I am more of a voyeur than an exhibitionist).  I think it would produce the best results if everyone had the chance to get all their work fixed up to just how they wanted it shown before tours were even possible, so there aren't any feelings of being rushed or pressured.

Then again, I am probably quite biased and simply might just be impatient to get at the editor again...

Which brings up an interesting thought... I've been dusting off my base demos in anticipation of being able to work on them again, and noticed that I have demos of several people's bases that were not my own (I took a LOT of base tours). I know I always got permission from the owner before I made any recordings of their work, but that was at a point in time when we couldn't do anything but watch the recordings...

You guys mentioned the ability to import demos into the PC version of the base editor, so am I right in assuming I could then technically import someone else's base and modify it if I had a recording of it?  Going a step further... if touring bases becomes possible in PC, would others visiting your base then be able to make a demo of it while on tour, and then be able to later go on to edit up their own version of your base? 

Not sure if that would be an amazing feature, or a scary one...

I have no knowledge of the technical side of how the program works, so this is probably a stupid question, but, theoretically, COULD it be possible to prevent people from making demos while on a base tour? Or is the ability to record demos at any point something that isn't a toggle type option?  I'm not suggesting it should be a feature, or anything like that.. I was just curious if it was possible to lock up the demorecord ability of the client software for certain situations or instances.  Like, for example, could it be set up so that demorecording wasn't possible while zoned into Pocket D, but would work just fine in any other zone?

Anyway, I do thank you all for your efforts, and impatiently await the day I can stack desks and rotate bookshelves for hours on end, again...

-War Base-

Title: Re: Technical discussion of bases
Post by: FloatingFatMan on August 21, 2016, 06:33:36 PM
What's undecided is if a refined version of that will go into a release build at some point

Please do this! Be great to get started on some designs...

If we share the base file manually, can other people enter or are they literally solo only atm?
Title: Re: Technical discussion of bases
Post by: War-Base on August 21, 2016, 07:01:46 PM
Hey WB! Long time no see.

:)

Hiya Dare!  It has indeed been a long time!

In fact, if I recall correctly (but that is always questionable)... I think I was even trying to hunt you down for a while, maybe about 5 or so years ago, trying to get a tour of your entry in the PERC Disaster Base contest...  That is how long it has been...

I hope you have a demo of it, so that I can eventually get that tour...  Better late than never! :)

But it is indeed good to see some of the old great base builders, like you, and the rabbit, hanging around in here... 

Random, only semi-realated question, but do you know how to get in contact with Madame Pistacio by any chance? 

I think I still owe her a base, (or more specifically, a room designed in one of her bases) and now, may soon be able to finally pay up on that debt - if I can track her down, and get her to join PC...

-WarBase-
He who always tries to pay his debts... eventually
Title: Re: Technical discussion of bases
Post by: Ulysses Dare on August 21, 2016, 07:08:20 PM
I hope you have a demo of it, so that I can eventually get that tour...  Better late than never! :)
I just checked my demo files and sadly it appears I don't. I saved my Bases on a Budget base but not the Disaster Base apparently.

Random, only semi-realated question, but do you know how to get in contact with Madame Pistacio by any chance? 
Sorry I haven't had any contact with Madame P since the shut down.
Title: Re: Technical discussion of bases
Post by: Codewalker on August 22, 2016, 01:24:21 AM
Please do this! Be great to get started on some designs...

That's probably what's going to happen unless by some miracle finishing the pubsub stuff takes the same amount of time as fixing the remaining problems with the editor.

If we share the base file manually, can other people enter or are they literally solo only atm?

They're literally solo-only at this point. A lot of the XMPP synchronization is incompatible with the way base maps are loaded and needs redesigning to remove some of the assumptions about static zones that were necessary to get Paragon Chat out the door in the first place. The simplest way to resolve that was to split some of the code out into modular backends that can be swapped out. The base editor prototype was implemented in the "Local" backend first (which is also used for offline mode). That backend doesn't know how to synchronize with anyone over XMPP.

Once the underlying issues are fixed, which is also a prerequisite for private instance support, it will be ported to the XMPP backend.
Title: Re: Technical discussion of bases
Post by: Codewalker on August 22, 2016, 01:34:52 AM
You guys mentioned the ability to import demos into the PC version of the base editor, so am I right in assuming I could then technically import someone else's base and modify it if I had a recording of it?  Going a step further... if touring bases becomes possible in PC, would others visiting your base then be able to make a demo of it while on tour, and then be able to later go on to edit up their own version of your base?

Yes, the base has to be transmitted in full even to visit the base, just because of how the system was originally designed. Which makes sense, as even normal zone maps can and have been modded - one if these is so awesome I'm planning on including it with the official distribution once PC grows a more sophisticated patcher.

I have no knowledge of the technical side of how the program works, so this is probably a stupid question, but, theoretically, COULD it be possible to prevent people from making demos while on a base tour? Or is the ability to record demos at any point something that isn't a toggle type option?  I'm not suggesting it should be a feature, or anything like that.. I was just curious if it was possible to lock up the demorecord ability of the client software for certain situations or instances.  Like, for example, could it be set up so that demorecording wasn't possible while zoned into Pocket D, but would work just fine in any other zone?

It might be possible to lock out the command when a certain flag is set via client modifications. However, it would effectively be a voluntary system; when sending the base the flag would ask other Paragon Chat instances to kindly disable demorecording if they please.

As with anything clientside, that can be subverted. I can think of several ways off the top of my head: using a stock Issue 24 client instead of letting Paragon Chat launch the modded one, swiping the data from memory, enabling clientside accesslevel via a simple memory switch (could be done with Cheat Engine without needing sophisticated knowledge) and using the clientsave command, or even swiping it from the XMPP stream via packet capture or the debug flag.

The rule of thumb with digital data is that if you can see it, you can copy it. Trying to block that ability usually just leads to an arms race that is impossible to win.

Anyway, I do thank you all for your efforts, and impatiently await the day I can stack desks and rotate bookshelves for hours on end, again...

If you're really interested, you could join the prerelease testing program (https://www.cohtitan.com/forum/index.php?topic=11791.0) and see for yourself. It's open to anyone who wants to be a part of it. While there are still some caveats and limitations, the editor is already mostly functional at this point.
Title: Re: Technical discussion of bases
Post by: War-Base on August 23, 2016, 03:26:04 AM
I just checked my demo files and sadly it appears I don't. I saved my Bases on a Budget base but not the Disaster Base apparently.

Well, I would love to tour that base at some point if you are willing to showcase it!  In the future of course, if/when that option becomes available to us...

Plus, I would love to see any other bases you would be willing to show.  I've been suffering from some serious base tour withdrawal for years now.

Yes, that is a thing.  Just because I am probably the only one in the world with symptoms, doesn't mean it isn't real...

:)

-WarBase-
He who will tour any base, any time, for any reason!
Title: Re: Technical discussion of bases
Post by: Arcana on August 23, 2016, 03:55:06 AM
Well, I would love to tour that base at some point if you are willing to showcase it!  In the future of course, if/when that option becomes available to us...

Technically speaking, anyone can share their base(s) with anyone else if they a) demorecorded it and b) are willing to share the demorecord file.  It is specifically because this was a possibility due to the camera free flight mode that I started posting everywhere I could for people to demorecord their bases before shutdown.  If you playback a demorecord that was recorded within a base and then activate free flight mode, you can tour any part of the base without regard to how the demorecord was originally recorded.  In fact, you could have just stood in one place and did a ten second demorecord and that would be enough to wander around the entire base today.

Until the feature arrives in Paragon Chat, these would, of course, be solo excursions.
Title: Re: Technical discussion of bases
Post by: War-Base on August 23, 2016, 04:40:15 AM

If you're really interested, you could join the prerelease testing program (https://www.cohtitan.com/forum/index.php?topic=11791.0) and see for yourself. It's open to anyone who wants to be a part of it. While there are still some caveats and limitations, the editor is already mostly functional at this point.

Believe me, I have thought about it, long and hard.  I was ready to beg, bribe, and even offer my first born for a chance to work with an even barely functional base editor again, let alone a mostly functional one...

But sadly, my time is very limited these days.  I wouldn't be able to do a proper job of helping with the testing right now, and all I would really be doing is looking for "early access to new shineys", as you put it somewhere (sorry, I remember the quote, but not where I saw it exactly), and that wouldn't be right in my book.  I have done a lot of Beta Testing of games (in fact, I was one of the BTers for CoH) and when I commit to something like that, I take it quite seriously.  And I simply can't do it properly right now.

Plus, I used to sometimes spend series of 18 to 20 hour sessions editing bases.  If I could suddenly edit bases again right now, especially after the years of withdrawal, I would probably stop sleeping and miss several days of work... Non-good things would soon follow for me.  I don't think I would take well to homelessness and no internets...

It is tempting, (you really can't even begin to understand how much so), but for now, I must stay away.  For my own good, and my sense of integrity.

If there is any other less time-consuming way I can help, please let me know. I am pretty sure I worked on and owned more bases than just about anyone else in the game (every toon, on every server, on three accounts was in a different SG, and I was the base editor or owner for all of them, and that isn't counting the nearly 200 or so bases I built, sold, traded, or was hired to make for others).  Yes, I actually paid monthly for multiple accounts just so I could have more bases. I am not bragging.  I am unwell, and probably need professional help. But I digress... where was I?  Oh yeah, helping out...

For example, if you want to stress test the editor you are working on, I can send you a demo of one of my bases that used to really strain the in-game editor, so much so I never got to finish it.  Or if you want suggestions, I think I still have copies of some of the multi-page suggestion posts and messages I used to send the devs regarding improvements, issues, and simple QoL changes that they could make to the editor.  I even suggested ways they could monetize the base editing system.  And mostly they were practical suggestions, things like interface changes, displaying X,Y,Z and rotation degree numbers - not just the standard "we need moving NPCs, running water, ICON and market interface stations" most basers in the suggestion forums would make.  Though I'm not a programmer, I think I know the difference between realistic helpful suggestions, and "would be nice", "not worth the effort", and "probably never going to happen" suggestions.  I'm sure a lot of the points I made in the past are beyond the scope of your project - I think you are seeking to make editing simply work as a priority, not change things to fill wish lists - but there may be a few nuggets of helpfulness in there. If I can find them, of course.  You guys could probably re-write the entire editor from scratch before I could find any specific copy of a base suggestion post written years ago, tucked away in nearly 2 TBs of poorly labeled text files.  Besides being unwell, I am poorly organized...  But I am digressing again...

I may have some time to be of proper, more direct help in mid/late September. With my luck, by the time I have a chance to help, it will be too late and you will have already finished your work on it. But I will always be willing to help in any way I can with the time I can spare...

-WarBase-
He who likes playing with early shineys... but must also live with his (stupidly honest) conscience... 
Title: Re: Technical discussion of bases
Post by: War-Base on August 23, 2016, 05:07:43 AM
Technically speaking, anyone can share their base(s) with anyone else if they a) demorecorded it and b) are willing to share the demorecord file.  It is specifically because this was a possibility due to the camera free flight mode that I started posting everywhere I could for people to demorecord their bases before shutdown.  If you playback a demorecord that was recorded within a base and then activate free flight mode, you can tour any part of the base without regard to how the demorecord was originally recorded.  In fact, you could have just stood in one place and did a ten second demorecord and that would be enough to wander around the entire base today.

Until the feature arrives in Paragon Chat, these would, of course, be solo excursions.
By the way, I don't think I ever got to properly thank you for that advice...

Thank you!

Before the end, I did in fact make demos of all my bases, plus several others that I really loved (shout-out to Mr. Wentworth and his Country Club base!), with the owner's permission of course..

However, I must disagree with you...

In my humble opinion, solo flights through someone else's work, while interesting, is not really a "tour".  I have played back demos of others bases, and in fact, it  sort of makes me feel "creepy"...  Kind of like you are breaking into someone's house while they are away on vacation and looking at all their stuff.

Yes, I fully admit, I am weird and need help...

A proper base tour (and I have taken literally hundreds of them) involves the base maker guiding you through their work, highlighting certain things they are proud of, pointing out things you might have missed, and conveying a sense of pride in their accomplishment, especially when another baser recognises the time it took or the creativity needed to make a certain piece or room.  The chatting, and stories, and questions and suggestions are a huge part of the process, perhaps even the most important part.  But maybe that is just me...  In any case, I would far rather have someone like Dare walk me through their base and point things out, than to just simply "see" it.  If I recall correctly, there is even a YouTube video of it out there somewhere (one of the PERC contest entry videos), so anyone can see it even without the demo file. But that isn't a real base tour.  Not even close...

I think a base tour is like sex... sure, you can do it alone, but it is a much more gratifying experience when others are with you...

-WarBase-
He who needs to stop seeking porn that involves the option to "upgrade plot size" or "place personal object"...
Title: Re: Technical discussion of bases
Post by: Ulysses Dare on August 23, 2016, 06:59:43 AM
Title: Re: Technical discussion of bases
Post by: Codewalker on August 23, 2016, 12:57:46 PM
Plus, I used to sometimes spend series of 18 to 20 hour sessions editing bases.  If I could suddenly edit bases again right now, especially after the years of withdrawal, I would probably stop sleeping and miss several days of work... Non-good things would soon follow for me.  I don't think I would take well to homelessness and no internets...

That's cool, I get it. I'm quite familiar with how much of a time sink base building can be, and if you think jumping back into that would be unhealthy for you, then you're doing the right thing by staying away.

Do you want a heads up or anything when we're getting close to the public release? In case you want to plan to take a forum vacation for a while right before that, just so you don't get sucked in and have it interfere with real life?

Or if you want suggestions, I think I still have copies of some of the multi-page suggestion posts and messages I used to send the devs regarding improvements, issues, and simple QoL changes that they could make to the editor.

I already have a laundry list of ideas, but yeah, right now the priority is just to get the thing fully functional.

Things are quite a bit different now due to the way that Paragon Chat works, though. Ironically, things like NPCs in bases are an order of magnitude easier to do than anything involving a UI change.
Title: Re: Technical discussion of bases
Post by: War-Base on August 24, 2016, 06:41:26 AM
Title: Re: Technical discussion of bases
Post by: War-Base on August 24, 2016, 07:13:59 AM
That's cool, I get it. I'm quite familiar with how much of a time sink base building can be, and if you think jumping back into that would be unhealthy for you, then you're doing the right thing by staying away.

Do you want a heads up or anything when we're getting close to the public release? In case you want to plan to take a forum vacation for a while right before that, just so you don't get sucked in and have it interfere with real life?

I already have a laundry list of ideas, but yeah, right now the priority is just to get the thing fully functional.

Things are quite a bit different now due to the way that Paragon Chat works, though. Ironically, things like NPCs in bases are an order of magnitude easier to do than anything involving a UI change.

:)

No, no need for a heads up...  I will wait patiently, though eagerly, for any news when it comes, and when things are ready, I will adapt RL accordingly.  A person can go 3 days or so without sleep before they start to go insane, right?  What if you are already insane - does that mean you can go without sleep for longer?  The voices in my head aren't telling me the answer...

:)

All the base talk has had me going through my demos, and I came across something interesting that I wanted to ask about...

I have a demo of a small party in "The Matrix Room".  I'm assuming that I wouldn't be able to load that into the PC version of the base editor and start adding desks and stuff to it, would I?  Would I be correct to further assume that it is treated more like a zone, or maybe a sub-zone of Peregrin Island, and not really like an editable room or base, so it probably wouldn't even load at all into the editor?

Speaking of the Matrix Room, is it even accessible in PC?  I think some weird "nothing to see here" type message comes up when you try to find the hole in the map that grants you access to it.  Did you guys do that, or was it something the devs added in their last update?

Would you guys ever consider adding it to the /mapmove list?  Perhaps in the extradimentional section?

-WarBase-
He who took the red pill and the blue pill together, and now sees the reality in the virtual world...
Title: Re: Technical discussion of bases
Post by: Arcana on August 24, 2016, 08:37:58 PM
-WarBase-
He who took the red pill and the blue pill together, and now sees the reality in the virtual world...

I thought that meant you saw the virtual world in 3D, but only with special glasses.
Title: Re: Technical discussion of bases
Post by: GamingGlen on August 25, 2016, 06:09:26 PM
Would it be possible to spawn base items in Icon?  I don't need a base, just the ability to spawn the base items on the regular maps.
Title: Re: Technical discussion of bases
Post by: Codewalker on August 25, 2016, 06:50:21 PM
I really doubt it. Base items aren't spawned. When you place one, the entire "world" map is rebuilt in order to have that item in the correct place. They aren't entities like NPCs but static props just like the Atlas Statue or City Hall. It's also done clientside and only works in a base map.

The only exceptions would be the items that use an entity so that they can be targeted in a base raid. Even those typically have an invisible costume and use a special override to create the geometry. Generators and control items might be able to be spawned in Icon. I know for sure the turrets wouldn't work.
Title: Re: Technical discussion of bases
Post by: AmberOfDzu on August 25, 2016, 07:10:02 PM
I really doubt it. Base items aren't spawned. When you place one, the entire "world" map is rebuilt in order to have that item in the correct place. They aren't entities like NPCs but static props just like the Atlas Statue or City Hall. It's also done clientside and only works in a base map.
Which explains why bases were so awful with flashing while someone else was editing them?
Title: Re: Technical discussion of bases
Post by: Codewalker on August 25, 2016, 07:15:36 PM
Yes, that's the base being rebuilt. It takes a frame or two for all of the vertex lighting to be recalculated.
Title: Re: Technical discussion of bases
Post by: GamingGlen on August 25, 2016, 08:36:43 PM
Drats.  I mostly want the regular furniture items.  Thanks for the explanations.
Title: Re: Technical discussion of bases
Post by: Nyghtshade on August 29, 2016, 06:11:09 AM
Back when COH was up and running, I know around the holidays each year many base-builders would ask the Devs if we could ever import "Christmas Trees" into the bases (even like the white-lit ones around the Ski Chalet,, but it never happened.  Once the bugs are worked out and the bases are up and running again in Paragon Chat, might there be any chance of importing lit fir trees like the ones in the Chalet, or will we only be working with what was available at shutdown?
Title: Re: Technical discussion of bases
Post by: War-Base on August 29, 2016, 07:30:14 AM
Back when COH was up and running, I know around the holidays each year many base-builders would ask the Devs if we could ever import "Christmas Trees" into the bases (even like the white-lit ones around the Ski Chalet,, but it never happened.  Once the bugs are worked out and the bases are up and running again in Paragon Chat, might there be any chance of importing lit fir trees like the ones in the Chalet, or will we only be working with what was available at shutdown?

Even if we can't ever get actual trees imported, with a little creativity using the limited trees, lights, plants and colored bottles we had to play with, some pretty nice custom Christmas trees could be built.  They tended to be a bit on the large scale, but they were still passable and obvious as to what they were.

Though, an actual tree in the proper scale without the hours of work needed to make it look pretty might be a nice option...

:)


-WarBase-
He who had a Christmas tree up year round in his main base...  Mainly because he didn't want to spend a few hours re-making it every holiday season....
Title: Re: Technical discussion of bases
Post by: Ulysses Dare on August 29, 2016, 05:07:16 PM
Like this one: (https://i472.photobucket.com/albums/rr88/TBoaz/JLN%20Embassy/screenshot_111217-13-56-56.jpg) (http://s472.photobucket.com/user/TBoaz/media/JLN%20Embassy/screenshot_111217-13-56-56.jpg.html)
I was quite proud of it, but yeah a lot of work.
Title: Re: Technical discussion of bases
Post by: War-Base on August 30, 2016, 03:22:57 AM
Like this one: (https://i472.photobucket.com/albums/rr88/TBoaz/JLN%20Embassy/screenshot_111217-13-56-56.jpg) (http://s472.photobucket.com/user/TBoaz/media/JLN%20Embassy/screenshot_111217-13-56-56.jpg.html)
I was quite proud of it, but yeah a lot of work.

Beautiful work!

Are one of those presents for me?

Is is a base tour?  Nicely wrapped in a shiney bow?

-WarBase-
He who was always on the naughty list and never got the presents he wanted...