Titan Network
Community => Multimedia => Demo Recording => Topic started by: therain93 on December 02, 2012, 05:34:11 PM
-
It woudl be kind of nice to have a spot where people can upload demorecords with tags associated with it. Some people might find it nice to browse, others for finding certain "parts".
Any thoughts?
-
I've been trying to get something like that going since September. I guess I should get off my butt and do it myself.
-
I could definitely get behind this. In the last few months, every time I'd mention some old demo I'd played back, people in the global channels would tell me, "Throw that bad boy up on YouTube." Well, my current machine is crap, so it'd hardly be suited to rendering the demos into a shape anyone would want to see. But demo recordings themselves are nature's perfect food--compact, ripe for editing, and now with added camera freedom.
A place where people could not only post their cool stuff but also request things's they'd like to see ("Hey, anyone got a demo of Manti's wedding?"), ask for info on maps, models, MOVs, or even share tips on how to do different things would be great.
-
So, what would be an ideal solution? We want it to be free. We want it to be self-serve. We'd like to attach some meta-data to it. We want it to have some sort of search capability.
We could always default over to Google. I'm not sure what part Titan would be interested in playing. What were you cooking up mk?
-
So, what would be an ideal solution? We want it to be free. We want it to be self-serve. We'd like to attach some meta-data to it. We want it to have some sort of search capability.
We could always default over to Google. I'm not sure what part Titan would be interested in playing. What were you cooking up mk?
"Free" is a matter of hosting.
Self-serve, metadata and search have almost nothing to do with the hosting. As long as the hosting service supports scripts and a modest database, all of those can be done.
-
-
This is something I've been wracking my brain about for weeks now.
Unfortunately, I have never done anything even remotely similar and would have to start totally from scratch.
The best I was able to come up with that I am certain I could implement is having everyone upload their demos individually and send me the link along with tags and whatever metadata.
I would then manually (with help from local scripts) update a database (mysql or simple table) and update a post on this forum or a google doc from it.
People could then use their browsers search function to find content.
As you can tell, this is obviously suboptimal; so I'd greatly appreciate it if someone more skilled at this could come up with something.
-
Okay, guys... I am familiar with PHP and MySQL. Can help with the software side - it shouldn't be more than a few days' worth of effort for me.
But I need a little cooperation from you. Namely, a slightly more detailed spec of what functionalities you expect the system to provide - and I don't mean UI-wise but rather backend-wise; the UI will logically come from there.
1. What sort of information you want available regarding the demo records once they are uploaded?
2. What do you want the entries to be tagged with? Give me a few examples.
3. What other information you want to store on your website? Screenshots, character profiles (I've seen there is some interest in that among other users), chat logs (that could be associated with the demos?)?
4. Anonymous upload or user accounts? If accounts, do they need to be approved as "uploaders" first?
5. Do you want any sort of comment or rating system?
6. Do you want the content-related section of the database to be exportable by public in any part?
7. Do you have any sort of page layout already in mind (or even already designed)?
-
-
-
* There's no record about the client version in the demo file. The closest thing to "automatic" I can provide in this regard would require you filling in data on the release dates of each issue to live, test and beta servers beforehand. Then the uploader would only need to fill in the recording date and which of the three server categories it was recorded on.
* Indexing maps, FX, entities and costume parts is doable. Indexing player characters by name can be done if you like, but be aware it may lead to confusion when demos from characters named the same but existing on different servers are uploaded.
* Login code will be as self-contained as possible, so that you can replace it in the future.
-
-
Hey guys, just FYI, we've been talking about this behind the scenes among our Titan Network developers--a combo demorecord and Sentinel+ site where you could store your stuff and other people could reference it, also serving as a library in case anyone needs it. ("Crap, what is the map name for Skyway City again?")
We haven't talked about specific implementations of the idea, but personally, I imagine a site where you upload your demorecords, it pulls out metadata such as:
- Characters contained within the demorecord file and their costumes
- NPCs contained within the demorecord file, along with their costumes and/or model name
- Location of the demorecord
- Who uploaded it
- Possibly different commands executed in the demorecord, such as animations, power effects, emotes, etc.
- Maybe (within the limits of not violating people's privacy) chat that takes place
...and so on.
As for as Sentinel+ files, I'd eventually like for them to be able to be imported into CIT.
As you can imagine, though, we've all been really busy, so new development is a bit beyond our grasp at the moment. As time goes on, though, I'm hoping that the issue becomes moot if City of Heroes is somehow relaunched. Otherwise, as other activities progress, we'll probably have more time to circle back and look at this.
-
TonyV, we know you (collectively) have a lot on your plate right now. The demos will be here if and when the Titan Network needs them. I know Chaos Ex Machina was also grabbing a ton of stuff in the last days both to fill in holes in the Titan Network's info and to have something to show people when we say, "See how awesome this game was?" Whether that's links from mission articles to demo files and/or YouTube vids of playthrus, there's plenty that can be done when the dust settles a bit.
-
I once did some demohacking and created my own site that I uploaded demorecords too. It then analysed the content and made you group new animations and models into categories (that I probably nicked from Codex).
It's not been updated for years, but it still works. You can try it out here:
http://www.randomdice.com/cohdemo/
Let me know if you need any help with this project. I don't have much time but I would love to contribute.
-
I once did some demohacking and created my own site that I uploaded demorecords too. It then analysed the content and made you group new animations and models into categories (that I probably nicked from Codex).
It's not been updated for years, but it still works. You can try it out here:
http://www.randomdice.com/cohdemo/ (http://www.randomdice.com/cohdemo/)
Let me know if you need any help with this project. I don't have much time but I would love to contribute.
HOLY <bleeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeep>, DUDE! I don't have time to dig in as I have to go to work, but that's most of it right there. Outstanding!
-
I once did some demohacking and created my own site that I uploaded demorecords too. It then analysed the content and made you group new animations and models into categories (that I probably nicked from Codex).
It's not been updated for years, but it still works. You can try it out here:
http://www.randomdice.com/cohdemo/
Let me know if you need any help with this project. I don't have much time but I would love to contribute.
Nice work! Perhaps you could share notes with The Fifth Horseman, who is starting on a similar project?
-
As you can imagine, though, we've all been really busy, so new development is a bit beyond our grasp at the moment. As time goes on, though, I'm hoping that the issue becomes moot if City of Heroes is somehow relaunched. Otherwise, as other activities progress, we'll probably have more time to circle back and look at this.
TonyV, clearly you and the team are already doing a tremendous amount for the community, and are overwhelmed with projects. Since there are skilled people here with an interest in this project, would it be possible for us to try to do some of the basic work to get it started? I am reasonably certain we can get an alpha-level application running with minimal involvement from the Titan team, probably nothing more than making sure we don't use inherently incompatible systems. Spindisc has the bones of one working already, and The Fifth Horseman was about to start on a slightly more detailed version.
And yes, I am willing to put my money where my mouth is, and volunteer to coordinate the efforts. I've got fifteen years experience as an IT project manager supervising just this sort of multi-team "fast first, good later, pretty someday" insanity.
The more quickly we can get something simple in front of interested players, the less chance there is of that interest fading.
-
I did some digging and found some fragments of my old code. It dates back to 2005 using old technologies... and looking at the source it's a bit messy. :roll:
But I could clean it up and hand it over to anyone who's interested. It's nothing fancy, but it works. Basically it hacks the demofile into string rows analyzing it's content and making arrays of models, fx and stuff, then compares that to what's already in the database. I guess I need to check if the demofileformat have changed so I wont miss things. [Ok, just looked int the files and they still say Verion 2 so I guess they are compatible.] We should of course look fro other things to like costumes and maps and stuff.
Data is currently stored in an Access database.
So, should I clean this up? I could make it as a separate DLL that you could plugin wherever. Oh, and I code in Visual Basic in VS 2012. Sorry, but VB has been my main language since ... since... well, since they introduced it in 1991. Gosh! Now I seem old. :-\
-
I uploaded a file and it threw an error, but seems to work otherwise. Of course, there are some categories missing, but that is easily enough handled.
VB 2012. Heh. I remember sitting down to upgrade our applications to VB6. (And I first starting doing "web apps" using Classic ASP. Kids today and their functional languages and useful APIs...)
-
I did some digging and found some fragments of my old code. It dates back to 2005 using old technologies... and looking at the source it's a bit messy. :roll:
But I could clean it up and hand it over to anyone who's interested. It's nothing fancy, but it works. Basically it hacks the demofile into string rows analyzing it's content and making arrays of models, fx and stuff, then compares that to what's already in the database. I guess I need to check if the demofileformat have changed so I wont miss things. [Ok, just looked int the files and they still say Verion 2 so I guess they are compatible.] We should of course look fro other things to like costumes and maps and stuff.
Data is currently stored in an Access database.
So, should I clean this up? I could make it as a separate DLL that you could plugin wherever. Oh, and I code in Visual Basic in VS 2012. Sorry, but VB has been my main language since ... since... well, since they introduced it in 1991. Gosh! Now I seem old. :-\
You have a fantastic site there, spindisc. You should indeed clean it up.
I believe it's easy enough (at least IMO) to go through the current day piggs and pull out costumes and maps and stuff.
I guess I'll post this archive of mine I compiled yesterday then.
http://planz.basketcasesoftware.com/uploads/1354765161.rar
21 RTF files. Plaintext. From beta's bin.pigg. Contains maps, npc model names, all sorts of stuff.
When you see .minimap and .mapstats in there, you should be able to change them to .txt and then view them properly (working form) in demorecords.
Enjoy.
-
Have a functional demo parser now. Didn't identify 2 of 28 costume part slots, but that's something for later. Will work on the database tables to store the extracted information.
-
Perfect Fifth. Then I don't need to upgrade my old code. :D
Let me know if there's anything else I can help with.
-
I would like to take a look at your code as it is now anyhow, in case I missed something important. :)
EDIT: For public use, here's the copy of my current extractor code. It's not storing the information anywhere yet, but it will in due time:
<?php
$temp_dir="temp\\";
function parse_demo($demo_file)
{
global $temp_dir;
$allowed_extensions=array("cohdemo");
// If RAR and ZIP extensions are installed, allow upacking and processing those files.
if (extension_loaded("rar"))
$allowed_extensions[]="rar";
if (extension_loaded("zip"))
$allowed_extensions[]="zip";
if (file_exists($demo_file))
{
$ext = strtolower(pathinfo($demo_file, PATHINFO_EXTENSION));
$file_contents=file_get_contents($demo_file);
if (!in_array($ext, $allowed_extensions))
{
Error("Not an allowed extension");
die();
}
$header=substr($file_contents, 0, 4);
if ($header=="PK\x3\x4" && $ext="zip")
{
// Open the ZIP, unpack and parse any .cohdemo found
$zip = zip_open($demo_file);
if ($zip)
{
// Note: This potentially allows upload of multiple demos in a single ZIP
// Take into account when coding upload interface.
while ($zip_entry = zip_read($zip))
{
$file = basename(zip_entry_name($zip_entry));
$ext = strtolower(pathinfo($file, PATHINFO_EXTENSION));
// Ignore files that are not .cohdemos
if ($ext=="cohdemo" && zip_entry_open($zip, $zip_entry, "r"))
{
$buffer = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
zip_entry_close($zip_entry);
file_put_contents($temp_dir.basename($file), $buffer);
// Parse the unpacked file then delete it.
// Note: For actual upload script we might want to keep them or repack.
parse_demo($temp_dir.$file);
unlink($temp_dir.$file);
}
}
zip_close($zip);
}
else Error("Error reading ZIP archive!");
}
else if ($header=="Rar!" && $ext="rar")
{
Error("RAR archive support not mplemented");
// TODO: Business logic - open the RAR, unpack and parse any .cohdemo found
}
else if ($header=="1 " && $ext="cohdemo")
{
$file_contents=str_replace("\r", "", $file_contents);
$file_contents=explode("\n", $file_contents);
$base_time=0;
//$frames=array();
$costumes=array();
$extract=array('NPC','PARTSNAME','MOV','Map','FX','FXTINT', 'SEQ', 'geometry', 'textures', 'patterns');
foreach ($extract as $type)
$extracted[$type]=array();
$extracted['skin_colors']=array();
$extracted['costume_colors']=array();
$extracted['MOV_assoc']=array();
$animations=array();
$models=array();
foreach ($file_contents as $line)
{
$line=str_replace(" ", " ",$line);
$line=str_replace(" ", " ",$line);
$line=explode(" ", $line);
$time_offset=array_shift($line);
$entity=array_shift($line);
$command=array_shift($line);
$base_time+=$time_offset;
// The values are used as string keys, yes.
// It's a little illogical until you realize we want to collect unique values.
if ($entity!='')
{
//Breakdown of operations by entity/time/command.
//$frames[$entity][$base_time][$command][]=implode(" ", $line);
switch($command)
{
// We're keeping an index of models assigned to each ID
// This is what we use to determine association of animations to models
case 'NPC':
case 'SEQ':
$models[$entity]=$line[0];
$extracted[$command][$line[0]]=1;
break;
case 'MOV':
$extracted[$command][$line[0]]=1;
if (array_key_exists($entity, $models))
$animations[$models[$entity]][$line[0]]=1;
else
$animations['Player'][$line[0]]=1;
break;
case 'DEL':
unset($models[$entity]);
break;
case 'Map':
case 'DYNLIB':
$extracted[$command][$line[0]]=1;
break;
case 'PARTSNAME':
//NOTE: parts number is variable depending on game version and model
// some NPC and pet models use same structure but different part numbers
// Solution: divide them by both indices and entities
$extracted['PARTSNAME'][$entity][$base_time][]=array($line[0],$line[1],$line[2]);
$extracted['geometry'][$line[0]]=1;
$extracted['textures'][$line[1]]=1;
$extracted['patterns'][$line[2]]=1;
$extracted['costume_colors'][$line[3]]=1;
$extracted['costume_colors'][$line[4]]=1;
break;
case 'FX':
$extracted['FX'][$line[2]]=1;
break;
case 'COSTUME':
$extracted['skin_colors'][$line[1]]=1;
break;
case 'FXTINT':
$extracted['FXTINT'][$line[0]]=1;
$extracted['FXTINT'][$line[1]]=1;
break;
//case 'Chat':
default:
break;
}
}
}
$keys_to_extract=array('NPC', 'MOV', 'Map', 'FX', 'skin_colors', 'costume_colors', 'FXTINT', 'geometry', 'textures', 'patterns');
foreach ($keys_to_extract as $key)
$extracted[$key]=array_keys ($extracted[$key]);
foreach ($animations as $id => $object)
$animations[$id]=array_keys ($object);
$extracted['MOV_assoc']=$animations;
unset($animations);
$extracted['PARTSNAME']=array_values($extracted['PARTSNAME']);
foreach ($extracted['PARTSNAME'] as $entity=> $base_time)
{
$extracted['PARTSNAME'][$entity]=array_values($extracted['PARTSNAME'][$entity]);
foreach ($base_time as $costume)
{
$indexed_costume=array();
switch (count($costume))
{
//i23 and i24 costumes have 28 slots
case 28:
$indexes=array(
0=> "Lower Body: Bottoms",
1=> "Upper Body: Chest",
2=> "Head: Face",
3=> "Upper Body: Gauntlets/Left Arm",
4=> "Lower Body: Boots",
5=> "Upper Body: Belt",
6=> "Head: Hair",
9=> "Upper Body: Chest Detail",
10=> "Upper Body: Shoulders",
11=> "Back: Backpacks",
12=> "Weapons: Left",
13=> "Head: Face detail 2",
14=> "Upper Body: Arms",
15=> "Back: Capes: Mantle",
16=> "Back: Capes: Brooch",
17=> "Back: Capes: Cape",
18=> "Aura",
19=> "Lower Body: Skirts",
20=> "Upper Body: Jackets: Jacket",
21=> "Upper Body: Jackets: Sleeves",
22=> "Head: Helmet Detail 1 / Ears",
23=> "Head: Helmet Detail 2",
24=> "Weapons: Right",
25=> "",
26=> "",
27=> "Lower Body: Tail"
// Unknown: Two-part pants?
);
break;
default:
//echo "Costume piece count (".count($costume).") does not match known versions";
//print_r($costume);
//Error("Costume piece count (".count($costume).") does not match known versions");
break;
}
}
}
// print_r($extracted);
}
else
Error("I don't know what you uploaded, but it doesn't look like a demo.");
}
else Error("Failure when reading uploaded file");
}
function Error($msg)
{
echo "Fatal Error: ".$msg;
die;
}
?>
-
Here's the list of costume parts for players by internal name, might help fill in some gaps.
- Pants
- Chest
- Head
- Gloves
- Boots
- Belt
- Hair
- Face
- Eye Detail
- Chest Detail
- Shoulders
- Back
- Right Weapon
- Neck
- Right Upper Arm
- Collar
- Broach
- Cape
- Aura
- Skirt
- Top
- Sleeves
- Cranium
- Jaw
- Left Weapon
- Two Handed Left Weapon
- Two Handed Right Weapon
- Tail
-
Thank you, that is most helpful. :)
-
Harvesting data from demos works fine now. Not sure if the table structure for pattern-slot association makes much sense, but that's something for later.
-
Nice job, Horseman.
;D
-
I went and changed the way MOVs are linked with NPC names - now it will discard underscores and numbers from the end of the NPC name to get a general "NPC class" they are a part of (there's something like 70 generic female NPCs alone) and link the MOV with that.