Author Topic: Tomax's site temporarily down  (Read 32438 times)

Baaleos

  • Boss
  • ****
  • Posts: 117
Re: Tomax's site temporarily down
« Reply #20 on: January 03, 2019, 04:23:12 PM »
If anyone is interested in the data dump
Your wish is my command:

I've also provided this information to the guys over @SEGS
They needed detailed information with regards to the power implementations - as the client files can only get them so far.

https://ipfs.infura.io/ipfs/QmcsEaZBh2d6MsYg9AuNigWHeTM2o43o1GPRAbRez4T3Ki
https://gateway.ipfs.io/ipfs/QmcsEaZBh2d6MsYg9AuNigWHeTM2o43o1GPRAbRez4T3Ki
https://ipfs.io/ipfs/QmcsEaZBh2d6MsYg9AuNigWHeTM2o43o1GPRAbRez4T3Ki

Note - the JSON data for the powers is stored on something called IPFS.
Interplanetary File System is a type of hosting in the 'ether'.
The json file exists in many places - it will remain online and available so long as it is looked at regularly.
The data is static, cannot be manipulated and has no risk of tampering.

I would suggest to a moderator that they consider pinning a topic with the links to the data dump - if people keep looking at the data, then the data will remain available for all of time.

(Data dump is from version 2.4 beta)
Inside the json you will find certain things like
"PVEPowerScaleImage": "QmSMQgC7YX26UNsrHtkj1uihbqtxk2oTKU6YG2C75VEgsU"

For completeness - I took the images that show the level 50 power scale and added them to the IPFS too.
If you really want to look at the scale images, you just need to suffix the hash shown to the end of an IPFS gateway url.
Eg:
https://ipfs.infura.io/ipfs/QmSMQgC7YX26UNsrHtkj1uihbqtxk2oTKU6YG2C75VEgsU

Gives you





This data covers:
Primary, Secondary, Ancillary/Epic powers:

I am still working on Pool Powers and maybe Temporary Powers - if I can get access to Incarnate powers, I will do that too.

spectre1989

  • Minion
  • **
  • Posts: 38
Re: Tomax's site temporarily down
« Reply #21 on: January 03, 2019, 09:22:25 PM »
You sexy mf! Did you manage to find the stuff that wasn't on wayback machine?

Baaleos

  • Boss
  • ****
  • Posts: 117
Re: Tomax's site temporarily down
« Reply #22 on: January 03, 2019, 09:46:05 PM »
I wont know unless someone gives me an example.
Some powers were missing their individual power pages, however I was able to get the power data for them from the shared powerset pages.

This only includes Primary, Secondary and Ancillary powers.
That being said - I have been looking into Mid's Hero Designer and have found a way to write all his power data to json files.
It includes everything from Inspirations to invention enhancement proc powers and even incarnate powers.

9000 powers which creates a JSON file that is 126 mb in size

Sekoia

  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 1,848
Re: Tomax's site temporarily down
« Reply #23 on: January 03, 2019, 10:08:49 PM »
I'll try to get you an export from our database later tonight. I don't see any reason to make you scrape it from Wayback when I can just dump it for you.

Baaleos

  • Boss
  • ****
  • Posts: 117
Re: Tomax's site temporarily down
« Reply #24 on: January 03, 2019, 10:41:26 PM »
That would be very handy -
The guys over at SEGS would also benefit from it too - as they need to find some way to implement the logic for the powers behind the scenes - client files only get you so far.


Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Tomax's site temporarily down
« Reply #25 on: January 04, 2019, 02:06:24 AM »
It looks like you're doing a lot of work that you probably don't need to. I posted a dump of the powers data 6 years ago.

There is one 95% complete one that I know of, however it's not publicly available due to being fairly dependant on my specific development environment to build, and tailored in a way most non-programmers would find unwieldy to use.

Here are text dumps of some of the more useful bin files; they still take a bit of work to find the information you want (a lot of it is structured the way it is due to engine internals), but are easier to read than having to guess at the binary format.

https://docs.google.com/folder/d/0B1YFYcIfwXcoUlBKdVdjRkxHYm8/edit

Grab powers.zip from that google drive.

That would be very handy -
The guys over at SEGS would also benefit from it too - as they need to find some way to implement the logic for the powers behind the scenes - client files only get you so far.

Client files get you all you need if the client is Issue 11.5 or later. The early client that the SEGS folks are using lacks the effect data, but the devs added it in later for the real numbers display to work, and it's the full thing that the server would use to process it.

Sekoia might can get you the COD database, but I'd urge you not to use it. COD and latter-day Mids both got their data from dumps of powers.bin and related files, so the data in that DB is the same as you'll find in powers.zip above, just with some of it stripped out and in a much worse format to do anything useful with. Trust me, I built the 2nd-gen utility to extract powers data and generate csvs for COD, and it was a huge PITA due to the bizarre table structure the site wanted.

Better to use the source game data that it was built from and save yourself the headache.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Tomax's site temporarily down
« Reply #26 on: January 04, 2019, 02:44:38 AM »
Since that was dumped a while back and there may have been some minor improvements to the tools, here's a fresh dump.

https://www.dropbox.com/s/3y95iiq7x3ykxqa/cohdata.7z?dl=0

That one includes a lot more files, some of which may be more useful than others to what you're trying to do. You'll need classes.txt (which was missing from my original upload) if you want to calculate actual numbers from damage scale.

Baaleos

  • Boss
  • ****
  • Posts: 117
Re: Tomax's site temporarily down
« Reply #27 on: January 04, 2019, 03:09:43 AM »
Ah - I didnt know there was a data dump previously - as all the posts I found online were of people asking for a dump but no replies.

Thanks for the dumps - I assume it is ok to share this info with the guys at SEGS?
I also managed to extract the power data from Mids Hero Designer and json-ify it.
Not sure if anyone is interested in that etc
The only pro it might have is being in a more object oriented structure

Thanks again for the data

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Tomax's site temporarily down
« Reply #28 on: January 04, 2019, 03:27:28 AM »
Yeah, it came from the game files and was previously used for COD and other things, so I consider it public info.

Sekoia

  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 1,848
Re: Tomax's site temporarily down
« Reply #29 on: January 04, 2019, 03:34:46 AM »
It's probably not necessary given CW's data, but just in case it helps here's the database dump from Tomax:

https://drive.google.com/uc?id=1qczGH8VcOa7V0BeZcmPiD5CYUsK6ErVV&export=download

Share as you like.