I thought I would try collecting some suggestions for the upcoming emulator database. Does anyone know of any sites that have excellent hierarchical application databases? We used to have an excellent Finder-like column view, but it has some problems with newer browsers.

If you want, I could probably come up with the code for this.

I also have the beginning of a backend I was working on if you'd like that. Includes a TOSEC parser (for the most part) to populate the DB (I only have it set for Nintendo, and the files need slight modifications because modifying the files was easier than trying to parse for testing.

Wow... this thread is like a soap opera... you can go away for five years and see the same conversation!

I've still got my old SNES database code sitting around somewhere, but it's all XSLT so it won't render on older browsers (older than 5 years?).

Anyway, I'd like to eventually see this take shape... we've been trying to get it off the ground for what, 10 years now?

www.mystationhouse.com/EmuCompat/Lookup1.php

Apparently I didn't get much further than an idea.

!Mario is the only game with test data.

The funny thing is I was simply referring to the application database (not the ROM database). What is interesting is every time I say 'emulator database' everyone gets excited about a ROM database.

On the subject of an emulator database, I'm still looking for recommendations of well-implemented application database sites (such as versiontracker). Every one I see seems to leave much to be desired.

On the topic of a ROM database, this would be quite an epic endeavor. I remember working on Emu many years ago attempting to properly parse TOSEC and GoodTools. About a year ago I was talking with some MacScene members as well as longtime gamers, and we had started concocting an idea for a 'game database' which included screenshots, descriptions, and emulator compatibility. We had begun looking into easy ways to extract this information for various locations, as well as using TOSEC and GoodTools.

If there is a significant push for developing such a database, I'd be happy to begin organizing a Fellowship of the ROMs. Thoughts?

For a ROM database, is it not just simply easier to setup something similar to TMDb and have any application wanting to pull metadata from the site do it in a similar way to XBMC or Plex does (called 'scraping' in their terminology)?

The scraping aspect is not foolproof. It requires parsing filenames to guess what a movie is (a game in our scenario here) and then pull the metadata for it from the database if available. If, there is some way to 100% determine what the game is then this is even better and that marker or indicator can also be held in the online database.

Or do I wholey misunderstand the problem?

The parser I made for TOSEC DB Dat files actually works fairly well. It currently needs to have the top headers removed, but easy enough to change it (this was coded for quick results).

The system I was creating was meant to do "everything". I t was going to be a repository for all ROM info, screenshots, compatibility with emulators, game reviews, along with an emulator DB with lists of all version of emulators (put in by a moderator or something like that as new releases came out), as well as reviews of the emulators. Notes for everything. Its in very basic form, but the idea is sound.

I just think since we are looking to do one, might be the time to do both.. Even integration with Joomla accounts and account levels is trivial now

menace690 wrote:


I'd stay away from Joomla; based on my professional background, I happen to know that some major botnets these days use Joomla installs as attack vectors for site poisoning. Until the (many) security holes are fixed, a custom solution would be MUCH better.

There are two topics here right?

1. List (db) of Emualtion software that run your Roms etc. (e.g. Nestopia, Snes9x)

2. Database of games/rom info with box art, manuals, screenshots and additional information. (e.g. Super Mario Bros. 3)

The second one, I can certainly help with. I know Travis at TMDb and I could talk to him about this. Seems there would be a lot of similarities with plans here and a site like that. TMDb is going to be getting a serious redesign sometime after the summer (by me) and if anyone wanted to have a games site in a similar vein I could design something for that or help get something moving.

Mucx I sent you an email. Once I've assembled some interested people I think we might move forward with this.

Got it...and replied

Hi, I just recently got into emulation on the mac and ended up reading these forums. This sounds like a very interesting project, I would absolutely love a TMDB-style database for games. But since I have no programming skills whatsoever, I figured I could at least point out these running projects to you:

RessurectionXtras
A collection of all existing roms including coverart, synopsis, manuals and even commercials and more. Not put into a web-based database, but spread through torrents, so people can download it onto their own computers. If this could be put into a TMDB-like database (without the roms, as that would probably be illegal..)
Originated from the xbox-scene forums: http://forums.xbox-scene.com/index.php?showtopic=660135

Plex Forums (especially the posts on page 2)
You probably all know Plex, and that they are rebuilding their library into version 0.9. I happened to stumble onto this forum thread where someone says he's working with Elan (one of the main developers) to get a gaming-library solution. Maybe you get in touch with them to work together on this?

Once again, I have no technical knowledge, so no idea of how feasible any of this is. But the way I see it there is the data (from resurrectionxtras), there is the database structure (from TMDB) and there is the interface/frontend which could use this data (from Plex). Put this together and you might have a great retro-gaming experience!

Hopefully this was of any help to you guys

I know Elan at Plex. I helped out on the project in the early days along with Alexis (who produced a mac Rom frontend and launcher called EmuLaunch). Anyway, I wouldnt hold your breathe about such a thing in Plex unless there is a dedicated developer with a real passion to make it happen and keep it ticking over as a plugin. I know the main Plex developers have their own agenda for the project and things they want to see/do and from my knowledge none of them were serious gamers...but you never know? Hard to gauge as they were a private bunch who kept their plans close to their chest.

I have an app I am producing with a developer friend that would be a good fit for ROM management and launching. So, there are options in that regard and the game aspect can be free. Not sure about PSX or N64 stuff where emulation is flakey and require messing about with settings to get a game to run...need to fit the "just works" mantra. All conjecture though...my app isnt ready to debut until Q4 and games would be Q1. (OpenEmu seems like a project that is crying for a frontend to their code...that interests me for implementation). As you can see, I do have artwork ready as I'd LOVE to do it

Resurrection Xtras is interesting, never come across it before (been away from the emulation scene for a while), but it is all revolving around downloading packages of information via torrents at Underground Gamer? I think the idea here is to create a true place to look at games, artwork, videos etc online...and then engage with other people there as well. (with a good solid API for outside apps to pull info). The information they have collected would be a good starting point to fleshing out a database though if that information could be taken.

Regarding the database, having talked to Travis last night. He is willing of course to help and give advice on it. There is unfortunately no pre-made database schema for video games as his stuff is all tailored for video. But creating some sort of database network or something is one possible idea.

Anyway, I have yet to talk to Niemann properly about this all. There are a lot of options. Need to all get on the same page, organise it, plan it all out (important one for me) and go for it really.

I'm officially attached to this project as well. The databases are easy. Getting data into them, thats the hard part

Since there seems to be a bit of interest growing on this project, I thought I'd give a quick status update.

I've started constructing a basic goals, timeline, and proposal for a database which includes video game information, screenshots, speed runs, walkthroughs, cheats, emulator compatibility lists, etc. Fundamentally the goal is to create a database with the most complete collection of video game information on the web.

We have a unique approach for filling up the database, something that I'm currently putting a proof of concept together for. The goal is to achieve the most complete database of this type on the web, in on year from launch.

The system will also have an advanced tracking tool (similar to wiki), and will utilize HTML5 and other modern technologies. I am also planning an iPhone and iPad browser with some features unseen before on the platforms.

Right now our team consists of menace, who I've worked with before. He's a very talented brute-force PHP and mySQL developer. Mucx, who has extensive knowledge of projects like Plex (media centers) and others, and has also proven to be a capable designer. And I've spoken to a couple of other individuals about ways to contribute.

I currently have two people who are interested in investing significantly to the project (less than $1000 each, but something little to motivate us).

I once heard that a great idea on the internet is one you cannot keep up with, and in the past week I've heard a lot of great ideas. This project is certainly not something new (ROM Organizer from Bannister, countless websites, GoodROMs, TOSEC, etc.), but this is a consolidation of all these ideas. I have a large project deadline coming up in the next two weeks, but since I'm not one to remain idle, I'd be interested in getting started on this very quickly.

I would guess we can use this forum as a temporary 'idea' playground (as people have already shared; and thanks). And I will do my best to keep the progress reports coming.

I'm interested too, but don't know how -- or even if -- I could help.

I have coded back-ends in python, and studied mySQL and database design for an elective last semester. But that's about all the relevant practical experience I have (unless you count adding stuff to GiantBomb's custom video game wiki).

It's great to hear that you guys are seriously thinking about this project!
As I said before, the technical stuff is not for me, I'm just an interested end-user

So, if I understand it correctly, the plan is to create something similar to TMDb, but then theGameDatabase? So, a place on the internet where you can find coverart, screenshots, etc, for games? I believe the people over at ressurectionxtras have already collected and organized most of the data you would need, although I have no idea how difficult it would be to put that into a database structure..

And then as Mucx said, a good API so other developers can pull that information for use in their apps? I can see so many cool things coming from this Even though it might not be very likely, I'm still dreaming of a plex-like emulation app, where you can browse through all your old games with beautiful artwork and stuff and just launch it from there. My game collection would be so much more interesting that way!

If there is a chance to do a front-end, I will be there all over it...hard . As I mentioned above, I have an application in production (called 'Tiles') that I intend to extend to include ROM organisation and launching at some point if I can. It has fullscreen media center capabilities as well. The app needs to be released of course (late Autumn I hope) and maybe same time next year I can see about extending it to games.

There are in my opinion two things holding back some quality emulator frontends. On all platforms not just Mac OSX...

Firstly, the availability of good quality metadata (game info). The better the API and quality of data (e.g. clean hi res artwork) the better. You make the content and the API solid and at a good quality, it almost encourages developers to utilise it in the 'right way' and do exciting things with it.

Secondly, I feel the reason better emulation frontend or interfaces are not being produced is because it is about hardcore geeky developers aiming their projects at computer literate gamer geeks. In my experience in software design, be it OpenSource or at a commercial level, one thing has struck true...most developers hate UI design. [1] In the emulation world, as the 'end users' are geeks themselves they almost don't know any better or are knowledgable enough to accept that level of presentation and UI use. [2] That cycle needs to be broken.

I feel a simple to use platform needs to be setup and should treat these emulator project(s) as a plugin or part of a larger application and not seperate geeky entities that are awkward to get into and use. It would change a lot of things and setup an iTunes like environment rather than lots of different apps for lots of different ROMs with lots of different settings and lots of different folders and OCD levels...one app to rule them all in a simple way.[3]


1. I did take part in an opensource frontend called MAME Library. (see video and more by clicking here) and it was nice, but when the developer of MAME OSX was asked if he would like it to be the front-end of his application. He more or less said he had no interest in having to worry about UI work as well as the emulator core. We still built the front end and it can launch any game in MAME OSX or SDL MAME but it is annoying having to juggle between two different app setups and is far from consumer friendly (You can download MAME Library here)

2. Richard Bannister did something smart though. It looks like he created a fairly usable UI template for his emulators. Something easy he could reuse so he could concentrate on emulating the hardware and not UI design. Still, each emulator is a seperate app and requires a lot of 'app hopping' when enjoying an afternoon with your game collection on various systems.

3. I know projects like MESS and OpenEmu exist, and I encourage them, but they still seem to be designed and built by geeky developers for geeky end users. I am advocating for an iTunes organisational approach...as close to zero setup as possible so you can concentrate on building your game collection, organising it in one interface and playing them. With a nice looking engaging interface.

One of the things I want to add specifically, is a way to rate a game on a specific emulator (including version) and the settings used. It could even be done with a hierarchical system.

Pick Emu->pick version->pick subversion

Checkboxes for settings.

Granted, this would require knowing all the settings for every version of every emulator, but with a big enough community, this could be a real possibility.

This way as much info will be passed down as possible.

That means, every emulator on Mac, Windows and Linux variations along with the setting they use will need to be documented and kept up to date?

As soon as an emulator adds something new, people need to be all over it knowledge wise and change all the game settings again and then re-rate?

What about previous versions of the emulator? Do we keep hold of those as well?

What about weird quirky things like Intel vs PPC or graphics cards which can be additional peculiarities?

How feasable an undertaking is that and how much benefit is it?
Is that needed by 80% of the user base.... or 2%?

What priority would it have on a feature list for the website? (e.g more important than gameplay trailers or a forum?)

I suppose a 'Compatibility' subsection for a game listing could be a way. Select your OS, then see an initial list rating of all the latest emulators versions known to run the game...click one to see notes on best known settings. (popup available to change and see different version of that emulator).

I aska lot of questions I know...but it gets to the heart of the problem and aids the communication design needed.

(Although, if every game needs that level of tweaking, I would be harsh and say it is the emulator that needs fixed so people can use it better...ideal world I know).