Viewing a single comment thread. View all comments

rajatarya OP t1_j0ij9if wrote

Would love to talk more. DM me your work email and I will follow up to set up time. We have heard of this use case and some of us (myself included) have used Perforce ~20 years ago.

One thing I would love to learn more about is the expected overall workflow. Meaning, what do game development teams expect their workflow to be? How does XetHub (or any other tool for code & asset management) fit into that workflow?

8

ZorbaTHut t1_j0kosbx wrote

DM sent!

I'm actually not sure how much of game-industry workflow is because that's the workflow we'd want to use, and how much of it is because we're stuck with Perforce and that's the workflow Perforce supports. But in general, it's:

  • You make changes and then check them in to the main branch. Everyone else is also doing the same thing.

If you're doing a release (assuming you're working on a game-as-service that's expected to have rolling releases), then often that gets branched into a Test branch and eventually a Stable branch. Besides that, branches are extremely rare and everyone is basically just working on the main branch all the time.

The reason for a lot of this is artists and designers. I love artists and designers, but complicated tools are just a non-starter for a lot of 'em; hell, there's programmers who have trouble with Git. Perforce's GUI is actually pretty clean and easy to use even for non-technical people and turns out to be a pretty good fit here.

The other thing to note is that any replacement needs to have file locking and Unreal Engine/Unity integration. File locking already exists on Git as a plugin, Git integration exists on those two platforms as well. But if you were building something that merely kinda looked like Git but wasn't actually Git-backed, you might need to do some work there. Unity's source control plugins can be distributed as .dll's, Unreal Engine probably requires that you submit a pull request to Epic (and contact the company first so you can get it through in less than half a year, they're pretty backed up.)

The reason we can't use Git is pure size. I've found it to be a surprisingly accurate rule of thumb that the final build of a game is 10% the size of a clean checkout. So, if your game is 10gb, the raw checkout is 100gb. The X-COM 2 full install was 78gb and it would not surprise me if this means the clean checkout is 780gb.

But that's just the clean checkout of the last version! I'm not sure what all the incremental updates look like, but a lot of that size consists of binary files that traditionally do not diff very well. I'm not sure if your system will be able to make this better, since many of these files weren't designed for it - it might be one of those deals where you move a single vertex on a model and the entire file gets changed because the designers didn't bother with binary stability.

Anyway, if you're including the entire history, it would not surprise me if this regularly gets another order of magnitude, and now the repo as a whole is passing 10tb.

Git in theory now supports doing partial syncs, so it can retrieve data in-flight from the server. Git in theory also supports partial tree syncs so you can choose a set of directories that you have in your filesystem, which would also be necessary.

In practice, I think this entire product looks like:

  • Take a very recent version of Git, make sure all of that works
  • Grab an open-source Git client, make sure it's usable by artists (including those partial sync features listed above!) and doesn't bog down when you're dealing with 10tb+ repos
  • Bundle this together (respecting licenses of course)
  • Go to a few mid-size-but-reputable companies, offer them your product for free if they'll test it for you
  • Fix all the issues they report
  • Start advertising it to studios as a less expensive and less painful Perforce replacement
  • If you've done the work right, you'll have a slow but steady takeoff as people gradually gain trust in this

I have no idea if this ends up being worth the effort for your company, but god, I wish someone would do it :V

2