How I create and maintain Lumberyard projects with Git

Lately, I’ve been using the Lumberyard Github repository to stay up to date on the latest version of the engine. I have multiple submodule repositories for Gems and projects, and have updated to multiple engine versions so I thought it might be useful to post my set up and how I keep up to date.

Project setup

As recommended by the docs I have a fork of the main Lumberyard repository and a branch for my personal project.

petrocket/Lumberyard <- a fork of aws/Lumberyard
petrocket/Lumberyard:flite <- my project branch

Inside my public project branch (flite) I use git submodules that point to private repositories on Bitbucket for my custom Gems and for the actual game project assets and code.  The folder structure looks like this:

dev/
    .. all the usual Lumberyard folders ...
    flite - the game project submodule
    gems/Toolkit - ToolKit gem submodule
    gems/ProceduralMesh - ProceduralMesh gem submodule
    gems/DynamicMesh - DynamicMesh gem submodule

As you can probably guess, if I start another project and want to use one of those gems I just pull it in as a submodule. The potential gotcha with this approach is that every time you update a submodule you need to also update the repository that uses that submodule so it points to the latest commit. This all makes sense to me, but I’ve worked with other developers who find it confusing and may prefer git subtree.

Updating a project

When Amazon releases a new version of Lumberyard I use git rebase to update my master and project branch.  Sounds pretty simple, but so does the phrase “make a simple game”.  I always forget the git commands and there are some gotchas so I’m recording the process here.  

Step 1. Update the master branch of your Lumberyard fork
NOTE: this is basically the process recommended by Github but using rebase instead of merge

1. Open a command window (I like cmder but I disable the git prompt integration because it is slow in git repos)
2. Navigate to your repository root (e.g. cd c:\Amazon\Lumberyard\git)
3. Commit and push any changes made to your master and project branch locally (or stash them) before updating.
4. If you haven’t already set up a remote for the main Lumberyard repository (upstream) do that now by following these instructions from Github.  It’s a one time thing that a lot of people forget to do when they first create their project.
5. Run git fetch upstream to get latest from the main Lumberyard repository
6. Run git checkout master to switch to your master branch locally
7. Run git rebase upstream/master to rebase your master branch locally and bring it up to date with upstream.
8. Run git push origin master to push your update to your master branch up to Github (origin).  Now your master branch is up to date, but we still need to update the game branch.

Step 2. Update your personal project branch

1. Run git checkout <your project branch> to switch to your project branch locally
2. Run git rebase master to rebase your project branch changes on top of the changes made to master locally.   At this point you may have to resolve merge conflicts if you made any changes to the engine.  After you resolve those conflicts run git rebase –continue to continue the rebase until there are no more conflicts.
3. Run git push –force-with-lease origin <your project branch> to push the changes up to Github (origin).  If I have engine changes locally that I don’t want to push I will push up to a specific commit using git push –force-with-lease origin <commit>:<your project branch>.  Using –force-with-lease option is a downside of git rebase but it allows you to not have a commit for the merge operation.  There are pros/cons of rebase and merge, I prefer the rebase option.
4. Now your project branch is updated to the latest version of the engine, run git_boostrap.exe to pull down the latest binaries from Amazon.
5. After the binaries are done downloading, SetupAssistant should open automatically.  Make sure to install all the required 3rd party libraries (push the Install All orange button on the required SDKs page).
6. Next, run ProjectConfigurator and click on the Enable Gems button for your project.  This is important because if new required gems are added, this step will force an automatic check to ensure your project has those required gems.  On the Gem’s page press Save and close the ProjectConfigurator.
7. Now you are basically done and should be able to configure and compile.  There may be compile errors you need to resolve if you made engine changes or if you were using any deprecated APIs – no surprise there.

You may wonder why I don’t use a shiny GUI like Tower or SourceTree for this and the main reason is I have been unable to figure out how to only run the necessary commands in those GUIs.  Very often Tower and SourceTree try to make operations more intuitive, but that means I don’t know what actual git commands they’re actually running which makes me nervous.  The other reason I don’t use the GUIs for this is they’re typically much slower than the command line because they refresh their GUI after each command.

Future Improvements

So that’s my current upgrade process.  Right now it’s too complicated for a non-engineer in my opinion and if anything goes wrong along the way, be prepared to spend a lot of time in Google/StackOverflow figuring out git error messages and commands.  To help with the complexity, I think it’d be nice to have a shiny wizard/GUI but that doesn’t exist yet.  The other thing I’d like to have to simplify this further is to have serverless Git LFS to store all the binaries on S3 so I don’t have to run git_bootstrap.exe.  I’m currently testing this approach and if it works out I think it could be a really nice low-cost way to have a mix of public and private code and assets.

Hope this post was helpful!