One of the things that is of a small annoyance with GitLab CE is that it does not allow you to have a “Pull Mirror”. In case you do not know what that is, you are probably not in need to read this article; but if you want to learn, please continue anyway.
A git mirror is a secondary repository which is identical to another one. Usually mirrors are either hosted at different places, controlled by different people; or they are owned by different people or projects to maintain a copy. A Pull Mirror is a kind of git mirror which tries to keep itself up by “pulling” changes from another one.
In my case, most of my work is on GitHub which I am trying to slowly migrate to my own server (and maybe later a Raspberry Pi that I own). But till that migration is complete I wanted to make GitLab (which I host myself) host a “mirror” of some repos from GitHub. Now, the free version, GitLab CE does not allow you to do that out of box. There are 3 options now:
- Hack into the code of GitLab and build the feature yourself.
- Write a script to automate it yourself.
- Setup some third party tool.
Modifying GitLab is not feasible (or pragmatic)
The first option - to hack into GitLab’s source code is too much for the task that is needed to be accomplished. You would have to understand the codebase of a super complicated project to make one single change which you would have to keep doing with every new update that you install.
The second solution obviously looks much more practical.
The script
I did write the script. It is not very different from what I already have to sync my Obsidian vault. You pull from GitHub and push to GitLab. That should be it. Run it at any frequency you want and that should work!
There are two problems with it:
- If the script fails, it would probably be silent. So I won’t know when it stopped working. This has happened to me already.
- Scripts need to be registered with Cron scheduler. I don’t like making that file a mess. This might just be how I feel about cron but more than 4 or 5 entries, and it feels chaotic to me. Now, depending on how many repositories you want to “sync”, you would either have a big and complicated script or a crontab file that looks quite messy.
- It is not an elegant solution. There are many questions that arise:
- What if there is a git error?
- What if the script file somehow loses the “executable” bit?
- What if my git token expired?
- How do I know when the last successful attempt was? Do I go through a large log file to locate that failure?
- How do I find out which failure halted it?
Those might sound like inhibitions, or inconveniences. If you are brave to setup that script, go ahead, a simple enough will work most of the time and you should be fine. But I wanted something better.
The third party tool
When I was setting up GitLab, before using GitLab, I already had a Gitea setup running. Now, Gitea is fantastic. If you are not in posession of a machine where you can spare around 12 GB of RAM and at least 30 GB of empty space, GitLab is probably not for you and Gitea does most of the stuff that you would probably need from a self-hosted browser-based git hosting solution. So give Gitea (or Forgejo, its fully open-source fork) a try.
Now, gitea has an option on repository level which allows you to create a “migration” which can be marked as a “mirror”. When you do that, you are basically setting up a “pull mirror” in gitea. You can then go this mirrored repo and add a “push mirror” as well. The screenshot below should be helpful!
NOTE: You will have to provide authorization details. I created a token in GitHub for pulling the updates and a token in GitLab to push the updates into it.
So far, I have not tested (or tasted) a failure with this setup except that there is a slight delay in the interval - for example, if I setup the mirroring to happen in 11 minutes, it might take somewhere between 12 to 15 minutes for the mirroring to be done (for both push and pull). I guess, it is git taking its own time as I see it with GitLab’s paid versions as well.
So yes, that is how I have hacked together a solution to effectively create a “pull mirror” on GitLab where Gitea acts as a glue.
PS: I am interested in Forgejo and Gitea and might explore them later (or maybe not, who knows). So if you have any tips, let me know.
Cheers!