School of Engineering and Computer Science Te Kura Mātai Pūkaha, Pūrorohiko

Technical Note - GitLab: Using the ECS Git Repo

Summary

GitLab is ECS's local hosting system for git repositories. It is very similar to GitHub.
Any ECS student can create personal or group git repositories to share with other students. GitLab is reasonably complex but in most cases has full instructions in side it. If you don't already know git you will need to learn up on using git in order to use GitLab.

Details

Goto https://gitlab.ecs.vuw.ac.nz and login using your ECS username & password.

GitLab has full instructions inside it when you go to create a project but one of the first things you should do is add a SSH key.
It will prompt you to do this the first time you create a new project but if you are using a pre created project you will still need to do this.

In the top right hand corner is a little icon, click on that and choose settings. Click on SSH Keys in the top bar.

Now you need to add a SSH Key click on the "generate it" link. Read all the instructions and follow them.

Once you have added a SSH key, you can now use git to communicate with GitLab.
Remember if you want to access it from a home computer you will need to add another SSH key generated on that computer.

You can now create a project inside GitLab. It will give you instructions on how to create a git repository.

Sensible use of GitLab

Only add necessary files.
Avoid adding large files and if you have to, use LFS as stated in Binary Files section below.
Use .gitignore file to filter out unwanted files.
Don't add third-party libraries or data/log files generated by your program.

Binary Files & LFS

Files such as images/videos and any large binary files should use Git Large File System (LFS) to be pushed up to GitLab. This is much more efficient than committing directly into a repo, which wasn't really build for it. Please go to Git Large File Storage (LFS) to see how to set it up.

Repository Space Quota

Currently, there is a default Quota of 100Mb per GitLab project. If you have a need for more space and you are using GitLab sensibly E.g. As in sections above, using LFS, using .gitignore and so on, Email jobs@ecs.vuw.ac.nz asking for more space. Note: Please don't try deleting files, Git is designed to record history. It will not delete committed files. It is possible, but it is not easy, normally you have to use special tools and there is a chance of corrupting the repository.

Groups

If a group has been created for you, for a team project, create your projects in that group so every one in your team can work on the project. 
Important: Due to a bug recently introduced in GitLab, you may not be able to find your project
group via the web interface when the group is empty. There is no problem once a project is created.

To find it the first time, you must use the URL. This should be similar to below but confirm 
with your course lecture as the last part changes depending on if it's an 
assignment/ individual or group project.
   
https://gitlab.ecs.vuw.ac.nz/course-work/<coursecode>/<year>/<project1/(team or username)>
E.g.
https://gitlab.ecs.vuw.ac.nz/course-work/swen301/2022/assignment-1/myusername
https://gitlab.ecs.vuw.ac.nz/course-work/engr300/2022/project/project5

If someone in your group has already created a project you will have seen it on logging in.

Otherwise to create a new group project go to the menu in the upper left hand corner, and select Groups. GitLab will display any Groups you are in. Click on the group and from there you can create a new project in the group.

When creating a new project make sure you read all the instructions, GitLab shows.

GitLab pages

GibLab allows static websites to be hosted. The default domain for ECS is https:<user or group name>.glp.ecs.vuw.ac.nz/projectName

When reading Gitlabs pages documentation, they assume the general domain name is example.io just replace that domain with glp.ecs.vuw.ac.nz for ECS's gitlab.

Using Gitlab Container Registry

There are several limitations on our ECS machines due to security reasons, so read this carefully.

This gives you basic instructions on how to store a image in the Gitlab Container Registry from out side Gitlab and how to access it in the CI/CD environment using our shared Gitlab runners.

CI/CD shared runners

We have two shared runners, one for running shell scripts and the other docker containers. They have tags "shell" and "docker" this means you must add the tags parameter to the .gitlab-ci.yml file to run on one of them. 
tags:
- shell

Setting up GitLab Container Registry access

In GitLab, enable the Container Registry for your project in,
Project Settings -> General -> Visibility, project features, permissions

This uses docker login to access the container, thus you need to set up a GitLab personal access token

Go into "User Setting" -> "Access Tokens" and create a token with API and read_regisistry access. Make a safe copy of this token, creating a file that only you can read, with only the token in it.

Now we need to use docker login but first we need to check for an existing dockers configuration file.

$ ls ~/.docker/config.json

If it exists you will want to rename it out of the way, so docker creates a new one.

Login into docker container registry using the token. The command below assumes the token was stored in a file called myToken

docker login -u <username> gitlab.ecs.vuw.ac.nz:45678 --password-stdin < myToken

Now we need to create a Gitlab CI/CD variables.

In your Gitlab project, Settings -> "CI/CD" -> Variables Create a variable DOCKER_AUTH_CONFIG with the contents of ~/.docker/config.json file

It should look something like this 
cat ~/.docker/config.json
{
        "auths": {
                "gitlab.ecs.vuw.ac.nz:45678": {
                        "auth": "cmUIO2U6cHNFRVRBV2fdfdfdRRNFGHYU="
                }
        },
        "HttpHeaders": {
                "User-Agent": "Docker-Client/18.09.6-ce (linux)"
        }
}

You can now build and push a docker image to the container registry using your gitlab project

Read Gitlabs container registry docs.
Also if you don't know how to push a image read Docker command line push

The gitlab container registry docs, state the naming schema is 
<registry URL>/<namespace>/<project>/<image>
The <registry URL> for us is 'gitlab.ecs.vuw.ac.nz:45678'
<namespace> would be your 'username' or any 'group path'

Thus if your username was 'johnking' and you had a project call 'test-registry' you would tag your docker image like this

gitlab.ecs.vuw.ac.nz:45678/johnking/test-registry/imagename

You can now build and push this image to GitLab's container registry

docker build -t gitlab.ecs.vuw.ac.nz:45678/johnking/test-registry/imagename .
docker push gitlab.ecs.vuw.ac.nz:45678/johnking/test-registry/imagename

Assuming you have done all the steps above correctly, you now should be able to add a .gitlab-ci.yml file that allows you to access your image in our shared runners.

Eg.

$ cat .gitlab-ci.yml 
image: gitlab.ecs.vuw.ac.nz:45678/johnking/test-registry/imagename

job:
    script: 
        - some command
        
    tags:
      - docker

Building a Docker image in CI/CD

This gives you a basic idea of how to go about building a docker image and pushing it back to GitLab's Container Registry from with in our shared shell GitLab runner. This doesn't cover how to actually build a docker image, read the docker doc's for that.

You need to read all "Setting up GitLab Container Registry access" above and do all the steps up to and including creating the DOCKER_AUTH_CONFIG variable.

Create a second variable DOCKER_TOKEN and store your personal token in it.

Now you can create a .gitlab-ci.yml that allows you to login to GitLab's Container Registry
docker login will create a config file when the script runs, which is deleted when the job finishes so don't worry about all the security risk messages that come out when it runs. 

variables:
    DOCKER_CONFIG: $CI_PROJECT_DIR
    DOCKER_IMAGE:  gitlab.ecs.vuw.ac.nz:45678/<user/groups>/<project>/<image>
    
job:

    before_script:
        - docker login -u $CI_REGISTRY_USER -p $DOCKER_TOKEN $DOCKER_IMAGE        

    script:
        # your docker commands to build and push a image
        - docker build -t $DOCKER_IMAGE .
        - docker push $DOCKER_IMAGE 
        
    tags:
      - shell
Print version | Backlinks |