The project is intended to create a multi-user integration of SonarQube and GitLab. Project visibility and user permissions will be set in SonarQube in the same way as project permissions in GitLab.
This document is available in languages: eng ๐ฌ๐ง, ua ๐บ๐ฆ, rus ๐ท๐บ
This utility consists of a front-end API that accepts requests to update
permissions from the GitLab CI pipeline, where all trust is built around
CI_JOB_TOKEN
, trusted tasks are added to the RQ job queue (Redis Queue).
Jobs are processed by the backend workers, and a separate prometheus metrics
exporter is used to display statistics.
Permissions are divided by the access level of SonarQube roles according to
the interests of group users in GitLab.
It takes into account how GitLab implements user access rights in the
project, and users from invited groups, where access levels are limited
in accordance with GitLab.
SONARQUBE_ALM_KEY
;SONARQUBE_TOKEN
for the worker to work correctly;SONARQUBE_TOKEN
used in CI pipeline to
perform analysis and project management.GITLAB_TOKEN
for worker, needed to get a
list of all members of a group or project, including inherited and invited
members$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
and the project key is recommended to
be set to gitlab:$CI_PROJECT_ID
. Only with these settings, synchronization
will be successful, other states are possible but have not been tested.The project is implemented on flask, for WSGI it is used bjoern, the work with the GitLab API is done through python-gitlab, and the SonarQube API through python-sonarqube-api. To process the message queue, RQ is used, whose metrics are returned by rq-exporter.
You can pull image from registries:
ghcr.io/woozymasta/guassp:latest
quay.io/woozymasta/guassp:latest
docker.io/woozymasta/guassp:latest
For a quick start, you can use the example from docker-compose:
docker-compose up -d
To launch the desired utility, transfer to the container or script
guassp.sh
argument:
worker
(default) - task handler from the queueapi
- API to control tasksexporter
- Prometheus metricall-in-one
- Launch worker
,api
and exporter
immediatelyapi-dev
- Launching API via Flask DEV serverLISTEN_ADDRESS
=0.0.0.0
- API listen addressLISTEN_PORT
=5000
- API listen portLOG_LEVEL
=INFO
- Logging levelSECRET_KEY
=secret
- Secret keyQUEUE_RESULT_TTL
=7200
- Queue result storage timeMORE_ACCURATE_SYNC
=true
- Search users in SonarQube by GitLab
email else by username. It call one more API request per user is
more accurate but slower x2GITLAB_URL
=https://gitlab.com
- GitLab server URLGITLAB_TOKEN
- GitLab access token, must have permissions to view
contributors and their permissions for maintained projectsGITLAB_SKIP_USERS
- comma-separated list of GitLab user IDs that will
be skipped during synchronizationSONARQUBE_URL
- SonarQube server URLSONARQUBE_TOKEN
- access token to SonarQube with
administrative privilegesSONARQUBE_ALM_KEY
- [ALM][] key GitLab integration nameSONARQUBE_SKIP_GROUPS
- comma-separated list of SonarQube groups that
will be skipped during synchronizationREDIS_URL
=redis://localhost:6379/0
- Redis server URLEXPORTER_LISTEN_ADDRESS
=0.0.0.0
- Exporter listen addressEXPORTER_LISTEN_PORT
=9726
- Exporter listen portAlso you can pass RQ args and environment variables
POST
/task
{"job_token": str}
Headers:
JOB-TOKEN
orAuthorization: Bearer
You need to pass the task token in any of the options:
curl -sL http://127.0.0.1:5000/task \
-H "Content-Type: application/json" \
-d '{"job_token": "'$CI_JOB_TOKEN'"}' | jq
curl -sL http://127.0.0.1:5000/task -X POST \
-H "JOB-TOKEN: $CI_JOB_TOKEN" | jq
curl -sL http://127.0.0.1:5000/task -X POST \
-H "Authorization: Bearer $CI_JOB_TOKEN" | jq
The task token can be passed in the JOB-TOKEN
header, or
Authorization: Bearer
or be the value of the job_token
key in JSON
GET
/tasks
curl -sL http://127.0.0.1:5000/tasks | jq
curl -sL http://127.0.0.1:5000/tasks | jq -er '.tasks | keys'
GET
/task/<uuid:job_uuid>
curl -sL http://127.0.0.1:5000/task/8b155172-cfcf-4777-b9f4-bfce53b6eb0e | jq
DELETE
/task/<uuid:job_uuid>
curl -sL http://127.0.0.1:5000/task/8b155172-cfcf-4777-b9f4-bfce53b6eb0e \
-X DELETE | jq
POST
/task_manual/<int:prj_id>
curl -sL http://127.0.0.1:5000/task_manual/111 \
-X POST | jq
GET
/health
API health, application availability and job queues.
In the GitLab CI pipeline, first of all, you must make sure that the [ALM][] setup is done and refers to your project, after which you can submit a task for synchronization in guassp. Now you can start the analysis.
For publish Guassp API behind Nginx as part of SonarQube API check Nginx config example
: "${SONARQUBE_PROJECT_KEY:=gitlab:$CI_PROJECT_ID}"
curl --location --fail --user "$SONARQUBE_TOKEN:" \
"$SONARQUBE_URL/api/alm_settings/set_gitlab_binding" \
-d "almSetting=$SONARQUBE_ALM_NAME" \
-d "project=$SONARQUBE_PROJECT_KEY" \
-d "repository=$CI_PROJECT_ID"
curl --location --fail -X POST -H "JOB-TOKEN: $CI_JOB_TOKEN" \
"$SONARQUBE_URL/api/guassp/task"
You can see a more voluminous example of a script for executing SonarQube
in the pipeline in the sq-integration-taks.sh
file
Metrics implemented using project rq-exporter
Dashboard ID 12196
is suitable for visualization in Grafana or use it
adaptation which will output only metrics from guassp.
A set of commands for fast local debugging in container
# Build
podman build -t guassp .
# Redis
podman run --rm -d -p 6379:6379 --name redis redis
# API
podman run --rm -d -p 5000:5000 --env-file .env --name guassp-api localhost/guassp:latest api
# Workers
podman run --rm -d --env-file .env --name guassp-worker-1 localhost/guassp:latest worker
podman run --rm -d --env-file .env --name guassp-worker-2 localhost/guassp:latest worker
# Exporter
podman run --rm -d -p 9726:9726 --env-file .env --name guassp-exporter localhost/guassp:latest exporter
# Check
curl 0.0.0.0:9726 -s | grep -v '^#'
curl 0.0.0.0:5000/tasks -s | jq
curl 0.0.0.0:5000/task -s -X POST -H "JOB-TOKEN: $CI_JOB_TOKEN" | jq
Or run locally, for this you need to install dependencies
apt-get install -y libev-dev libevdev2
python -m venv .venv
./.venv/bin/activate
pip install requirements.txt
And for simplicity, run through a script guassp
./guassp.sh api
./guassp.sh worker
./guassp.sh exporter