Generation of text from Merge-Requests automatically in LLM

Note: This is an example, the implementation of the application may not be completely accurate. The purpose is to show you the general approach and the code may need to be adjusted to your specific needs.

Foreword:

The following guide aims to provide a comprehensive understanding of the process and code for generating a descriptive text from a Merge-Request. The objective is to automate this task, relieving developers from the need to manually write lengthy descriptions.

Prerequisites:

Before embarking on this endeavor, ensure you have the following prerequisites:

Context:

In the world of software development, merge-requests are crucial for integrating code changes into a main branch. However, composing comprehensive descriptions for each merge-request can be time-consuming and sometimes tedious.

Objective:

The objective is to create a Python script that automatically generates a descriptive text for each merge-request based on the changes included. This text can include:

Implementation:

The script utilizes the GitLab API to retrieve information about the merge-request and its associated commits. This information is then processed and formatted into a comprehensive description.

The Python code is presented below, along with explanations of each section.

import base64
import vertexai
from vertexai.generative_models import GenerativeModel, Part, FinishReason

from vertexai.preview.generative_models import generative_models as generative_models

import gitlab
import os, json
import argparse;

GCP_PROJECT_ID = os.getenv("GCP_PROJECT_ID", "sandbox-mgoulin")
GCP_LOCATION = os.getenv("GCP_LOCATION", "us-central1")
GCP_MODEL = os.getenv("GCP_MODEL", "gemin-1.5-flash-preview-0514")

GITLAB_HOST = os.getenv("GITLAB_HOST", "https://gitlab.com")
GITLAB_TOKEN = os.getenv("GITLAB_TOKEN", False)
GITLAB_SSL_VERIFY = os.getenv("GITLAB_SSL_VERIFY", False)

LLM_PROMPT = """As a senior Devops engineer reviewing a merge request for a Terraform module. Please provide a comprehensive description of the changes in french, focusing on the following:
* **What is the overall purpose of the changes?**
* **What specific resources are affected?**
* **What are the main changes made to those resources?**
* **Are there any security or performance implications of the changes?**
* **Are there any potential risks or considerations?**
"""

def generate(diff):
    vertexai.init(project=GCP_PROJECT_ID, location=GCP_LOCATION)
    model = GenerativeModel(
        GCP_MODEL,
    )
    responses = model.generate_content(
        [LLM_PROMPT, diff],
        generation_config=generation_config,
        safety_settings=safety_settings,
        stream=True,
    )

    resp = ""
    for response in responses:
        resp += response.text
    return resp

generation_config = {
    "max_output_tokens": 8192,
    "temperature": 1,
    "top_p": 0.95,
}

safety_settings = {
    generative_models.HarmCategory.HARM_CATEGORY_HATE_SPEECH: generative_models.HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    generative_models.HarmCategory.HARM_CATEGORY_DANGER_OR_VIOLENCE: generative_models.HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    generative_models.HarmCategory.HARM_CATEGORY_SEXUAL_CONTENT: generative_models.HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    generative_models.HarmCategory.HARM_CATEGORY_HARASSMENT: generative_models.HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
}

def find_mr(project, mr_id):
    gl = gitlab.Gitlab(url=GITLAB_HOST, private_token=GITLAB_TOKEN, ssl_verify=GITLAB_SSL_VERIFY)
    proj = gl.projects.get(project)
    return proj.mergerequests.get(mr_id)

parser = argparse.ArgumentParser(
    prog='generate-mr-text.py',
    description='Use Gemini to update a merge request with a description',
)

parser.add_argument('-p', '--gitlab_project', required=True)
parser.add_argument('-m', '--gitlab_mr_id', required=True, type=int)

project = parser.parse_args().gitlab_project
mr_id = parser.parse_args().gitlab_mr_id
mr = find_mr(project, mr_id)
changes = mr.changes()
#print(json.dumps(changes, indent=4))
#diff = find_diff()
text = generate(json.dumps(changes, indent=4))
mr.description = mr.description.split("=== START BLOCK GENERATED BY generate-mr-text.py ===") [0] + "=== START BLOCK GENERATED BY generate-mr-text.py ===\n" + text + "\n=== END BLOCK GENERATED BY generate-mr-text.py ==="
mr.save()

In this script:

  1. Import necessary modules
  2. Get Environment variables: this step retrieves the necessary information for connecting to GitLab and Vertex AI
  3. Define the prompt: This sets up the instructions for the language model.
  4. Define the generate function: This function uses the Vertex AI client library to connect to the Gemini model and generate text based on the provided input.
  5. Define the find_mr function: This function retrieves the Merge-Request from GitLab using the specified project and merge-request ID.
  6. Define the argument parser: This section creates a user-friendly interface for passing project and merge-request ID as arguments.
  7. Retrieve the Merge-Request: The script uses the argument parser to get the project and merge-request ID and retrieves the relevant Merge-Request from GitLab.
  8. Generate the description: The changes are fetched, processed, and used to generate a description.
  9. Update the Merge-Request: The generated description is then added to the merge-request description field in GitLab.

How to use the script:

  1. Set up the environment variables: Replace the placeholders in the script with your actual values.
  2. Install the required libraries: Ensure that you have installed all the necessary libraries using pip install -r requirements.txt.
  3. Run the script: Execute the script using a command similar to python generate-mr-text.py -p <gitlab_project> -m <gitlab_mr_id>.

Next Steps:

This is just a basic example of what can be achieved with this approach. There are many other ways to enhance this script, such as:

Conclusion:

By leveraging the power of the Vertex AI Gemini model, this script simplifies the process of generating descriptive text for merge-requests. This automation streamlines the development workflow and ensures that all merge-requests are well-documented.

Add application sources with correct permissions for OpenShift

USER 0 ADD . . RUN chown -R 1001:0 ./* USER 1001

Install the dependencies

RUN pip install -r requirements.txt && \ pip install -r requirements-dev.txt

Run the dockerfile

RUN pip install –upgrade pip && \ pip install -r requirements.txt && \ pip install -r requirements-dev.txt

Build the dockerfile

RUN pip install –upgrade pip && \ pip install -r requirements.txt && \ pip install -r requirements-dev.txt

**The command `docker build -t generate-mr-text .`  will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:**

```bash
docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.

The command docker build -t generate-mr-text . will generate the docker image. Then, you can run the docker image and specify the project and the merge request id as follows:

docker run -it  generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> 

**The command docker run -it generate-mr-text -p <gitlab_project> -m <gitlab_mr_id> will run the container and generate a description for the specified merge-request. **

Note: You need to replace <gitlab_project> and <gitlab_mr_id> with your project and merge-request id.

The docker image will be built with the following steps:

  1. ADD . .: Copy all files from the current directory to the Docker image.
  2. RUN chown -R 1001:0 ./*: Change the ownership of all files in the image to user 1001.
  3. USER 1001: Switch to user 1001.
  4. RUN pip install -r requirements.txt && pip install -r requirements-dev.txt: Install the Python dependencies.
  5. RUN pip install –upgrade pip && pip install -r requirements.txt && pip install -r requirements-dev.txt: Update pip and install the Python dependencies.

The docker image will be built and tagged with the name generate-mr-text. You can then run the container using the docker run command.