如何删除gitlab CI作业管道日志/构建和历史记录

时间：2018-11-17 21:05:36

标签： gitlab gitlab-ci gitlab-ci-runner

我们如何配置gitlab以仅保留最后10个CI作业/内部版本，并继续删除其余部分？

例如，在Jenkins中，我们可以将作业配置为仅保留最后的X版本。

10 个答案:

答案 0 :(得分：4)

针对懒惰的批量删除脚本，从最早的X管道中删除。

注意：需要jq。

#!/bin/bash
set -e

TOKEN=""
PROJECT=""
# How many to delete from the oldest.
PER_PAGE=100

for PIPELINE in $(curl --header "PRIVATE-TOKEN: $TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT/pipelines?per_page=$PER_PAGE&sort=asc" | jq '.[].id') ; do
    echo "Deleting pipeline $PIPELINE"
    curl --header "PRIVATE-TOKEN: $TOKEN" --request "DELETE" "https://gitlab.com/api/v4/projects/$PROJECT/pipelines/$PIPELINE"
done

答案 1 :(得分：2)

_gitlab_pipeline_cleanup() {
echo GITLAB PIPELINE CLEANUP TOOL
echo DELETING ALL EXCEPT RUNNING AND THE MOST RECENT PIPELINE
if [ -z $GITLABURL ]; then echo GITLAB_URL:; read GITLABURL;fi
if [ -z $GITLAB_TOKEN ]; then echo TOKEN:; read GITLAB_TOKEN;fi
if [ -z $PROJECT ]; then echo PROJECT_NUM:; read PROJECT;fi

list=$(curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "$GITLABURL/api/v4/projects/$PROJECT/pipelines?per_page=100" |jq -c '.[] | select( .status != "running" )| .id ' |tail -n+2 |grep -v ^$)

echo removing from $GITLABURL Project number $PROJECT
while echo $(echo -n "$list" |wc -c  ) |grep -v ^0$; do 

list=$(curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "$GITLABURL/api/v4/projects/$PROJECT/pipelines?per_page=100" |jq -c '.[] | select( .status != "running" )| .id ' |tail -n+2 |grep -v ^$)

for item in $list ;do 
echo -n "| -p $item |"
curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" --request "DELETE" "$GITLABURL/api/v4/projects/$PROJECT/pipelines/$item"
done 
done 
echo ; } ;

那你可以做

for PROJECT in 12345 23476234 2138734876238746 ;do _gitlab_pipeline_cleanup ;done

答案 2 :(得分：2)

根据之前的答案，修改脚本以检索多个项目，并为每个项目删除早于配置日期的管道。

#!/bin/bash
set -e

TOKEN=""
# How many to delete from the oldest.
PER_PAGE=100
UPDATED_BEFORE=2021-02-01T00:00:00Z
GITLAB_URL=


while : ; do
  COUNTER=0

  for PROJECT in $(curl -s --header "PRIVATE-TOKEN: $TOKEN" "$GITLAB_URL/api/v4/projects?per_page=$PER_PAGE" | jq '.[].id') ; do
    echo "Deleting in project $PROJECT"
    for PIPELINE in $(curl -s --header "PRIVATE-TOKEN: $TOKEN" "$GITLAB_URL/api/v4/projects/$PROJECT/pipelines?per_page=$PER_PAGE&sort=asc&updated_before=$UPDATED_BEFORE" | jq '.[].id') ; do
        echo "Deleting pipeline $PIPELINE"
        curl --header "PRIVATE-TOKEN: $TOKEN" --request "DELETE" "$GITLAB_URL/api/v4/projects/$PROJECT/pipelines/$PIPELINE"
        (( COUNTER++ )) || true
    done
  done

  echo $COUNTER

  if [[ "$COUNTER" -le 0 ]]; then
    break;
  fi
done

答案 3 :(得分：2)

要使用 python 删除项目中的所有管道，可以使用以下代码。您可以前往Jupyter在线试用

import requests

def confirmDeletion():
    confirmDelete = input('Do you want to delete all pipelines in the project Y/N ?')
    if confirmDelete == 'Y' or confirmDelete == 'y'  :
        return True
    else:
        return False

projectId = input('Provide the Gitlab Project ID')
token = input('Provide the Gitlab Access Token')
proceed = bool()
pipelinesQueryUrl = f'https://gitlab.com/api/v4/projects/{projectId}/pipelines'
if confirmDeletion():
    print('Deleting pipelines')
    # The Gitlab API does
    while True:
        response = requests.get(pipelinesQueryUrl, headers = {"PRIVATE-TOKEN": token})
        if response.ok:
            pipelines = response.json()
            if len(pipelines) == 0:
                print('No more pipelines found, exiting')
                break
            else:    
                for pipeline in pipelines:
                    pipelineId = pipeline.get('id')
                    url = f'https://gitlab.com/api/v4/projects/{projectId}/pipelines/{pipelineId}'
                    response = requests.delete(url, headers = {"PRIVATE-TOKEN": token})
                    if response.ok:
                        print(f'Pipeline {pipelineId} succesfully deleted')
                    else:
                        print (f'Deleting pipeline {pipelineId} on path failed: {response.url} : ({response.status_code}) {response.reason}')
                        if response.status_code == 429:
                            # Watch out for rate specific limits https://docs.gitlab.com/ee/user/gitlab_com/index.html#gitlabcom-specific-rate-limits
                            print ('Rate Limits have been reached, wait and try again later')
                            break
        else:
            print (f'Querying for pipelines failed: {response.url}: ({response.status_code}) {response.reason}')
            if response.status_code == 429:
                # Watch out for rate specific limits https://docs.gitlab.com/ee/user/gitlab_com/index.html#gitlabcom-specific-rate-limits
                print ('Rate Limits have been reached, wait and try again later')
            break
else:
    print('No pipelines deleted')

答案 4 :(得分：1)

我认为Gitlab不支持此功能。但是，您可以使用Gitlab API和webhooks自己创建此功能。

当您推动回购（并启动管道）时，它将触发可通过API =>读取您的CI历史记录的webhook。您可以删除所需的任何内容。

以下是pipeline events

的文档

以下是job API

的文档

仅供参考，我使用类似的解决方案。我已经为每个分支机构部署了服务器（每个分支机构都有MR）。 MR关闭时，将删除已部署的服务器。非常可靠。

答案 5 :(得分：1)

对于懒惰者，请在https://stackoverflow.com/a/55815040/1041691上展开

获取PROJECT和TOKEN并运行它，直到删除所有管道

for PIPELINE in $(curl --header "PRIVATE-TOKEN: $TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT/jobs?per_page=100" | jq '.[].pipeline.id') ; do
    echo "deleting $PIPELINE"
    curl --header "PRIVATE-TOKEN: $TOKEN" --request "DELETE" "https://gitlab.com/api/v4/projects/$PROJECT/pipelines/$PIPELINE"
done

答案 6 :(得分：1)

作为 Mog 答案的一个小扩展，因为我的 gitlab 一方面提供每页最多 100 个条目，另一方面我想根据管道的状态删除：

#!/bin/bash
set -e

TOKEN="---your token here--"
PROJECT="PROJECT-ID"
BASEURL="https://gitlab.com/api/v4/projects" # Base Url, if you host your 
# Status values: created, waiting_for_resource, preparing, pending, running, success, failed, canceled, skipped, manual, scheduled
STATUS=(failed canceled skipped) # separate status by space

# How many to delete from the oldest.
DELETE_CNT=500

# -----
CNT=0
declare -A STATUS_RESULT

for CUR_STATUS in "${STATUS[@]}";
do
  TEMP_CNT=$CNT
  DOLOOP=true
  while [ "$DOLOOP" = true ]
  do
    DOLOOP=false

    for PIPELINE in $(curl --header "PRIVATE-TOKEN: $TOKEN" "$BASEURL/$PROJECT/pipelines?per_page=$DELETE_CNT&sort=asc&status=$CUR_STATUS" | jq '.[].id') ; do
      if [[ "$CNT" -lt "$DELETE_CNT" ]]; then
        echo "#$CNT Deleting pipeline $PIPELINE with status $CUR_STATUS"
        curl --header "PRIVATE-TOKEN: $TOKEN" --request "DELETE" "$BASEURL/$PROJECT/pipelines/$PIPELINE"
        let "CNT+=1"
        DOLOOP=true
      fi
    done

    if [ "$DOLOOP" = true ] ; then
      if [[ "$CNT" -ge "$DELETE_CNT" ]]; then
       DOLOOP=false
      fi
    fi
  done

  TEMP_CNT=$((CNT-TEMP_CNT))
  STATUS_RESULT[$CUR_STATUS]=$TEMP_CNT
  echo "Removed $TEMP_CNT pipelines with status $CUR_STATUS"
done

echo "===================================================="
echo "= Summary of removed pipelines (max: $DELETE_CNT)"
echo "=   Total: $CNT"
echo "="
for key in "${!STATUS_RESULT[@]}"; do
    CNT=${STATUS_RESULT[$key]}
    echo "=   $key: $CNT"
done

答案 7 :(得分：1)

所有解决方案都有效，但请记住，您必须生成个人访问令牌而不是 API 令牌。如果您需要一个无处不在的令牌，您必须是管理员。

答案 8 :(得分：0)

从Gitlab Release 11.6开始，现在仅可以通过API删除管道了。

您需要：

  • API令牌
  • 项目的id
  • 您要删除的管道的pipeline_id。

在项目id: 1和pipeline_id: 4中使用文档中的curl的示例：

curl --header "PRIVATE-TOKEN: <your_access_token>" --request "DELETE" "https://gitlab.example.com/api/v4/projects/1/pipelines/46"

文档为here

答案 9 :(得分：0)

这会删除一个项目的所有管道，但我相信你可以找出 Perl 的东西来跳过前 10 个

curl -s --header "PRIVATE-TOKEN: ******" --request "GET" "https://gitlab.com/api/v4/projects/********/pipelines?per_page=32767" \
| perl -MJSON -le '$d = decode_json(<>); map { print $_->{"id"} } @$d' - | while read i; do
  curl -s --header "PRIVATE-TOKEN: ********" --request "DELETE" "https://gitlab.com/api/v4/projects/******/pipelines/$i"
done

项目 ID 写在项目名称（项目页面）下，您可以从“编辑配置文件”获得访问令牌。“访问令牌”并选中“API”复选框。

通过以下方式在 Linux 上的 Perl 中安装 JSON 模块：

sudo apt -y install libjson-perl
相关问题
最新问题