Jenkins on Windows 心得分享 (06):如何指定 Mirror 鏡像網站下載外掛 | The Will Will Web

The Will Will Web

記載著 Will 在網路世界的學習心得與技術分享

Jenkins on Windows 心得分享 (06):如何指定 Mirror 鏡像網站下載外掛

我們之前有個 Jenkins CI/CD 的導入顧問案,由於客戶是政府部門,客戶端的防火牆已經設定了無法連接到任何中國大陸的 IP 或 URL,但是 Jenkins 的更新伺服器預設會判定來自台灣的 IP 全部導向到清华大学Jenkins Mirror 伺服器,因此當你要安裝或更新 Plugins 外掛的時候,一定會連線失敗。也因為這樣,客戶那邊的 Jenkins Plugins 都將無法自動更新!不僅如此,安裝新的 Plugins 時通常也需要一併安裝相依外掛,因此各種困難接踵而來。這篇文章我將說明我們是如何解決這個難題的!

使用 Docker 安裝與啟動 Jenkins 最新版本

  1. 取得最新版 jenkins/jenkins:lts image

    docker pull jenkins/jenkins:lts
    
  2. 建立 jenkins-data volume 用來保存 Jenkins 所有設定

    docker volume create jenkins-data
    
  3. 啟動 Jenkins 容器

    docker run -d --name jenkins -p 8080:8080 -p 50000:50000 -v jenkins-data:/var/jenkins_home jenkins/jenkins:lts
    

    注意:Port 8080 是 Jenkins 預設 Web 介面。Port 50000 則是用來讓 Jenkins Agent 建立連線的 Port 埠號。

    注意:請勿加上 --rm 參數,因為我們在安裝 Plugins 的過程中需要重新啟動 Jenkins 容器。

  4. 取得預設管理者密碼(initialAdminPassword)

    docker exec -it jenkins cat /var/jenkins_home/secrets/initialAdminPassword | clip
    

    Windows 10 有內建一個 C:\Windows\System32\clip.exe 執行檔,可以將 Console 輸出直接放入「剪貼簿」中,對於複製 Jenkins 預設管理者密碼非常方便!👍

  5. 啟動 Jenkins 介面

    start http://localhost:8080/
    

    Unlock Jenkins

    因為步驟 4 已經複製了預設管理者密碼,所以直接 Ctrl-V 貼上即可登入成功!

模擬 Jenkins 無法連接中國大陸鏡像伺服器的狀況

因為 Jenkins 在下載任何 Plugins 時,都會連到 mirrors.tuna.tsinghua.edu.cn 域名,因此我打算將這個域名設定為無法連線,藉此模擬客戶端無法連線的狀況:

docker exec -it -u root jenkins bash

echo "127.0.0.1 mirrors.tuna.tsinghua.edu.cn" > /etc/hosts

首次啟動請跳過所有 Plugins 安裝的過程

  1. 在首次安裝的時候,你就可以選取 Select plugins to install 按鈕

    Customize Jenkins

  2. 請按下 None 取消所有建議外掛的安裝步驟

改用 jenkins-plugin-cli 安裝 Plugins 外掛

如果你是使用 Docker 執行 Jenkins 伺服器,可以直接使用內建的 /bin/jenkins-plugin-cli 來管理 Plugins 的安裝與執行。如果是其他的安裝方式,就要到 Plugin Installation Manager Tool for Jenkins 下載 jenkins-plugin-manager-*.jar 檔案 (jenkins-plugin-manager-2.9.2.jar) 來執行。

這裡最重要的步驟,就是設定一個 JENKINS_UC_DOWNLOAD 環境變數,並指向到一個可用的 Jenkins Mirror Sites (Jenkins 鏡像伺服器)。

目前官方維護的 Jenkins Mirror Sites 有分三個區域,截至今日為止可用的站台與網址如下:

  1. cn (中國大陸)
    1. http://mirrors.tuna.tsinghua.edu.cn/jenkins/
  2. jp (日本)
    1. http://ftp.yz.yamagata-u.ac.jp/pub/misc/jenkins/
    2. http://mirror.esuni.jp/jenkins/
  3. us (美國)
    1. ftp://mirror.xmission.com/jenkins/
    2. http://mirror.xmission.com/jenkins/
    3. http://archives.jenkins-ci.org/
    4. ftp://ftp-chi.osuosl.org/pub/jenkins/
    5. http://ftp-chi.osuosl.org/pub/jenkins/

要正確設定 JENKINS_UC_DOWNLOAD 環境變數的值,必須從 Jenkins Mirror Sites 複製正確的網址才行!

  1. 我們先進入 Jenkins 容器中

    docker exec -it jenkins bash
    
  2. 先執行以下命令安裝 Folders 外掛

    jenkins-plugin-cli --verbose -d $JENKINS_HOME/plugins/ --plugins cloudbees-folder
    

    注意:請務必加上 -d $JENKINS_HOME/plugins/ 參數,明確指定 Jenkins Plugins 安裝的資料夾!

    此時你應該可以看見 Unable to resolve plugin URL 的錯誤,也可以看到 Tried downloading cloudbees-folder from https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi 的錯誤訊息:

    No .txt or .yaml file containing list of plugins to be downloaded entered.
    Plugin download location: /var/jenkins_home/plugins
    Using update center https://updates.jenkins.io/update-center.json from JENKINS_UC environment variable
    Using experimental update center https://updates.jenkins.io/experimental/update-center.json from JENKINS_UC_EXPERIMENTAL environment variable
    Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
    No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
    No war entered. Will use default of /usr/share/jenkins/jenkins.war
    
    Retrieving update center information
    Returning cached value for: update-center-2.277.3
    Returning cached value for: experimental-update-center-2.277.3
    Returning cached value for: plugin-versions
    Couldn't find checksum for cloudbees-folder at version: latest
    
    cloudbees-folder depends on:
    credentials 2.3.18
    Setting checksum for: cloudbees-folder to vqUU9iK9k3WlVkbwGJ9sqBGVeQ+1kpm4dykagSSO07Q=
    Setting checksum for: cloudbees-folder to vqUU9iK9k3WlVkbwGJ9sqBGVeQ+1kpm4dykagSSO07Q=
    Will install new plugin cloudbees-folder 6.15
    Will use url: https://updates.jenkins.io/download/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi to download cloudbees-folder plugin
    Unable to resolve plugin URL https://updates.jenkins.io/download/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi, or download plugin cloudbees-folder to file
    Tried downloading cloudbees-folder from https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi
    Unable to resolve plugin URL https://updates.jenkins.io/download/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi, or download plugin cloudbees-folder to file
    Tried downloading cloudbees-folder from https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi
    Unable to resolve plugin URL https://updates.jenkins.io/download/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi, or download plugin cloudbees-folder to file
    Tried downloading cloudbees-folder from https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi
    Downloaded file is not a valid ZIP
    java.io.FileNotFoundException: /tmp/plugin-installation-manager-downloads2553358770239004420/cloudbees-folder.jpi (No such file or directory)
            at java.util.zip.ZipFile.open(Native Method)
            at java.util.zip.ZipFile.<init>(ZipFile.java:225)
            at java.util.zip.ZipFile.<init>(ZipFile.java:155)
            at java.util.jar.JarFile.<init>(JarFile.java:166)
            at java.util.jar.JarFile.<init>(JarFile.java:130)
            at io.jenkins.tools.pluginmanager.impl.PluginManager.downloadToFile(PluginManager.java:1276)
            at io.jenkins.tools.pluginmanager.impl.PluginManager.downloadToFile(PluginManager.java:1198)
            at io.jenkins.tools.pluginmanager.impl.PluginManager.downloadPlugin(PluginManager.java:1127)
            at io.jenkins.tools.pluginmanager.impl.PluginManager.lambda$downloadPlugins$4(PluginManager.java:537)
            at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
            at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
            at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
            at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
            at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
            at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
            at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
            at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
            at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
            at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
            at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
            at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
            at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
            at io.jenkins.tools.pluginmanager.impl.PluginManager.lambda$downloadPlugins$5(PluginManager.java:536)
            at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
            at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
            at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
            at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
            at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
    io.jenkins.tools.pluginmanager.impl.DownloadPluginException: Unable to download cloudbees-folder
            at io.jenkins.tools.pluginmanager.impl.PluginManager.lambda$downloadPlugins$4(PluginManager.java:542)
            at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
            at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
            at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
            at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
            at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
            at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
            at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
            at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
            at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
            at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
            at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
            at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
            at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
            at io.jenkins.tools.pluginmanager.impl.PluginManager.lambda$downloadPlugins$5(PluginManager.java:536)
            at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
            at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
            at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
            at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
            at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
    Unable to download cloudbees-folder
    

    此時,你只要先設定好 JENKINS_UC_DOWNLOAD 環境變數,就可以順利安裝 Plugins 外掛:

    export JENKINS_UC_DOWNLOAD=http://ftp.yz.yamagata-u.ac.jp/pub/misc/jenkins/
    jenkins-plugin-cli --verbose -d $JENKINS_HOME/plugins/ --plugins cloudbees-folder
    

    注意:請務必加上 -d $JENKINS_HOME/plugins/ 參數,明確指定 Jenkins Plugins 安裝的資料夾!

    以下是成功安裝外掛的訊息:

    No .txt or .yaml file containing list of plugins to be downloaded entered.
    Plugin download location: /var/jenkins_home/plugins
    Using update center https://updates.jenkins.io/update-center.json from JENKINS_UC environment variable
    Using experimental update center https://updates.jenkins.io/experimental/update-center.json from JENKINS_UC_EXPERIMENTAL environment variable
    Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
    No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
    No war entered. Will use default of /usr/share/jenkins/jenkins.war
    
    Retrieving update center information
    Returning cached value for: update-center-2.277.3
    Returning cached value for: experimental-update-center-2.277.3
    Returning cached value for: plugin-versions
    Couldn't find checksum for cloudbees-folder at version: latest
    
    cloudbees-folder depends on:
    credentials 2.3.18
    Setting checksum for: cloudbees-folder to vqUU9iK9k3WlVkbwGJ9sqBGVeQ+1kpm4dykagSSO07Q=
    Setting checksum for: cloudbees-folder to vqUU9iK9k3WlVkbwGJ9sqBGVeQ+1kpm4dykagSSO07Q=
    Will install new plugin cloudbees-folder 6.15
    Will use url: http://ftp.yz.yamagata-u.ac.jp/pub/misc/jenkins/plugins/cloudbees-folder/6.15/cloudbees-folder.hpi to download cloudbees-folder plugin
    Downloaded plugin cloudbees-folder
    Checksum valid for: cloudbees-folder
    Done
    

    冷知識:這個工具在 Oct 28, 2020 之前一直無法正常使用,直到 Remove '/download' subpath for JENKINS_UC_DOWNLOAD (#213) 這個 commit 才把問題給解決!

  3. 在確認可以成功安裝之後,就可以批次安裝所有建議的外掛

    以下是 Jenkins 預設建議安裝的外掛:

    • https://plugins.jenkins.io/cloudbees-folder
    • https://plugins.jenkins.io/antisamy-markup-formatter
    • https://plugins.jenkins.io/build-timeout
    • https://plugins.jenkins.io/credentials-binding
    • https://plugins.jenkins.io/timestamper
    • https://plugins.jenkins.io/ws-cleanup
    • https://plugins.jenkins.io/ant
    • https://plugins.jenkins.io/gradle
    • https://plugins.jenkins.io/workflow-aggregator
    • https://plugins.jenkins.io/github-branch-source
    • https://plugins.jenkins.io/pipeline-github-lib
    • https://plugins.jenkins.io/pipeline-stage-view
    • https://plugins.jenkins.io/git
    • https://plugins.jenkins.io/ssh-slaves
    • https://plugins.jenkins.io/matrix-auth
    • https://plugins.jenkins.io/pam-auth
    • https://plugins.jenkins.io/ldap
    • https://plugins.jenkins.io/email-ext
    • https://plugins.jenkins.io/mailer

    其實不只這些外掛會被安裝,而是包含「相依的外掛」也都會全部自動安裝完成!

    以下則是我經常安裝的外掛:

    • https://plugins.jenkins.io/locale
    • https://plugins.jenkins.io/msbuild

    我們只要用一個命令,就可以全部安裝完畢:

    export JENKINS_UC_DOWNLOAD=http://ftp.yz.yamagata-u.ac.jp/pub/misc/jenkins/
    
    jenkins-plugin-cli -d $JENKINS_HOME/plugins/ --plugins cloudbees-folder antisamy-markup-formatter build-timeout credentials-binding timestamper ws-cleanup ant gradle workflow-aggregator github-branch-source pipeline-github-lib pipeline-stage-view git ssh-slaves matrix-auth pam-auth ldap email-ext mailer locale msbuild
    

    jenkins-plugin-cli 安裝 Plugins 不但方便,而且安裝速度極快!

  4. 重新啟動 Jenkins 伺服器

    docker restart jenkins
    

    注意:所有透過 jenkins-plugin-cli 安裝的 Plugins 都需要重啟 Jenkins 伺服器才會生效!

預設的 Jenkins Plugins 路徑

上述技巧是透過直接將 Jenkins Plugins 安裝到指定的目錄下,在 Linux 環境下通常是 $JENKINS_HOME/plugins 目錄。

但是 Windows 環境預設則是依據安裝時設定的啟動身份而有不同的目錄:

  • 一般本機帳戶 (Local or Domain users)

    C:\Users\[Username]\AppData\Local\Jenkins\.jenkins\plugins
    

    注意:若要使用一般本機帳戶執行 Jenkins 服務,必須先將帳戶授予服務執行權限才行。詳細步驟請參見 Assigning a user account Logon as Service Rights 文章說明。

  • 本機系統帳戶 (LocalSystem)

    C:\Windows\System32\config\systemprofile\AppData\Local\Jenkins\.jenkins\plugins
    

相關連結