Auto-merging via docker-versions-create

Co-authored-by: Tane Barriball <tanebarriball@gmail.com>

.github/workflows/main.yml

@@ -1,6 +1,4 @@
-# This is a basic workflow to help you get started with Actions
-
-name: Build non-multiarch branches/tags
+name: Build and Publish
 
 on:
   push:
@@ -16,8 +14,6 @@ on:
       - "[0-9]+.[0-9]+.[0-9]+-openj9-nightly"
       - "[0-9]+.[0-9]+.[0-9]+-adopt11"
       - "[0-9]+.[0-9]+.[0-9]+-adopt13"
-  pull_request:
-    branches: [ master ]
 
 jobs:
   build:
@@ -33,7 +29,7 @@ jobs:
         username: ${{ secrets.DOCKER_USER }}
         password: ${{ secrets.DOCKER_PASSWORD }}
         tag_with_ref: true
-        tag_with_sha: true
+        tag_with_sha: false
         cache_froms: itzg/minecraft-server:latest
         add_git_labels: true
-        push: false
+        push: true

.github/workflows/pr.yml

@@ -0,0 +1,19 @@
+name: Validate PR
+
+on:
+  pull_request:
+    branches: [ master ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+
+      - name: Build Docker image
+        uses: docker/build-push-action@v1.1.0
+        with:
+          tag_with_sha: true
+          cache_froms: itzg/minecraft-server:latest
+          push: false

DEVELOPMENT.md

@@ -1,4 +1,17 @@
-Individual scripts can be iteratively developed and tested using the following procedure.
+## Adding a server type
+
+Adding a new server `TYPE` can vary due to the complexity of obtaining and configuring each type; however, the addition of any server type includes at least the following steps:
+
+1. Copy an existing "start-deploy*" script, such as [start-deployMohist](start-deployMohist) and rename it accordingly making sure to retain the "start-deploy" prefix
+2. Modify the type-specific behavior between the "start-utils" preamble and the hand-off to `start-finalSetup01World` at the end of the script 
+3. Develop and test the changes using the [iterative process described below](#iterative-script-development)
+4. Add a case-entry to the `case "${TYPE^^}"` in [start-configuration](start-configuration)
+5. Add a section to the [README](README.md). It is recommended to copy-modify an existing section to retain a similar wording and level of detail
+6. [Submit a pull request](https://github.com/itzg/docker-minecraft-server/pulls)
+
+## Iterative script development
+
+Individual scripts can be iteratively developed, debugged, and tested using the following procedure.
 
 First, build a baseline of the image to include the packages needed by existing or new scripts:
 
@@ -9,7 +22,7 @@ docker build -t mc-dev .
 Using the baseline image, an interactive container can be started to iteratively run the scripts to be developed. By attaching the current workspace directory, you can use the local editor of your choice to iteratively modify scripts while using the container to run them.
 
 ```shell script
-docker run -it --rm -v ${PWD}:/scripts -e SCRIPTS=/scripts/ --entrypoint bash mc-dev 
+docker run -it --rm -v ${PWD}:/scripts -e SCRIPTS=/scripts/ --entrypoint bash mc-dev
 ```
 
 From within the container you can run individual scripts via the attached `/scripts/` path; however, be sure to set any environment variables expected by the scripts by either `export`ing them manually:
@@ -26,3 +39,36 @@ VANILLA_VERSION=1.12.2 /scripts/start-magma
 ```
 
 > NOTE: You may want to temporarily add an `exit` statement near the end of your script to isolate execution to just the script you're developing.
+
+## Multi-base-image variants
+
+Several base-image variants are maintained in order to offer choices in JDK provider and version. The variants are maintained in their respective branches:
+- openj9 
+- openj9-nightly
+- adopt11
+- adopt13
+- multiarch
+
+The [docker-versions-create.sh](docker-versions-create.sh) script is configured with the branches to maintain and is used to merge changes from the master branch into the mulit-base variant branches. The script also manages git tagging the master branch along with the merged branches. So a typical use of the script would be like:
+
+```shell script
+./docker-versions-create.sh -s -t 1.2.0
+```
+
+> Most often the major version will be bumped unless a bug or hotfix needs to be published in which case the patch version should be incremented.
+
+> The build and publishing of those branches and their tags is currently performed within Docker Hub.
+
+## multiarch support
+
+The [multiarch branch](https://github.com/itzg/docker-minecraft-server/tree/multiarch) supports running the image on amd64, arm64, and armv7 (aka RaspberryPi). Unlike the mainline branches, it is based on Ubuntu 18.04 since the openjdk package provided by Ubuntu includes full JIT support on all of the processor types.
+
+The multiarch images are built and published by [a Github action](https://github.com/itzg/docker-minecraft-server/actions?query=workflow%3A%22Build+and+publish+multiarch%22), which [is configured in that branch](https://github.com/itzg/docker-minecraft-server/blob/multiarch/.github/workflows/build-multiarch.yml).
+
+## Generating release notes
+
+The following git command can be used to provide the bulk of release notes content:
+
+```shell script
+git log --invert-grep --grep "^ci:" --grep "^misc:" --pretty="- %s" 1.1.0..1.2.0
+```

README.md

@@ -2,16 +2,13 @@
 [![Docker Stars](https://img.shields.io/docker/stars/itzg/minecraft-server.svg?maxAge=2592000)](https://hub.docker.com/r/itzg/minecraft-server/)
 [![GitHub Issues](https://img.shields.io/github/issues-raw/itzg/docker-minecraft-server.svg)](https://github.com/itzg/docker-minecraft-server/issues)
 [![Discord](https://img.shields.io/discord/660567679458869252)](https://discord.gg/DXfKpjB)
+[![Build and Publish](https://github.com/itzg/docker-minecraft-server/workflows/Build%20and%20Publish/badge.svg)](https://github.com/itzg/docker-minecraft-server/actions)
 [![](https://img.shields.io/badge/Donate-Buy%20me%20a%20coffee-orange.svg)](https://www.buymeacoffee.com/itzg)
 
 This docker image provides a Minecraft Server that will automatically download the latest stable
 version at startup. You can also run/upgrade to any specific version or the
 latest snapshot. See the _Versions_ section below for more information.
 
-[![Click for more docs](https://i.imgur.com/jS02ebD.png)](https://github.com/itzg/docker-minecraft-server/blob/master/README.md)
-
-[Full docs available in Github](https://github.com/itzg/docker-minecraft-server/blob/master/README.md)
-
 To simply use the latest stable version, run
 
     docker run -d -p 25565:25565 --name mc itzg/minecraft-server
@@ -179,11 +176,13 @@ You can also query the container's health in a script friendly way:
 healthy
 ```
 
-## Autopause
+## Autopause (experimental)
 
 ### Description
 
-> There are various bug reports on [Mojang](https://bugs.mojang.com) about high CPU usage of servers with newer versions, even with few or no clients connected (e.g. [this one](https://bugs.mojang.com/browse/MC-149018), in fact the functionality is based on [this comment in the thread](https://bugs.mojang.com/browse/MC-149018?focusedCommentId=593606&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-593606)).
+> EXPERIMENTAL: this feature only works with default bridge networking using official Docker distributions. Host networking and container management software, such as Portainer, and NAS solutions do not seem to provide compatible networking.
+
+There are various bug reports on [Mojang](https://bugs.mojang.com) about high CPU usage of servers with newer versions, even with few or no clients connected (e.g. [this one](https://bugs.mojang.com/browse/MC-149018), in fact the functionality is based on [this comment in the thread](https://bugs.mojang.com/browse/MC-149018?focusedCommentId=593606&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-593606)).
 
 An autopause functionality has been added to this image to monitor whether clients are connected to the server. If for a specified time no client is connected, the Java process is stopped. When knocking on the server port (e.g. by the ingame Multiplayer server overview), the process is resumed. The experience for the client does not change.
 
@@ -193,7 +192,7 @@ From the server's point of view, the pausing causes a single tick to take as lon
 
 On startup the `server.properties` file is checked and, if applicable, a warning is printed to the terminal. When the server is created (no data available in the persistent directory), the properties file is created with the Watchdog disabled.
 
-The autopause functionality is not compatible with docker's host network_mode, as the `knockd` utility cannot properly listen for connections in that mode.
+A starting, example compose file has been provided in [examples/docker-compose-autopause.yml](examples/docker-compose-autopause.yml).
 
 ### Enabling Autopause
 
@@ -204,13 +203,13 @@ Enable the Autopause functionality by setting:
 ```
 
 There are 4 more environment variables that define the behaviour:
-* `AUTOPAUSE_TIMEOUT_EST`, default `3600` (seconds)  
+* `AUTOPAUSE_TIMEOUT_EST`, default `3600` (seconds)
 describes the time between the last client disconnect and the pausing of the process (read as timeout established)
-* `AUTOPAUSE_TIMEOUT_INIT`, default `600` (seconds)  
+* `AUTOPAUSE_TIMEOUT_INIT`, default `600` (seconds)
 describes the time between server start and the pausing of the process, when no client connects inbetween (read as timeout initialized)
-* `AUTOPAUSE_TIMEOUT_KN`, default `120` (seconds)  
+* `AUTOPAUSE_TIMEOUT_KN`, default `120` (seconds)
 describes the time between knocking of the port (e.g. by the main menu ping) and the pausing of the process, when no client connects inbetween (read as timeout knocked)
-* `AUTOPAUSE_PERIOD`, default `10` (seconds)  
+* `AUTOPAUSE_PERIOD`, default `10` (seconds)
 describes period of the daemonized state machine, that handles the pausing of the process (resuming is done independently)
 
 ## Deployment Templates and Examples
@@ -397,13 +396,6 @@ If you are hosting your own copy of Bukkit/Spigot you can override the download
 
 You can build spigot from source by adding `-e BUILD_FROM_SOURCE=true`
 
-**NOTE: to avoid pegging the CPU when running Spigot,** you will need to
-pass `--noconsole` at the very end of the command line and not use `-it`. For example,
-
-    docker run -d -v /path/on/host:/data \
-        -e TYPE=SPIGOT \
-        -p 25565:25565 -e EULA=TRUE --name mc itzg/minecraft-server --noconsole
-
 You can install Bukkit plugins in two ways...
 
 ### Using the /data volume
@@ -455,13 +447,6 @@ but you can also choose to run a specific build with `-e PAPERBUILD=205`.
         -e TYPE=PAPER \
         -p 25565:25565 -e EULA=TRUE --name mc itzg/minecraft-server
 
-**NOTE: to avoid pegging the CPU when running PaperSpigot,** you will need to
-pass `--noconsole` at the very end of the command line and not use `-it`. For example,
-
-    docker run -d -v /path/on/host:/data \
-        -e TYPE=PAPER \
-        -p 25565:25565 -e EULA=TRUE --name mc itzg/minecraft-server --noconsole
-
 If you are hosting your own copy of PaperSpigot you can override the download URL with:
 
 - -e PAPER_DOWNLOAD_URL=<url>
@@ -534,6 +519,15 @@ A [Mohist](https://github.com/Mohist-Community/Mohist) server can be used with
 
 > **NOTE** there are limited base versions supported, so you will also need to  set `VERSION`, such as "1.12.2"
 
+
+## Running a Catserver type server
+
+A [Catserver](http://catserver.moe/) type server can be used with
+
+    -e TYPE=CATSERVER
+
+> **NOTE** Catserver only provides a single release stream, so `VERSION` is ignored
+
 ## Running a server with a Feed the Beast modpack
 
 > **NOTE** requires `itzg/minecraft-server:multiarch` image
@@ -753,9 +747,13 @@ The server name (e.g. for bungeecord) can be set like:
 
 ### Server port
 
-The server port can be set like:
+> **WARNING:** only change this value if you know what you're doing. It is only needed when using host networking and it is rare that host networking should be used. Use `-p` port mappings instead.
 
-    docker run -d -e SERVER_PORT=25565 ...
+If you must, the server port can be set like:
+
+    docker run -d -e SERVER_PORT=25566 ...
+
+**however**, be sure to change your port mapping accordingly and be prepared for some features to break.
 
 ### Difficulty
 
@@ -1058,9 +1056,21 @@ Allows users to use flight on your server while in Survival mode, if they have a
 
 ### Other server property mappings
 
-Environment Variable | Server Property
----------------------|-----------------
-PLAYER_IDLE_TIMEOUT  | player-idle-timeout
+| Environment Variable              | Server Property                   |
+| --------------------------------- | --------------------------------- |
+| PLAYER_IDLE_TIMEOUT               | player-idle-timeout               |
+| BROADCAST_CONSOLE_TO_OPS          | broadcast-console-to-ops          |
+| BROADCAST_RCON_TO_OPS             | broadcast-rcon-to-ops             |
+| ENABLE_JMX                        | enable-jmx-monitoring             |
+| SYNC_CHUNK_WRITES                 | sync-chunk-writes                 |
+| ENABLE_STATUS                     | enable-status                     |
+| ENTITY_BROADCAST_RANGE_PERCENTAGE | entity-broadcast-range-percentage |
+| FUNCTION_PERMISSION_LEVEL         | function-permission-level         |
+| NETWORK_COMPRESSION_THRESHOLD     | network-compression-threshold     |
+| OP_PERMISSION_LEVEL               | op-permission-level               |
+| PREVENT_PROXY_CONNECTIONS         | prevent-proxy-connections         |
+| USE_NATIVE_TRANSPORT              | use-native-transport              |
+| ENFORCE_WHITELIST                 | enforce-whitelist                 |
 
 ## Miscellaneous Options
 
@@ -1127,8 +1137,7 @@ a companion squid proxy by setting the equivalent of
 
 ### Using "noconsole" option
 
-Some older versions of Spigot required `--noconsole` to be passed when detaching stdin. You can
-pass that at the end of `docker run` after the image name or set `-e CONSOLE=FALSE`.
+Some older versions (pre-1.14) of Spigot required `--noconsole` to be passed when detaching stdin, which can be done by setting `-e CONSOLE=FALSE`.
 
 ### Explicitly disable GUI
 
@@ -1139,6 +1148,6 @@ disable that by passing `-e GUI=FALSE`.
 
 To run this image on a RaspberryPi 3 B+, 4, or newer, use the image tag
 
-    itzg/minecraft-server:armv7
+    itzg/minecraft-server:multiarch
 
 > NOTE: you may need to lower the memory allocation, such as `-e MEMORY=750m`

docker-compose.yml

@@ -10,19 +10,7 @@ services:
       - "mc:/data"
     environment:
       EULA: "TRUE"
-      CONSOLE: "false"
-      ENABLE_RCON: "true"
-      RCON_PASSWORD: "testing"
-      RCON_PORT: 28016
     restart: always
-  rcon:
-    image: itzg/rcon
-    ports:
-      - "4326:4326"
-      - "4327:4327"
-    volumes:
-      - "rcon:/opt/rcon-web-admin/db"
 
 volumes:
-  mc:
-  rcon:
+  mc: {}

docker-versions-create.sh

@@ -10,7 +10,7 @@ function TrapExit {
 
 batchMode=false
 
-while getopts "hbt:" arg
+while getopts "hbt:s" arg
 do
   case $arg in
     b)
@@ -19,6 +19,9 @@ do
     t)
       tag=${OPTARG}
       ;;
+    s)
+      tagArgs="-s -m 'Signed during docker-versions-create"
+      ;;
     h)
       echo "
 Usage $0 [options]
@@ -28,6 +31,7 @@ Options:
           when any merge fails
   -t TAG  tag and push the current revision on master with the given tag
           and apply respective tags to each branch
+  -s      enable signed tags
   -h      display this help and exit
 "
       exit

examples/docker-compose-autopause.yml

@@ -0,0 +1,18 @@
+version: '3.8'
+
+services:
+  minecraft:
+    image: itzg/minecraft-server
+    ports:
+      - "25565:25565"
+    volumes:
+      - "mc:/data"
+    environment:
+      EULA: "TRUE"
+      ENABLE_AUTOPAUSE: "TRUE"
+      OVERRIDE_SERVER_PROPERTIES: "TRUE"
+      MAX_TICK_TIME: "-1"
+    restart: always
+
+volumes:
+  mc: {}

examples/docker-compose-paper.yml

@@ -7,11 +7,12 @@ services:
       EULA: "true"
       TYPE: PAPER
       VERSION: 1.9.4
-    command: --noconsole
+      # needed for Paper versions before 1.14
+      CONSOLE: "false"
     ports:
       - 25565:25565
     volumes:
       - mc-paper:/data
 
 volumes:
-  mc-paper:
+  mc-paper:

server.properties

@@ -37,3 +37,15 @@ motd=A Minecraft Server powered by Docker
 generator-settings=
 rcon.password=
 max-world-size=29999984
+broadcast-console-to-ops=true
+broadcast-rcon-to-ops=true
+enable-jmx-monitoring=false
+sync-chunk-writes=true
+enable-status=true
+entity-broadcast-range-percentage=100
+function-permission-level=2
+network-compression-threshold=256
+op-permission-level=4
+prevent-proxy-connections=false
+use-native-transport=true
+enforce-whitelist=false

start-configuration

@@ -33,6 +33,7 @@ fi
 if [[ $PROXY ]]; then
     export http_proxy="$PROXY"
     export https_proxy="$PROXY"
+    export JAVA_TOOL_OPTIONS+="-Djava.net.useSystemProxies=true"
     log "INFO: Giving proxy time to startup..."
     sleep 5
 fi
@@ -112,16 +113,21 @@ case "${TYPE^^}" in
   ;;
 
   MAGMA)
-    exec ${SCRIPTS:-/}start-magma "$@"
+    exec ${SCRIPTS:-/}start-deployMagma "$@"
   ;;
 
   MOHIST)
-    exec ${SCRIPTS:-/}start-mohist "$@"
+    exec ${SCRIPTS:-/}start-deployMohist "$@"
+  ;;
+
+  CATSERVER)
+    exec ${SCRIPTS:-/}start-deployCatserver "$@"
   ;;
 
   *)
       log "Invalid type: '$TYPE'"
-      log "Must be: VANILLA, FORGE, BUKKIT, SPIGOT, PAPER, FTB, CURSEFORGE, SPONGEVANILLA"
+      log "Must be: VANILLA, FORGE, BUKKIT, SPIGOT, PAPER, FTBA, CURSEFORGE, SPONGEVANILLA,"
+      log "         CUSTOM, CURSE_INSTANCE, MAGMA, MOHIST, CATSERVER"
       exit 1
   ;;
 

start-deployCatserver

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+. ${SCRIPTS:-/}start-utils
+set -o pipefail
+set -e
+
+latestAsset=$(
+  curl -fsSL https://api.github.com/repos/Luohuayu/CatServer/releases/latest | \
+    jq '.assets[] | select(.name | match(".*-universal.jar"))'
+)
+
+if [[ -z "${latestAsset}" ]]; then
+  log "ERROR: latest release of Catserver is missing universal.jar asset"
+  exit 1
+fi
+
+isDebugging && log "Latest asset ${latestAsset}"
+latestJarName=$(echo ${latestAsset} | jq --raw-output '.name')
+latestJarId=$(echo ${latestAsset} | jq --raw-output '.id')
+
+
+export SERVER="/data/${latestJarName}"
+
+if [ ! -f ${SERVER} ]; then
+  log "Downloading ${latestJarName}"
+  curl -H "Accept:application/octet-stream" -o "$SERVER" -fsSL https://api.github.com/repos/Luohuayu/CatServer/releases/assets/${latestJarId}
+fi
+
+export SKIP_LOG4J_CONFIG=true
+
+# Continue to Final Setup
+exec ${SCRIPTS:-/}start-finalSetup01World "$@"

start-deployMohist

@@ -1,17 +1,19 @@
 #!/bin/bash
 
 . ${SCRIPTS:-/}start-utils
-requireVar VANILLA_VERSION
 set -o pipefail
 set -e
 isDebugging && set -x
 
+requireVar VANILLA_VERSION
+
 mohistJobs=https://ci.codemc.io/job/Mohist-Community/job/
 mohistJob=${mohistJobs}Mohist-${VANILLA_VERSION}/
 
 if ! curl -X HEAD -o /dev/null -fsSL "${mohistJob}"; then
   log "ERROR: mohist builds do not exist for ${VANILLA_VERSION}"
   log "       check https://ci.codemc.io/job/Mohist-Community/ for available versions"
+  log "       and set VERSION accordingly"
   exit 1
 fi
 
@@ -24,17 +26,18 @@ baseName=$(basename "${latestBuildRelPath}")
 if [[ ${baseName} != *-server.jar* ]]; then
   log "ERROR: mohist build for ${VANILLA_VERSION} is not a valid server jar, found ${baseName}"
   log "       check https://ci.codemc.io/job/Mohist-Community/ for available versions"
+  log "       and set VERSION accordingly"
   exit 1
 fi
 
 export SERVER="/data/${baseName}"
 
 if [ ! -f ${SERVER} ]; then
-  log "Downloading ${SERVER}"
+  log "Downloading ${baseName}"
   curl -o "${SERVER}" -fsSL "${mohistJob}lastSuccessfulBuild/artifact/${latestBuildRelPath}"
 fi
 
 export SKIP_LOG4J_CONFIG=true
 
 # Continue to Final Setup
-exec ${SCRIPTS:-/}start-finalSetup01World $@
+exec ${SCRIPTS:-/}start-finalSetup01World "$@"

start-finalSetup01World

@@ -5,7 +5,7 @@ set -e
 isDebugging && set -x
 
 if [ $TYPE = "FEED-THE-BEAST" ]; then
-  worldDest=$FTB_BASE_DIR/$LEVEL
+  worldDest=$FTB_DIR/$LEVEL
 else
   worldDest=/data/$LEVEL
 fi

start-finalSetup04ServerProperties

@@ -77,6 +77,18 @@ function customizeServerProps {
   setServerProp "resource-pack" "$RESOURCE_PACK"
   setServerProp "resource-pack-sha1" "$RESOURCE_PACK_SHA1"
   setServerProp "player-idle-timeout" "$PLAYER_IDLE_TIMEOUT"
+  setServerProp "broadcast-console-to-ops" "$BROADCAST_CONSOLE_TO_OPS"
+  setServerProp "broadcast-rcon-to-ops" "$BROADCAST_RCON_TO_OPS"
+  setServerProp "enable-jmx-monitoring" "$ENABLE_JMX"
+  setServerProp "sync-chunk-writes" "$SYNC_CHUNK_WRITES"
+  setServerProp "enable-status" "$ENABLE_STATUS"
+  setServerProp "entity-broadcast-range-percentage" "$ENTITY_BROADCAST_RANGE_PERCENTAGE"
+  setServerProp "function-permission-level" "$FUNCTION_PERMISSION_LEVEL"
+  setServerProp "network-compression-threshold" "$NETWORK_COMPRESSION_THRESHOLD"
+  setServerProp "op-permission-level" "$OP_PERMISSION_LEVEL"
+  setServerProp "prevent-proxy-connections" "$PREVENT_PROXY_CONNECTIONS"
+  setServerProp "use-native-transport" "$USE_NATIVE_TRANSPORT"
+  setServerProp "enforce-whitelist" "$ENFORCE_WHITELIST"
 
   if [ -n "$DIFFICULTY" ]; then
     case $DIFFICULTY in

start-minecraftFinalSetup

@@ -190,6 +190,10 @@ elif [[ ${TYPE} == "CURSEFORGE" ]]; then
       cp -f /data/white-list.txt ${FTB_DIR}/
     fi
 
+    if [ ! -e "${FTB_DIR}/server-icon.png" -a -e /data/server-icon.png ]; then
+      cp -f /data/server-icon.png ${FTB_DIR}/
+    fi
+
     cp -f /data/eula.txt "${FTB_DIR}/"
 
     cat > "${FTB_DIR}/settings-local.sh" <<EOF