modrinth: resolve "only DEFLATED entries can have EXT" (#2332)

.github/workflows/build-multiarch.yml

@@ -163,8 +163,7 @@ jobs:
         env:
           MINECRAFT_VERSION: ${{ matrix.mcVersion }}
           VARIANT: ${{ matrix.variant }}
-          MODS_FORGEAPI_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
-          CF_API_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
+          CF_API_KEY: ${{ secrets.CF_API_KEY }}
         run: |
           tests/test.sh
 

.github/workflows/verify-pr.yml

@@ -83,8 +83,7 @@ jobs:
         env:
           MINECRAFT_VERSION: ${{ matrix.mcVersion }}
           VARIANT: ${{ matrix.variant }}
-          MODS_FORGEAPI_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
-          CF_API_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
+          CF_API_KEY: ${{ secrets.CF_API_KEY }}
           DEBUG: ${{ runner.debug }}
         run: |
           tests/test.sh

Dockerfile

@@ -35,14 +35,14 @@ RUN easy-add --var os=${TARGETOS} --var arch=${TARGETARCH}${TARGETVARIANT} \
   --from https://github.com/itzg/{{.app}}/releases/download/{{.version}}/{{.app}}_{{.version}}_{{.os}}_{{.arch}}.tar.gz
 
 RUN easy-add --var os=${TARGETOS} --var arch=${TARGETARCH}${TARGETVARIANT} \
-  --var version=0.12.1 --var app=mc-monitor --file {{.app}} \
+  --var version=0.12.2 --var app=mc-monitor --file {{.app}} \
   --from https://github.com/itzg/{{.app}}/releases/download/{{.version}}/{{.app}}_{{.version}}_{{.os}}_{{.arch}}.tar.gz
 
 RUN easy-add --var os=${TARGETOS} --var arch=${TARGETARCH}${TARGETVARIANT} \
   --var version=1.9.0 --var app=mc-server-runner --file {{.app}} \
   --from https://github.com/itzg/{{.app}}/releases/download/{{.version}}/{{.app}}_{{.version}}_{{.os}}_{{.arch}}.tar.gz
 
-ARG MC_HELPER_VERSION=1.32.8
+ARG MC_HELPER_VERSION=1.33.5
 ARG MC_HELPER_BASE_URL=https://github.com/itzg/mc-image-helper/releases/download/${MC_HELPER_VERSION}
 # used for cache busting local copy of mc-image-helper
 ARG MC_HELPER_REV=1

README.md

@@ -895,66 +895,6 @@ You may also download or copy over individual mods using the `MODS` environment
 
   docker run -d -e MODS=https://www.example.com/mods/mod1.jar,/plugins/common,/plugins/special/mod2.jar ...
 
-### ForgeAPI usage to use non-version specific projects
-
-**NOTE:** This potentially could lead to unexpected behavior if the Mod receives an update with unexpected behavior.
-
-This is more complicated because you will be pulling/using the latest mod for the release of your game. To get started make sure you have a [CursedForge API Key](https://docs.curseforge.com/#getting-started). Then use the environmental parameters in your docker build.
-
-Please be aware of the following when using these options for your mods:
-* Mod Release types: Release, Beta, and Alpha.
-* Mod dependencies: Required and Optional
-* Mod family: Fabric, Forge, and Bukkit.
-
-Parameters to use the ForgeAPI:
-
-* `MODS_FORGEAPI_KEY` - Required
-* `MODS_FORGEAPI_FILE` - Required or use MODS_FORGEAPI_PROJECTIDS (Overrides MODS_FORGEAPI_PROJECTIDS)
-* `MODS_FORGEAPI_PROJECTIDS` - Required or use MODS_FORGEAPI_FILE
-* `MODS_FORGEAPI_RELEASES` - Default is release, Options: [Release|Beta|Alpha]
-* `MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES` - Default is False, attempts to download required mods (releaseType Release) defined in Forge.
-* `MODS_FORGEAPI_IGNORE_GAMETYPE` - Default is False, Allows for filtering mods on family type: FORGE, FABRIC, and BUKKIT. (Does not filter for Vanilla or custom)
-* `REMOVE_OLD_FORGEAPI_MODS` - Default is False
-* `REMOVE_OLD_DATAPACKS_DEPTH` - Default is 1
-* `REMOVE_OLD_DATAPACKS_INCLUDE` - Default is *.jar
-
-Example of expected forge api project ids, releases, and key: 
-
-```yaml
-  MODS_FORGEAPI_PROJECTIDS: 306612,256717
-  MODS_FORGEAPI_RELEASES: Release
-  MODS_FORGEAPI_KEY: $WRX...
-```
-
-Example of expected ForgeAPI file format.
-
-**Field Description**: 
-* `name` is currently unused, but can be used to document each entry.
-* `projectId` id is the id found on the CurseForge website for a particular mod
-* `releaseType` Type corresponds to forge's R, B, A icon for each file. Default Release, options are (release|beta|alpha).
-* `fileName` is used for version pinning if latest file will not work for you.
-
-```json
-[
-  {
-      "name": "fabric api",
-      "projectId": "306612",
-      "releaseType": "release"
-  },
-  {
-      "name": "fabric voice mod",
-      "projectId": "416089",
-      "releaseType": "beta"
-  },
-  {
-      "name": "Biomes o plenty",
-      "projectId": "220318",
-      "fileName": "BiomesOPlenty-1.18.1-15.0.0.100-universal.jar",
-      "releaseType": "release"
-  }
-]
-```
-
 ### Generic pack files
 
 To install all the server content (jars, mods, plugins, configs, etc.) from a zip or tgz file, then set `GENERIC_PACK` to the container path or URL of the archive file. This can also be used to apply a CurseForge modpack that is missing a server start script and/or Forge installer.

docs/configuration/jvm-options.md

@@ -70,7 +70,9 @@ To enable remote JMX, such as for profiling with VisualVM or JMC, add the enviro
 
 ## Enable Aikar's Flags
 
-[Aikar has done some research](https://aikar.co/2018/07/02/tuning-the-jvm-g1gc-garbage-collector-flags-for-minecraft/) into finding the optimal JVM flags for GC tuning, which becomes more important as more users are connected concurrently. The set of flags documented there can be added using
+[Aikar has done some research](https://aikar.co/2018/07/02/tuning-the-jvm-g1gc-garbage-collector-flags-for-minecraft/) into finding the optimal JVM flags for GC tuning, which becomes more important as more users are connected concurrently. [PaperMC also has an explanation](https://docs.papermc.io/paper/aikars-flags) of what the JVM flags are doing.
+
+The set of flags documented there can be added using
 
     -e USE_AIKAR_FLAGS=true
 

docs/configuration/misc-options.md

@@ -61,7 +61,11 @@ If you would like to `docker attach` to the Minecraft server console with color
 
 To allow time for players to finish what they're doing during a graceful server shutdown, set `STOP_SERVER_ANNOUNCE_DELAY` to a number of seconds to delay after an announcement is posted by the server.
 
-> **NOTE** be sure to adjust Docker's shutdown timeout accordingly, such as using [the -t option on docker-compose down](https://docs.docker.com/compose/reference/down/).
+!!! warning "Increase stop grace period"
+
+    The Docker stop grace period must be increased to a value longer than the announce delay. The value to use that is longer than announce delay will vary based upon the amount of time it takes for final world data saving. If the container exits with exit code 137, then that indicates a longer grace period is needed. 
+    
+    The grace period can be increased using [the -t option on docker-compose down](https://docs.docker.com/compose/reference/down/) or set the [stop_grace_period](https://docs.docker.com/compose/compose-file/05-services/#stop_grace_period) in the compose file.
 
 ## OpenJ9 Specific Options
 

docs/misc/contributing/development.md

@@ -17,7 +17,6 @@ First, build a baseline of the image to include the packages needed by existing
 
 PowerShell: (Example of building and testing ForgeAPI)
 ```powershell
-$env:MODS_FORGEAPI_KEY='$2a$...'
 $env:FOLDER_TO_TEST="forgeapimods_projectids"
 $env:IMAGE_TO_TEST="mc-dev"
 docker build -t $env:IMAGE_TO_TEST .
@@ -36,7 +35,6 @@ docker build --build-arg BASE_IMAGE=$env:BASE_IMAGE -t $env:IMAGE_TO_TEST .
 
 Bash: (Example of building and testing ForgeAPI)
 ```bash
-export MODS_FORGEAPI_KEY='$2a$...'
 export FOLDER_TO_TEST="forgeapimods_file"
 export IMAGE_TO_TEST="mc-dev"
 docker build -t $IMAGE_TO_TEST .

docs/mods-and-plugins/curseforge-files.md

@@ -0,0 +1,45 @@
+# Auto-download from CurseForge
+
+Mods and plugins can be auto-downloaded and upgraded from CurseForge by setting `CURSEFORGE_FILES` to a comma or space delimited list of [project-file references](#project-file-references). References removed from the declaration are automatically cleaned up and setting `CURSEFORGE_FILES` to an empty string removes all previously managed project-files.
+
+A specific file can be omitted from each reference to allow for auto-selecting the newest version of the selected mod/plugin. The resolved `VERSION` and `TYPE` will be taken into consideration for selecting the appropriate file. 
+
+!!! warning "CurseForge API key usage"
+
+    A CurseForge API key must be allocated and set with `CF_API_KEY` [as described here](../types-and-platforms/mod-platforms/auto-curseforge.md#api-key).
+
+## Project-file references
+
+The following formats are supported in the list of project-file references:
+
+- A project page URL, such as `https://www.curseforge.com/minecraft/mc-mods/jei`. _The newest applicable file will be automatically selected._
+- A file page URL, such as `https://www.curseforge.com/minecraft/mc-mods/jei/files/4593548`
+- Project slug, such as `jei`. _The newest applicable file will be automatically selected._
+- Project ID, such as `238222`. _The newest applicable file will be automatically selected._
+- Project slug or ID, `:`, and a file ID, such as `jei:4593548` or `238222:4593548`
+- Project slug or ID, `@`, and a partial filename, such as `jei@10.2.1.1005`
+
+An `@` followed by the container path to a listing file can also be provided as a project-file reference. Each line is processed as a reference where blank lines and comments that start with `#` are ignored.
+
+For example, `CURSEFORGE_FILES` can be set to "@/extras/cf-mods.txt" where the container file `/extras/cf-mods.txt` contains
+
+```text
+# This comment is ignored
+jei:10.2.1.1005
+
+# This and previous blank line are ignore
+geckolib
+aquaculture
+naturalist
+```
+
+!!! tip "Docker Compose"
+
+    Making use of the space delimited option, compose file declarations can be organized nicely with a [multi-line string](https://yaml-multiline.info/), such as
+    
+    ```yaml
+          CURSEFORGE_FILES: |
+            geckolib
+            aquaculture
+            naturalist
+    ```

docs/mods-and-plugins/forge-api.md

@@ -1,65 +0,0 @@
-# Forge API
-
-## ForgeAPI usage to use non-version specific projects
-
-!!! warning "Deprecated"
-    This approach will soon be deprecated in favor of a variation of `AUTO_CURSEFORGE`.
-
-!!! warning
-    This potentially could lead to unexpected behavior if the Mod receives an update with unexpected behavior.
-
-This is more complicated because you will be pulling/using the latest mod for the release of your game. To get started make sure you have a [CursedForge API Key](https://docs.curseforge.com/#getting-started). Then use the environmental parameters in your docker build.
-
-Please be aware of the following when using these options for your mods:
-* Mod Release types: Release, Beta, and Alpha.
-* Mod dependencies: Required and Optional
-* Mod family: Fabric, Forge, and Bukkit.
-
-Parameters to use the ForgeAPI:
-
-* `MODS_FORGEAPI_KEY` - Required
-* `MODS_FORGEAPI_FILE` - Required or use MODS_FORGEAPI_PROJECTIDS (Overrides MODS_FORGEAPI_PROJECTIDS)
-* `MODS_FORGEAPI_PROJECTIDS` - Required or use MODS_FORGEAPI_FILE
-* `MODS_FORGEAPI_RELEASES` - Default is release, Options: [Release|Beta|Alpha]
-* `MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES` - Default is False, attempts to download required mods (releaseType Release) defined in Forge.
-* `MODS_FORGEAPI_IGNORE_GAMETYPE` - Default is False, Allows for filtering mods on family type: FORGE, FABRIC, and BUKKIT. (Does not filter for Vanilla or custom)
-* `REMOVE_OLD_FORGEAPI_MODS` - Default is False
-* `REMOVE_OLD_DATAPACKS_DEPTH` - Default is 1
-* `REMOVE_OLD_DATAPACKS_INCLUDE` - Default is *.jar
-
-Example of expected forge api project ids, releases, and key:
-
-```yaml
-  MODS_FORGEAPI_PROJECTIDS: 306612,256717
-  MODS_FORGEAPI_RELEASES: Release
-  MODS_FORGEAPI_KEY: $WRX...
-```
-
-Example of expected ForgeAPI file format.
-
-**Field Description**:
-* `name` is currently unused, but can be used to document each entry.
-* `projectId` id is the id found on the CurseForge website for a particular mod
-* `releaseType` Type corresponds to forge's R, B, A icon for each file. Default Release, options are (release|beta|alpha).
-* `fileName` is used for version pinning if latest file will not work for you.
-
-```json
-[
-  {
-      "name": "fabric api",
-      "projectId": "306612",
-      "releaseType": "release"
-  },
-  {
-      "name": "fabric voice mod",
-      "projectId": "416089",
-      "releaseType": "beta"
-  },
-  {
-      "name": "Biomes o plenty",
-      "projectId": "220318",
-      "fileName": "BiomesOPlenty-1.18.1-15.0.0.100-universal.jar",
-      "releaseType": "release"
-  }
-]
-```

docs/types-and-platforms/mod-platforms/ftb.md

@@ -19,6 +19,8 @@ This mod platform type will automatically take care of downloading and installin
   ```
 - `FTB_MODPACK_VERSION_ID`: optional, the numerical ID of the version to install. If not specified, the latest version will be installed. The "Version ID" can be obtained by hovering over a server file entry and grabbing [this ID in the URL](../../img/ftba-version-id-popup.png).
 
+- `FTB_FORCE_REINSTALL`: if the files become inconsistent, such as when accidentally removing them, the FTB can be forced to re-run by setting this to "true"
+
 ### Upgrading
 
 If a specific `FTB_MODPACK_VERSION_ID` was not specified, simply restart the container to pick up the newest modpack version. If using a specific version ID, recreate the container with the new version ID.

examples/curseforge-files/docker-compose.yml

@@ -0,0 +1,17 @@
+version: "3.8"
+
+services:
+  mc:
+    image: itzg/minecraft-server
+    environment:
+      EULA: true
+      TYPE: FORGE
+      # allocate from https://console.curseforge.com/ and set in .env file
+      CF_API_KEY: ${CF_API_KEY}
+      VERSION: 1.19.2
+      CURSEFORGE_FILES: |
+        geckolib
+        aquaculture
+        naturalist
+    ports:
+      - "25565:25565"

examples/docker-compose-rconcmd.yml

@@ -10,10 +10,12 @@ services:
     environment:
       EULA: "TRUE"
       TYPE: FABRIC
-      MODS_FORGEAPI_PROJECTIDS: 433175
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "TRUE"
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
+      MEMORY: "2G"
+      CURSEFORGE_FILES: |
+        fabric-api
+        chunky-pregenerator
+      # allocate from https://console.curseforge.com/ and set in .env file
+      CF_API_KEY: ${CF_API_KEY}
       # YAML Heredoc, be sure to use '|-' this will remove the first newline and final new line.
       # This is versus '|' that will leaving with two empty strings at top and bottom.
       RCON_CMDS_STARTUP:  |-

scripts/start-deployFTBA

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+: "${FTB_FORCE_REINSTALL:=false}"
+
 ftbInstallMarker=".ftb-installed"
 
 # shellcheck source=start-utils
@@ -32,7 +34,7 @@ elif ! [[ ${FTB_MODPACK_VERSION_ID} =~ [0-9]+ ]]; then
   exit 1
 fi
 
-if ! [ -f "${ftbInstallMarker}" ] || [ "$(cat "${ftbInstallMarker}")" != "${FTB_MODPACK_ID}=${FTB_MODPACK_VERSION_ID}" ]; then
+if isTrue "$FTB_FORCE_REINSTALL" || ! [ -f "${ftbInstallMarker}" ] || [ "$(cat "${ftbInstallMarker}")" != "${FTB_MODPACK_ID}=${FTB_MODPACK_VERSION_ID}" ]; then
   ftbInstaller=/data/ftb-installer
   if ! [[ -f "${ftbInstaller}" ]]; then
     log "Downloading FTB installer"

scripts/start-setupDatapack

@@ -84,4 +84,4 @@ if [[ ${VANILLATWEAKS_FILE} || ${VANILLATWEAKS_SHARECODE} ]]; then
     --pack-files="$VANILLATWEAKS_FILE"
 fi
 
-exec "${SCRIPTS:-/}start-setupForgeApiMods" "$@"
+exec "${SCRIPTS:-/}start-setupModpack" "$@"

scripts/start-setupForgeApiMods

@@ -1,244 +0,0 @@
-#!/bin/bash
-
-set -e -o pipefail
-
-: "${MODS_FORGEAPI_KEY:=}"
-: "${REMOVE_OLD_FORGEAPI_MODS:=false}"
-: "${MODS_FORGEAPI_PROJECTIDS:=}"
-: "${MODS_FORGEAPI_FILE:=}"
-: "${MODS_FORGEAPI_RELEASES:=RELEASE}"
-: "${MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES:=false}"
-: "${MODS_FORGEAPI_IGNORE_GAMETYPE:=false}"
-: "${REMOVE_OLD_MODS_DEPTH:=1} "
-: "${REMOVE_OLD_MODS_INCLUDE:=*.jar,*-version.json}"
-
-#  FORGEAPI_BASE_URL used in manifest downloads below
-FORGEAPI_BASE_URL=${FORGEAPI_BASE_URL:-https://api.curseforge.com/v1}
-RELEASE_NUMBER_FILTER=1
-MINECRAFT_GAME_ID=432
-FILTER_BY_FAMILY=false
-DOWNLOADED_MODIDS=()
-out_dir=/data/mods
-
-# shellcheck source=start-utils
-. "${SCRIPTS:-/}start-utils"
-isDebugging && set -x
-
-# Remove old mods/plugins
-if isTrue "${REMOVE_OLD_FORGEAPI_MODS}"; then
-  removeOldMods "/data/mods"
-fi
-
-# Family filter is on by default for Forge, Fabric, and Bukkit
-updateFamilyFilter(){
-  if isFamily "FORGE" "FABRIC" "BUKKIT"; then
-    FILTER_BY_FAMILY=true
-  fi
-}
-
-ensureModKey(){
-  if [ -z "$MODS_FORGEAPI_KEY" ]; then
-    log "ERROR: MODS_FORGEAPI_KEY REQUIRED to Connect to FORGE API, you supplied: ${MODS_FORGEAPI_KEY}"
-    exit 2
-  fi
-}
-
-# Set the global release type per the text.
-# NOTE: downcasing release type for comparing types.
-updateReleaseNumber(){
-  releaseType=$1
-  if [ "release" = "${releaseType,,}" ] || [ 1 = "${releaseType,,}" ]; then
-    RELEASE_NUMBER_FILTER=1
-  elif [ "beta" = "${releaseType,,}" ] || [ 2 = "${releaseType,,}" ]; then
-    RELEASE_NUMBER_FILTER=2
-  elif [ "alpha" = "${releaseType,,}" ] || [ 3 = "${releaseType,,}" ]; then
-    RELEASE_NUMBER_FILTER=3
-  fi
-}
-
-retrieveVersionTypeNumber(){
-  VERSION_NAME=$(get_major_version "$VERSION")
-  minecraft_types=$(curl -X GET -s \
-    "${FORGEAPI_BASE_URL}/games/${MINECRAFT_GAME_ID}/version-types" \
-    -H 'Accept: application/json' -H 'x-api-key: '${MODS_FORGEAPI_KEY}'')
-
-  if [ ! "$minecraft_types" ]; then
-    log "ERROR: unable to retrieve version types for ${VERSION_NAME} from ForgeAPI. Check Forge API key or supplied Minecraft version"
-    exit 2
-  fi
-
-  TYPE_ID=$(jq -n "$minecraft_types" | jq --arg VERSION_NAME "$VERSION_NAME" -jc '
-      .data[]? | select(.name==$VERSION_NAME) | .id')
-
-  if [ ! "$TYPE_ID" ]; then
-    log "ERROR: unable to retrieve version types for ${VERSION_NAME} from ForgeAPI"
-    exit 2
-  fi
-}
-
-modFileByProjectID(){
-  project_id=$(echo "$1" | tr -d '"')
-  project_id_release_type=$2
-  project_id_file_name=$3
-  unset PROJECT_FILE
-
-  # if Type id isn't defined use minecraft version to go get it.
-  if [ ! "$TYPE_ID" ]; then
-    retrieveVersionTypeNumber
-  fi
-
-  # JQ is struggling with larger page sizes so having to pagination for mods with a lot of releases
-  pageSize=42
-  index=0
-  total_count=1
-
-  while [ $index -lt $total_count ]; do
-    project_files=$(curl -X GET -s \
-      "${FORGEAPI_BASE_URL}/mods/${project_id}/files?gameVersionTypeId=${TYPE_ID}&index=${index}&pageSize=${pageSize}" \
-      -H 'Accept: application/json' -H 'x-api-key: '${MODS_FORGEAPI_KEY}'')
-
-    if [ ! "$project_files" ]; then
-      log "ERROR: unable to retrieve any project id files for ${project_id} from ForgeAPI"
-      exit 2
-    fi
-    # Use project files to grab out the total count of mods.
-    total_count=$(jq -n "$project_files" | jq -c '.pagination.totalCount' )
-
-    # Checking for a individual release type input, if not use global
-    if [ "$project_id_release_type" ]; then
-      updateReleaseNumber "$project_id_release_type"
-      unset project_id_release_type 
-    else
-      updateReleaseNumber $MODS_FORGEAPI_RELEASES
-    fi
-
-    # grabs the highest ID of the releaseTypes selected.
-    # Default is 1 for Release, Beta is 2, and Alpha is 3. Using less than we can validate highest release.
-    if [ "$project_id_file_name" ]; then
-      # Looks for file by name
-      current_project_file=$(jq -n "$project_files" | jq --arg FILE_NAME "$project_id_file_name" -jc '
-        .data | map(select(.fileName<=($FILE_NAME))) | .[0] // empty')
-    elif isFalse "${MODS_FORGEAPI_IGNORE_GAMETYPE}" && $FILTER_BY_FAMILY ; then
-      # Looks for file by version and server type in lowercase
-      current_project_file=$(jq -n "$project_files" | jq --arg RELEASE_FILTER "$RELEASE_NUMBER_FILTER" --arg GAME_TYPE "${FAMILY,,}" --arg VERSION "$VANILLA_VERSION" -jc '
-        .data | sort_by(.id) | reverse | map(select(.gameVersions[] | ascii_downcase | contains ($GAME_TYPE))) |  map(select(.gameVersions[] | contains ($VERSION))) | map(select(.releaseType<=($RELEASE_FILTER|tonumber))) | .[0] // empty')
-    else
-      # Looks for file by version only.
-      current_project_file=$(jq -n "$project_files" | jq --arg RELEASE_FILTER "$RELEASE_NUMBER_FILTER" --arg VERSION "$VANILLA_VERSION" -jc '
-        .data | sort_by(.id) | reverse | map(select(.gameVersions[] | contains ($VERSION))) | map(select(.releaseType<=($RELEASE_FILTER|tonumber))) | .[0] // empty')
-    fi
-
-    # Logic to grab the latest release over the entire pagination
-    if [ ! "$PROJECT_FILE" ]; then
-      PROJECT_FILE=$current_project_file
-    elif [ "$current_project_file" ]; then
-      current_project_file_id=$(jq -n "$current_project_file" | jq -jc '.id // empty' )
-      PROJECT_FILE_ID=$(jq -n "$PROJECT_FILE" | jq -jc '.id // empty' )
-      if (( current_project_file_id > PROJECT_FILE_ID )); then
-        PROJECT_FILE=$current_project_file
-      fi
-    fi
-    
-    # check to see if we have gone to far or lost our index and exit with an error
-    if [ -z "$index" ] || [ -z "$total_count" ] || [ $index -ge "$total_count" ]; then
-      log "ERROR: Unable to retrieve any files for ${project_id} from ForgeAPI also Validate files have release type associated with no. ${RELEASE_NUMBER_FILTER}"
-      exit 2
-    fi
-    # Increment start index to new set.
-    index=$((index + pageSize))
-  done
-  if [ ! "$PROJECT_FILE" ]; then
-    log "ERROR: Unable to retrieve any files for ${project_id}, Release Type: ${RELEASE_NUMBER_FILTER}, FAMILY_TYPE: ${FAMILY,,}"
-    exit 2
-  fi
-}
-
-downloadModPackfromModFile() {
-  if [ ! "$PROJECT_FILE" ]; then
-    log "ERROR: Project File not found from the ForgeAPI"
-    exit 2
-  fi
-  # trys to make the output directory incase it doesnt exist.
-  mkdir -p "$out_dir"
-  debug "DEBUG: PROJECT_FILE: ${PROJECT_FILE}"
-  # grabs needed values from our json return
-  file_name=$(jq -n "$PROJECT_FILE" | jq -jc '.fileName // empty' )
-  download_url=$(jq -n "$PROJECT_FILE" | jq -jc '.downloadUrl // empty' )
-  mod_id=$(jq -n "$PROJECT_FILE" | jq -jc '.modId // empty' )
-
-  if [ ! -f "${out_dir}/${file_name}" ]; then
-    echo "Downloading ${download_url}"
-    # Track the mods we have downloaded.
-    DOWNLOADED_MODIDS+=("${mod_id}")
-    if ! get --skip-up-to-date -o "${out_dir}/${file_name}" "${download_url}"; then
-      log "ERROR: failed to download from ${download_url}"
-      exit 2
-    fi
-  fi
-}
-
-downloadDependencies(){
-  if [ "$PROJECT_FILE" ]; then
-    dependencies=$(jq -n "$PROJECT_FILE" | jq -jc '.dependencies' )
-    required_dependencies=$(jq -n "$dependencies" | jq --arg REQUIRED_FILTER "3" -jc '
-      map(select(.relationType==($REQUIRED_FILTER|tonumber)))')
-    if [ "$required_dependencies" ]; then
-      while read -r current_dependency; do
-        mod_id=$(jq -n "$current_dependency" | jq -jc '.modId' )
-        # Validate we have not tried to download the mod yet.
-        if [[ ! "${DOWNLOADED_MODIDS[*]}" =~ $mod_id ]]; then
-          modFileByProjectID "$mod_id" "release"
-          downloadModPackfromModFile
-        fi
-      # needs to be piped in to keep look in main process
-      done < <(jq -n "$required_dependencies" | jq -c '.[]?')
-    fi
-  fi
-}
-
-# Use forge api json file to filter and download the correct mods
-if [ "$MODS_FORGEAPI_FILE" ] && [ -z "$MODS_FORGEAPI_PROJECTIDS" ]; then
-  ensureModKey
-  updateFamilyFilter
-  if [ ! -f "$MODS_FORGEAPI_FILE" ]; then
-    log "ERROR: given MODS_FORGEAPI_FILE file does not exist"
-    exit 2
-  fi
-  debug "DEBUG: MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_FILE}"
-  # Needs loop here to look up release types befor calling download.
-  while read -r current_project; do
-    debug "DEBUG: current_project: ${current_project}"
-    # Per stack overflow we can use //empty to return empty string that works with -z
-    project_id=$(jq -n "$current_project" | jq -r '.projectId // empty' )
-    current_release_type=$(jq -n "$current_project" | jq -r '.releaseType // empty' )
-    current_file_name=$(jq -n "$current_project" | jq -r '.fileName // empty' )
-
-    # Validate we have not tried to download the mod yet.
-    if [[ ! "${DOWNLOADED_MODIDS[*]}" =~ $project_id ]]; then
-      modFileByProjectID "$project_id" "$current_release_type" "$current_file_name"
-      downloadModPackfromModFile
-      if isTrue "${MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES}"; then
-        downloadDependencies
-      fi
-    fi
-  # needs to be piped in to keep look in main process
-  done < <(jq -c '.[]?' $MODS_FORGEAPI_FILE)
-fi
-
-# Use only project ids and global release data.
-if [ "$MODS_FORGEAPI_PROJECTIDS" ] && [ -z "$MODS_FORGEAPI_FILE" ]; then
-  ensureModKey
-  updateFamilyFilter
-  for project_id in ${MODS_FORGEAPI_PROJECTIDS//,/ }; do
-    # Validate we have not tried to download the mod yet.
-    if [[ ! "${DOWNLOADED_MODIDS[*]}" =~ $project_id ]]; then
-      modFileByProjectID $project_id 
-      downloadModPackfromModFile
-      if isTrue "${MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES}"; then
-        downloadDependencies
-      fi
-    fi
-  done
-fi
-
-exec "${SCRIPTS:-/}start-setupModpack" "$@"

scripts/start-setupModpack

@@ -143,51 +143,6 @@ function handleListings() {
   fi
 }
 
-function handleCurseForgeManifest() {
-if [[ "$MANIFEST" ]]; then
-    if [[ -e "$MANIFEST" ]]; then
-        EFFECTIVE_MANIFEST_FILE=$MANIFEST
-    elif isURL "$MANIFEST"; then
-        EFFECTIVE_MANIFEST_FILE=/tmp/manifest.json
-        EFFECTIVE_MANIFEST_URL=$(curl -Ls -o /dev/null -w "%{effective_url}" "$MANIFEST")
-        curl -Ls -o $EFFECTIVE_MANIFEST_FILE "$EFFECTIVE_MANIFEST_URL"
-    else
-        log "MANIFEST='$MANIFEST' is not a valid manifest url or location"
-        exit 2
-    fi
-
-case "X$EFFECTIVE_MANIFEST_FILE" in
-  X*.json)
-    if [ -f "${EFFECTIVE_MANIFEST_FILE}" ]; then
-      MOD_DIR=${FTB_BASE_DIR:-/data}/mods
-      if [ ! -d "$MOD_DIR" ]
-      then
-        log "Creating mods dir $MOD_DIR"
-        mkdir -p "$MOD_DIR"
-      fi
-      log "Starting manifest download..."
-      jq -r '.files[] | (.projectID|tostring) + " " + (.fileID|tostring)' "${EFFECTIVE_MANIFEST_FILE}" | while read -r p f
-      do
-        if [ ! -f $MOD_DIR/${p}_${f}.jar ]
-        then
-          redirect_url="$(curl -Ls -o /dev/null -w "%{effective_url}" "${CURSE_URL_BASE}/${p}")"
-          url="$redirect_url/download/${f}/file"
-          log Downloading curseforge mod $url
-          #  Manifest usually doesn't have mod names. Using id should be fine, tho
-          curl -sSL "${url}" -o $MOD_DIR/${p}_${f}.jar
-        fi
-      done
-    else
-      log "Could not find manifest file, insufficient privileges, or malformed path."
-    fi
-    ;;
-  *)
-    log "Invalid manifest file for modpack. Please make sure it is a .json file."
-    ;;
-esac
-fi
-}
-
 function handleGenericPacks() {
   : "${GENERIC_PACKS:=${GENERIC_PACK}}"
   : "${GENERIC_PACKS_PREFIX:=}"
@@ -296,16 +251,51 @@ function handleModrinthProjects() {
   fi
 }
 
+function handleCurseForgeFiles() {
+  args=()
+  if usesMods && ! usesPlugins; then
+    args+=(--default-category mc-mods)
+  elif usesPlugins && ! usesMods; then
+    args+=(--default-category bukkit-plugins)
+  fi
+
+  case "${TYPE,,}" in
+    forge|fabric|quilt)
+      args+=(--mod-loader "$TYPE")
+      ;;
+  esac
+
+  # shellcheck disable=SC2086
+  # since we want CURSEFORGE_FILES to expand
+  mc-image-helper curseforge-files \
+    "${args[@]}" \
+    ${CURSEFORGE_FILES}
+}
+
 handlePackwiz
 
 handleModpackZip
 
 handleListings
 
-handleCurseForgeManifest
+if [[ $MANIFEST ]]; then
+  log "ERROR: MANIFEST is no longer supported."
+  log "       Use MOD_PLATFORM=AUTO_CURSEFORGE and CF_MODPACK_MANIFEST instead"
+  exit 1
+fi
+
+if [[ $MODS_FORGEAPI_KEY || $MODS_FORGEAPI_FILE || $MODS_FORGEAPI_PROJECTIDS ]]; then
+  log "ERROR the MODS_FORGEAPI_FILE / MODS_FORGEAPI_PROJECTIDS feature is no longer supported"
+  log "      Use CURSEFORGE_FILES instead."
+  exit 1
+fi
 
 handleGenericPacks
 
 handleModrinthProjects
 
+if usesMods || usesPlugins; then
+  handleCurseForgeFiles
+fi
+
 exec "${SCRIPTS:-/}start-setupModconfig" "$@"

scripts/start-utils

@@ -208,14 +208,17 @@ eula=${EULA,,}
 function removeOldMods {
   if [ -d "$1" ]; then
     log "Removing old mods including:${REMOVE_OLD_MODS_INCLUDE} excluding:${REMOVE_OLD_MODS_EXCLUDE}"
-    mc-image-helper find \
-      --delete \
-      --type file,directory \
-      --min-depth=1 --max-depth "${REMOVE_OLD_MODS_DEPTH:-16}" \
-      --name "${REMOVE_OLD_MODS_INCLUDE:-*}" \
-      --exclude-name "${REMOVE_OLD_MODS_EXCLUDE:-}" \
-      --quiet \
-      "$1"
+    args=(
+      --delete
+      --type file
+      --min-depth=1 --max-depth "${REMOVE_OLD_MODS_DEPTH:-16}"
+      --name "${REMOVE_OLD_MODS_INCLUDE:-*}"
+      --exclude-name "${REMOVE_OLD_MODS_EXCLUDE:-}"
+    )
+    if ! isDebugging; then
+      args+=(--quiet)
+    fi
+    mc-image-helper find "${args[@]}" "$1"
   fi
 }
 

tests/setuponlytests/cf_files/docker-compose.yml

@@ -0,0 +1,22 @@
+version: "3"
+
+services:
+  mc:
+    restart: "no"
+    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
+    environment:
+      EULA: "TRUE"
+      SETUP_ONLY: "TRUE"
+      TYPE: FABRIC
+      VERSION: 1.18.2
+      # Contains mix of Forge and Fabric mods
+      CURSEFORGE_FILES: |
+        https://www.curseforge.com/minecraft/mc-mods/clumps/files/4153343
+        jei
+        306612
+        @/extras/listing.txt
+      CF_API_KEY: ${CF_API_KEY}
+    volumes:
+      - ./data:/data
+      - ./fake.jar:/servers/fake.jar:ro
+      - ./listing.txt:/extras/listing.txt:ro

tests/setuponlytests/cf_files/listing.txt

@@ -0,0 +1 @@
+architectury-api@4.11.93

tests/setuponlytests/cf_files/require.sh

@@ -0,0 +1 @@
+[[ $CF_API_KEY ]] || exit 1

tests/setuponlytests/cf_files/verify.sh

@@ -0,0 +1,5 @@
+mc-image-helper assert fileExists "/data/mods/architectury-*-fabric.jar"
+mc-image-helper assert fileExists "/data/mods/Clumps-fabric-*.jar"
+mc-image-helper assert fileExists "/data/mods/fabric-api-*.jar"
+mc-image-helper assert fileExists "/data/mods/jei-*-fabric-*.jar"
+

tests/setuponlytests/conflicting_world_for_spigot_server/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/conflicting_world_for_vanilla_server/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/forgeapimods_file/docker-compose.yml

@@ -1,27 +0,0 @@
-version: "3"
-
-services:
-  mc:
-    restart: "no"
-    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
-    environment:
-      EULA: "TRUE"
-      SETUP_ONLY: "TRUE"
-      DEBUG: "FALSE"
-      # Using custom to bypass Fabric setup
-      TYPE: CUSTOM
-      # Using family to test FORGEAPI Family filter.
-      FAMILY: FABRIC
-      CUSTOM_SERVER: /servers/fake.jar
-      VERSION: 1.18.2
-      MODS_FORGEAPI_FILE: /config/forgeapi_mods.json
-      # Key is passed by Github Workflow
-      # This should be coming from github secrets.
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
-      # Validates that Fabric API gets download as a dependency.
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "TRUE"
-    volumes:
-      - ./forgeapi_mods.json:/config/forgeapi_mods.json:ro
-      - ./data:/data
-      - ./fake.jar:/servers/fake.jar

tests/setuponlytests/forgeapimods_file/forgeapi_mods.json

@@ -1,11 +0,0 @@
-[
-  {
-    "name": "Flan (Land Claim Tool)",
-    "projectId": "404578"
-  },
-  {
-    "name": "Fabric Voice Mod",
-    "projectId": "416089",
-    "releaseType": "beta"
-  }
-]

tests/setuponlytests/forgeapimods_file/require.sh

@@ -1,2 +0,0 @@
-[[ $MINECRAFT_VERSION == LATEST ]] || exit 1
-[[ $MODS_FORGEAPI_KEY ]] || exit 1

tests/setuponlytests/forgeapimods_file/verify.sh

@@ -1,5 +0,0 @@
-# Validates specific beta call out for specific mod:
-mc-image-helper assert fileExists "/data/mods/voicechat-fabric*"
-mc-image-helper assert fileExists "/data/mods/flan*"
-# Dependent of flan:
-mc-image-helper assert fileExists "/data/mods/fabric-api*"

tests/setuponlytests/forgeapimods_gametype/docker-compose.yml

@@ -1,30 +0,0 @@
-version: "3"
-
-services:
-  mc:
-    restart: "no"
-    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
-    environment:
-      EULA: "TRUE"
-      SETUP_ONLY: "TRUE"
-      DEBUG: "FALSE"
-      # Using custom to bypass Fabric setup
-      TYPE: CUSTOM
-      # Using family to test FORGEAPI Family filter.
-      FAMILY: FABRIC
-      CUSTOM_SERVER: /servers/fake.jar
-      VERSION: 1.18.2
-      # Validate Skip Gametype Filter:
-      MODS_FORGEAPI_IGNORE_GAMETYPE: "TRUE"
-      # Validates that Biomes does not download terrablender
-      # Using default false for testing: 
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "FALSE"
-      # Contains mix of Forge and Fabric mods
-      MODS_FORGEAPI_PROJECTIDS: 416089,493246
-      # Allows for Beta releases of 416089 the Fabric Voice Mod 
-      MODS_FORGEAPI_RELEASES: BETA
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
-    volumes:
-      - ./data:/data
-      - ./fake.jar:/servers/fake.jar

tests/setuponlytests/forgeapimods_gametype/require.sh

@@ -1,2 +0,0 @@
-[[ $MINECRAFT_VERSION == LATEST ]] || exit 1
-[[ $MODS_FORGEAPI_KEY ]] || exit 1

tests/setuponlytests/forgeapimods_gametype/verify.sh

@@ -1,5 +0,0 @@
-# testing dependencies don't get downloaded when download dependencies is set to false.
-mc-image-helper assert fileExists "/data/mods/voicechat-fabric*"
-mc-image-helper assert fileExists "/data/mods/flan*"
-# Dependent of flan, but dependencies are set to false:
-! mc-image-helper assert fileExists "/data/mods/fabric-api*"

tests/setuponlytests/forgeapimods_projectids/docker-compose.yml

@@ -1,26 +0,0 @@
-version: "3"
-
-services:
-  mc:
-    restart: "no"
-    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
-    environment:
-      EULA: "TRUE"
-      SETUP_ONLY: "TRUE"
-      # Using custom to bypass Fabric setup
-      TYPE: CUSTOM
-      # Validate Skip Gametype Filter for vanilla 
-      #  - Currently we do not support filtering on vanilla.
-      FAMILY: VANILLA
-      CUSTOM_SERVER: /servers/fake.jar
-      VERSION: 1.18.2
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "TRUE"
-      # Contains mix of Forge and Fabric mods
-      MODS_FORGEAPI_PROJECTIDS: 416089,419697
-      # Allows for Beta releases of 416089 the Fabric Voice Mod 
-      MODS_FORGEAPI_RELEASES: BETA
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
-    volumes:
-      - ./data:/data
-      - ./fake.jar:/servers/fake.jar

tests/setuponlytests/forgeapimods_projectids/require.sh

@@ -1,2 +0,0 @@
-[[ $MINECRAFT_VERSION == LATEST ]] || exit 1
-[[ $MODS_FORGEAPI_KEY ]] || exit 1

tests/setuponlytests/forgeapimods_projectids/verify.sh

@@ -1,6 +0,0 @@
-# No family filter applied, DO NOT use Fabric or Forge specific name validation as it may cause random breakage.
-mc-image-helper assert fileExists "/data/mods/voicechat-fabric-1*"
-# Should be pull v4 and higher for 1.18.2:
-mc-image-helper assert fileExists "/data/mods/architectury-4*"
-mc-image-helper assert fileExists "/data/mods/fabric-api*"
-

tests/setuponlytests/icon-file-exact/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/icon-gif-multiframe/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/icon-png-atscale/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/spigot_world_for_vanilla_server/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/spongevanilla_version_compare/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/vanilla_world_for_spigot_server/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/vanillatweaks_sharecode/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/world_from_tar/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/world_from_tarbz2/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1

tests/setuponlytests/world_from_tarzst/require.sh

@@ -0,0 +1 @@
+[[ $EXTENDED_TESTS ]] || exit 1