From 354f6aeed656b690a62f5bc1d6ccd9277839431b Mon Sep 17 00:00:00 2001 From: Michael Eischer <9106997+MichaelEischer@users.noreply.github.com> Date: Sun, 15 Nov 2026 17:40:08 +0100 Subject: [PATCH] Untangle and clarify the contributing guide (#22169) * Small typos and clarification in contributing guide * cross-link contributing guide and issue/pr template * drop GOPATH mention * don't include merge commits in PRs * make high review delays explicit * explicitly ask for link to current PR in changelog template * clarify docs changes * use bug template as authoritative place' * consolidate development information in 090_participating Also move issue / PR creation relevant information into contributing.md. * clarify meaning of assigning to a pull request * link to minimum go version * minor typos * golanci-lint is always applied in full * add micro summary * more typo fixes * always run golangci-lint * Simplify issue/pr template * Fix IRC webchat link * Move security policy to SECURITY.md * address review comments --- .github/ISSUE_TEMPLATE/Bug.md | 25 ++--- .github/ISSUE_TEMPLATE/Feature.md | 12 +-- .github/PULL_REQUEST_TEMPLATE.md | 8 +- .github/workflows/tests.yml | 5 - CONTRIBUTING.md | 158 +++++++++++------------------- README.md | 10 +- SECURITY.md | 15 +++ changelog/TEMPLATE | 2 + doc/090_participating.rst | 144 ++++++++++++++++++--------- 9 files changed, 198 insertions(+), 181 deletions(-) create mode 100644 SECURITY.md diff --git a/.github/ISSUE_TEMPLATE/Bug.md b/.github/ISSUE_TEMPLATE/Bug.md index 532983773..5e639d065 100644 --- a/.github/ISSUE_TEMPLATE/Bug.md +++ b/.github/ISSUE_TEMPLATE/Bug.md @@ -10,18 +10,10 @@ Welcome! - We kindly ask that you: 1. Fill out the issue template below - not doing so needs a good reason. 2. Use the forum if you have a question rather than a bug or feature request. -The forum is at: https://forum.restic.net - -NOTE: Not filling out the issue template needs a good reason, as otherwise it -may take a lot longer to find the problem, not to mention it can take up a lot -more time which can otherwise be spent on development. Please also take the -time to help us debug the issue by collecting relevant information, even if -it doesn't seem to be relevant to you. Thanks! - -The forum is a better place for questions about restic or general suggestions -and topics, e.g. usage or documentation questions! This issue tracker is mainly -for tracking bugs and feature requests directly relating to the development of -the software itself, rather than the project. +The forum is at https://forum.restic.net and is a better place for questions +about restic or general suggestions and topics, e.g. usage or documentation +questions! This issue tracker is mainly for tracking bugs and feature requests +directly relating to the development of the software itself. Thanks for understanding, and for contributing to the project! @@ -49,11 +41,12 @@ This section should include at least: * The complete command line and any environment variables you used to configure restic's backend access. Make sure to replace sensitive values! - * The output of the commands, what restic prints gives may give us much - information to diagnose the problem! + * The output of the commands – what restic prints often provides a clue. - * The more time you spend describing an easy way to reproduce the behavior (if - this is possible), the easier it is for the project developers to fix it! + * Clear and concise steps describing how to reproduce the behavior (if possible). + +See https://github.com/restic/restic/blob/master/CONTRIBUTING.md#reporting-bugs +for ways to collect debug information if necessary. --> diff --git a/.github/ISSUE_TEMPLATE/Feature.md b/.github/ISSUE_TEMPLATE/Feature.md index 5292c9d5f..603647849 100644 --- a/.github/ISSUE_TEMPLATE/Feature.md +++ b/.github/ISSUE_TEMPLATE/Feature.md @@ -10,12 +10,10 @@ Welcome! - We kindly ask that you: 1. Fill out the issue template below - not doing so needs a good reason. 2. Use the forum if you have a question rather than a bug or feature request. -The forum is at: https://forum.restic.net - -The forum is a better place for questions about restic or general suggestions -and topics, e.g. usage or documentation questions! This issue tracker is mainly -for tracking bugs and feature requests directly relating to the development of -the software itself, rather than the project. +The forum is at https://forum.restic.net and is a better place for questions +about restic or general suggestions and topics, e.g. usage or documentation +questions! This issue tracker is mainly for tracking bugs and feature requests +directly relating to the development of the software itself. Thanks for understanding, and for contributing to the project! @@ -43,7 +41,7 @@ What are you trying to do? What problem would this solve? --------------------------------------------------------- diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 032c74c76..7e836c999 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -8,7 +8,7 @@ What does this PR change? What problem does it solve? ----------------------------------------------------- Was the change previously discussed in an issue or on the forum? @@ -32,11 +32,11 @@ check the last box. Enable a checkbox by replacing [ ] with [x]. Please always follow these steps: - Read the [contribution guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches). - Enable [maintainer edits](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork). -- Run `gofmt` on the code in all commits. +- Run `gofmt -w **/*.go` on the code in all commits. Use `golangci-lint run` to lint the code. The CI pipeline also runs those checks. - Format all commit messages in the same style as [the other commits in the repository](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits). --> - [ ] I have added tests for all code changes, see [writing tests](https://restic.readthedocs.io/en/stable/090_participating.html#writing-tests) -- [ ] I have added documentation for relevant changes (in the manual). -- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (see [template](https://github.com/restic/restic/blob/master/changelog/TEMPLATE)). +- [ ] I have added documentation for relevant changes (in the manual). NOTE: Do not edit files in `doc/man`, as they are auto-generated. +- [ ] There's a new file in `changelog/unreleased/` describing the changes for our users if relevant (see [template](https://github.com/restic/restic/blob/master/changelog/TEMPLATE)). - [ ] I'm done! This pull request is ready for review. diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 502b642d3..bd97be3bb 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -267,11 +267,6 @@ jobs: version: v2.14 args: --verbose --timeout 5m - # only run golangci-lint for pull requests, otherwise ALL hints get - # reported. We need to slowly address all issues until we can enable - # linting the master branch :) - if: github.event_name == 'pull_request' - - name: Check go.mod/go.sum run: | echo "check if go.mod and go.sum are up to date" diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d1900fa16..0390aaba9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,28 +1,27 @@ -This document describes the way you can contribute to the restic project. +This document describes how to report security issues and bugs, and how to provide +patches for restic. Ways to Help Out ================ -Thank you for your contribution! Please **open an issue first** (or add a -comment to an existing issue) if you plan to work on any code or add a new -feature. This way, duplicate work is prevented and we can discuss your ideas -and design first. Small bugfixes are an exception to this rule, just open a +Thank you for your contribution! Always check if there is an existing issue and +add a comment there or otherwise **open an issue first** if you plan to work +on any larger amount of code or add a new significant feature. +This way, duplicate work is prevented and we can discuss your ideas and +design first. Small bugfixes are an exception to this rule, just open a pull request in this case. -There are several ways you can help us out. First of all code contributions and -bug fixes are most welcome. However even "minor" details as fixing spelling -errors, improving documentation or pointing out usability issues are a great -help also. +Contributions are welcome in general, but expect long review times due to +a general overload of the maintainers. The restic project uses the GitHub infrastructure (see the [project page](https://github.com/restic/restic)) for all related discussions as well as the [forum](https://forum.restic.net/) and the `#restic` channel -on [irc.libera.chat](https://kiwiirc.com/nextclient/#ircs://irc.libera.chat:6697/#restic). +on [irc.libera.chat](https://web.libera.chat/#restic). -If you want to find an area that currently needs improving have a look at the -open issues listed at the -[issues page](https://github.com/restic/restic/issues). This is also the place -for discussing enhancement to the restic tools. +If you want to find an area that currently needs improving, have a look at the +open issues listed at the [issues page](https://github.com/restic/restic/issues). +This is also the place for discussing enhancements to the restic tools. If you are unsure what to do, please have a look at the issues, especially those tagged @@ -35,19 +34,11 @@ at the issues tagged as [help wanted](https://github.com/restic/restic/labels/he Reporting Bugs ============== -You've found a bug? Thanks for letting us know so we can fix it! It is a good -idea to describe in detail how to reproduce the bug (when you know how), what -environment was used and so on. Please tell us at least the following things: - - * What's the version of restic you used? Please include the output of - `restic version` in your bug report. - * What commands did you execute to get to where the bug occurred? - * What did you expect? - * What happened instead? - * Are you aware of a way to reproduce the bug? - -Remember, the easier it is for us to reproduce the bug, the earlier it will be -corrected! +You've found a bug? Thanks for letting us know so we can fix it! Please fill out +the [Bug issue template](https://github.com/restic/restic/blob/master/.github/ISSUE_TEMPLATE/Bug.md?plain=1), +but refer to [Security Policy](./SECURITY.md) for critical security issues. +We need the information to reproduce and investigate bugs. Remember, the easier +it is for us to reproduce the bug, the earlier it will be corrected! In addition, you can instruct restic to create a debug log by setting the environment variable `DEBUG_LOG` to a file, e.g. like this: @@ -66,64 +57,20 @@ and then exit immediately. This will not damage your repository, however, it might be necessary to manually clean up stale lock files using `restic unlock`. -On Windows, please set the environment variable `RESTIC_DEBUG_STACKTRACE_SIGINT` -to `true` and press `Ctrl-C` to create a stacktrace. +On Windows, please run restic with the environment variable +`RESTIC_DEBUG_STACKTRACE_SIGINT` set to `true` and press `Ctrl-C` to +create a stacktrace. -If you think restic uses too much memory or a too large cache directory, then -please include the output of `restic stats --mode debug`. +If you think restic uses too much memory or the cache directory is too large, +then please include the output of `restic stats --mode debug`. -Development Environment -======================= +Restic Development +================== -The repository contains the code written for restic in the directories -`cmd/` and `internal/`. - -Make sure you have the minimum required Go version installed. Clone the repo -(without having `$GOPATH` set) and `cd` into the directory: - - $ unset GOPATH - $ git clone https://github.com/restic/restic - $ cd restic - -Then use the `go` tool to build restic: - - $ go build ./cmd/restic - $ ./restic version - restic 0.14.0-dev (compiled manually) compiled with go1.19 on linux/amd64 - -To create a debug build use: - - $ go build -tags debug ./cmd/restic - -You can run all tests with the following command: - - $ go test ./... - - -Performance and Memory Usage Issues -=================================== - -Debug builds of restic support the `--block-profile`, `--cpu-profile`, -`--mem-profile`, and `--trace-profile` options which collect performance data -that later on can be analyzed using the go tools: - - $ restic --cpu-profile . [...] - $ go tool pprof -http localhost:12345 cpu.pprof - -To analyze a trace profile use `go tool trace -http=localhost:12345 trace.out`. - -As the memory usage of restic changes over time, it may be useful to capture a -snapshot of the current heap. This is possible using then `--listen-profile` -option. Then while restic runs you can query and afterwards analyze the heap statistics. - - $ restic --listen-profile localhost:12345 [...] - $ curl http://localhost:12345/debug/pprof/heap -o heap.pprof - $ go tool pprof -http localhost:12345 heap.pprof - -Further useful tools are setting the environment variable `GODEBUG=gctrace=1`, -which provides information about garbage collector runs. For a graphical variant -combine this with gcvis. +The [development environment](doc/090_participating.rst#development-environment) +section describes how to build restic and run the tests. The documentation also +contains guidance on debugging and writing tests. Providing Patches @@ -136,7 +83,8 @@ down to the following steps: 0. If you want to work on something, please add a comment to the issue on GitHub. For a new feature, please add an issue before starting to work on - it, so that duplicate work is prevented. + it, so that duplicate work is prevented. Small bugfixes are an exception + to this rule, just open a pull request in this case. 1. Next, fork our project on GitHub if you haven't done so already. @@ -144,30 +92,38 @@ down to the following steps: your changes. If you are working on the code itself, please set up the development environment as described in the previous section. - 3. Commit your changes to the new branch as fine grained as possible, as - smaller patches, for individual changes, are easier to discuss and merge. + 3. Commit in small, cohesive pieces. Smaller patches are easier to discuss and merge. 4. Push the new branch with your changes to your fork of the repository. - 5. Create a pull request by visiting the GitHub website, it will guide you - through the process. Please [allow edits from maintainers](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork). + 5. Create a pull request by visiting the GitHub website. It will guide you + through the process. You **must** fill out the [Pull Request template](https://github.com/restic/restic/blob/master/.github/PULL_REQUEST_TEMPLATE.md?plain=1). + Please also [allow edits from maintainers](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork). - 6. You will receive comments on your code and the feature or bug that they + 6. You will receive comments on your commits and the feature or bug that they address. Maybe you need to rework some minor things, in this case push new commits to the branch you created for the pull request (or amend the - existing commit, use common sense to decide which is better), they will be - automatically added to the pull request. + existing commit; use common sense to decide which is better), they will be + automatically added to the pull request. Do not include merge commits. + + Please note that it may take a while until a maintainer finds time to review + your pull request. Please refrain from pinging maintainers for a review after + a few days, unless the fix is critical. + + Maintainers may assign themselves to your pull request to simplify tracking + of pull requests to review, but you are still responsible for addressing review + comments. 7. If your pull request changes anything that users should be aware of (a bugfix, a new feature, ...) please add an entry as a new file in `changelog/unreleased` including the issue number in the filename (e.g. - `issue-8756`). Use the template in `changelog/TEMPLATE` for the content. + `pull-1234` or `issue-8756`). Use the template in `changelog/TEMPLATE` for the content. It will be used in the announcement of the next stable release. While writing, ask yourself: If I were the user, what would I need to be aware - of with this change? + of with this change? Documentation-only changes do not get a changelog entry. - 8. Do not edit the man pages under `doc/man` or `doc/manual_rest.rst` - - these are autogenerated before new releases. + 8. Do not edit the man pages under `doc/man` or the generated completion files + at `doc/*-completion.*` - these are autogenerated before new releases. 9. Once your code looks good and passes all the tests, we'll merge it. Thanks a lot for your contribution! @@ -190,17 +146,14 @@ The project is using the program [`golangci-lint`](https://github.com/golangci/golangci-lint) to run a list of linters and checkers. It will be run on the code when you submit a PR. In order to check your code beforehand, you can run `golangci-lint run` manually. -Eventually, we will enable `golangci-lint` for the whole code base. For now, -you can ignore warnings printed for lines you did not modify, those will be -ignored by the CI. For each pull request, several different systems run the integration tests on Linux, macOS and Windows. We won't merge any code that does not pass all tests -for all systems, so when a tests fails, try to find out what's wrong and fix +for all systems, so when a test fails, try to find out what's wrong and fix it. If you need help on this, please leave a comment in the pull request, and we'll be glad to assist. Having a PR with failing integration tests is nothing -to be ashamed of. In contrast, that happens regularly for all of us. That's -what the tests are there for. +to be ashamed of. That happens regularly for all of us. That's what the tests +are there for. More details of how to structure tests can be found here at [writing tests](https://restic.readthedocs.io/en/stable/090_participating.html#writing-tests). @@ -214,13 +167,16 @@ history and triaging bugs much easier. Git commit messages have a very terse summary in the first line of the commit message, followed by an empty line, followed by a more verbose description or a -List of changed things. For examples, please refer to the excellent [How to +list of changed things. For examples, please refer to the excellent [How to Write a Git Commit Message](https://chris.beams.io/posts/git-commit/). If you change/add multiple different things that aren't related at all, try to make several smaller commits. This is much easier to review. Using `git add -p` allows staging and committing only some changes. +Do not add merge commits in pull requests. Instead rebase your pull request +onto the master branch if there are merge conflicts. + Code Review =========== @@ -228,7 +184,7 @@ The restic project encourages actively reviewing the code, as it will store your precious data, so it's common practice to receive comments on provided patches. -If you are reviewing other contributor's code please consider the following +If you are reviewing another contributor's code please consider the following when reviewing: * Be nice. Please make the review comment as constructive as possible so all diff --git a/README.md b/README.md index 578b865b5..2ded67c80 100644 --- a/README.md +++ b/README.md @@ -91,17 +91,21 @@ reproduce a byte identical version from the source code for that release. Instructions on how to do that are contained in the [builder repository](https://github.com/restic/builder). -## News +# News You can follow the restic project on Mastodon [@restic](https://fosstodon.org/@restic) or subscribe to the [project blog](https://restic.net/blog/). -## License +# License Restic is licensed under [BSD 2-Clause License](https://opensource.org/licenses/BSD-2-Clause). You can find the complete text in [`LICENSE`](LICENSE). -## Sponsorship +# Contributing + +See our [contributing guide](CONTRIBUTING.md) and [documentation for restic development](doc/090_participating.rst). + +# Sponsorship Backend integration tests for Google Cloud Storage and Microsoft Azure Blob Storage are sponsored by [AppsCode](https://appscode.com)! diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 000000000..157a5d987 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,15 @@ +Security +======== + +**Important**: If you discover something that you believe to be a +possible critical security issue, please do *not* open a GitHub issue +but send an email directly to alexander@bumpern.de. If possible, please +encrypt your email using the following PGP key +([0x91A6868BD3F7A907](https://restic.net/gpg-key-alex.asc)): + +``` +pub 4096R/91A6868BD3F7A907 2014-11-01 + Key fingerprint = CF8F 18F2 8445 7597 3F79 D4E1 91A6 868B D3F7 A907 + uid Alexander Neumann + sub 4096R/D5FC2ACF4043FDF1 2014-11-01 +``` diff --git a/changelog/TEMPLATE b/changelog/TEMPLATE index f67d9fe00..c80fb9643 100644 --- a/changelog/TEMPLATE +++ b/changelog/TEMPLATE @@ -25,6 +25,8 @@ the default. # The first issue ID determines the filename for the changelog entry: # changelog/unreleased/issue-1234. If there are no relevant issue links, # use the PR ID and call the file pull-55555. +# +# Make sure to always include a link to the PR that introduces the change. https://github.com/restic/restic/issues/1234 https://github.com/restic/restic/pull/55555 diff --git a/doc/090_participating.rst b/doc/090_participating.rst index 82c079e45..50ae3d0ff 100644 --- a/doc/090_participating.rst +++ b/doc/090_participating.rst @@ -10,9 +10,65 @@ ^ for subsubsections " for paragraphs -############# -Participating -############# +########### +Development +########### + +This section covers how to develop, debug, profile and write tests for restic. For opening +issues and pull requests, see the next section. + +************ +Contributing +************ + +Contributions are welcome! More information can be +found in `CONTRIBUTING.md `__. + +A document describing the design of restic and the data structures stored on the +back end is contained in :ref:`repository-format`. + +If you'd like to start contributing to restic, but don't know exactly +what to do, have a look at this great article by Dave Cheney: +`Suggestions for contributing to an Open Source +project `__. + + +*********************** +Development Environment +*********************** + +The repository contains the code written for restic in the directories +``cmd/`` and ``internal/``. + +Make sure you have the `minimum required Go version installed +`__. Clone the repo +and ``cd`` into the directory: + +.. code-block:: console + + $ git clone https://github.com/restic/restic + $ cd restic + +Then use the ``go`` tool to build restic: + +.. code-block:: console + + $ go run build.go + $ ./restic version + restic 0.19.1-dev (compiled manually) compiled with go1.26.10 on linux/amd64 + +To create a debug build use: + +.. code-block:: console + + $ go run build.go -tags debug + +You can run all tests with the following command: + +.. code-block:: console + + $ go test ./... + ********** Debug Logs @@ -59,6 +115,21 @@ statements originating in functions that match the pattern ``*unlock*`` $ DEBUG_FUNCS=*unlock* restic check +*********** +Debug Hangs +*********** + +If restic gets stuck, you can generate a stacktrace as follows. +On non-Windows systems, you can send a ``SIGQUIT`` signal to restic or press +``Ctrl-\`` to achieve the same result. This causes restic to print a stacktrace +and then exit immediately. This will not damage your repository, however, +it might be necessary to manually clean up stale lock files using +``restic unlock``. + +On Windows, please run restic with the environment variable +``RESTIC_DEBUG_STACKTRACE_SIGINT`` set to ``true`` and press ``Ctrl-C`` to +create a stacktrace. + ********* Debugging @@ -73,37 +144,37 @@ The program can be built with debug support like this: This will make the ``restic debug `` available which can be used to inspect internal data structures. -In addition, this enables profiling flags such as ``--cpu-profile`` and -``--mem-profile`` which can help when investigating performance and memory usage -issues. See ``restic help`` for more details and a few additional -``--...-profile`` flags. -Running restic with profiling enabled generates a ``.pprof`` file such as -``cpu.pprof``. To view a profile in a web browser, first make sure that the -``dot`` command from `Graphviz `__ is in the PATH. Then, -run ``go tool pprof -http : cpu.pprof``. +*********************************** +Performance and Memory Usage Issues +*********************************** +Debug builds of restic support the ``--block-profile``, ``--cpu-profile``, +``--mem-profile``, and ``--trace-profile`` options which collect performance data +that later on can be analyzed using the go tools: -************ -Contributing -************ +.. code-block:: console -Contributions are welcome! Please **open an issue first** (or add a -comment to an existing issue) if you plan to work on any code or add a -new feature. This way, duplicate work is prevented and we can discuss -your ideas and design first. + $ restic --cpu-profile . [...] + $ go tool pprof -http localhost:12345 cpu.pprof -More information and a description of the development environment can be -found in `CONTRIBUTING.md `__. -A document describing the design of restic and the data structures stored on the -back end is contained in :ref:`repository-format`. +To analyze a trace profile use ``go tool trace -http=localhost:12345 trace.out``. -If you'd like to start contributing to restic, but don't know exactly -what to do, have a look at this great article by Dave Cheney: -`Suggestions for contributing to an Open Source -project `__. -A few issues have been tagged with the label ``help wanted``, you can -start looking at `those `_. +As the memory usage of restic changes over time, it may be useful to capture a +snapshot of the current heap. This is possible using the ``--listen-profile`` +option. Then while restic runs you can query and afterwards analyze the heap statistics. + +.. code-block:: console + + $ restic --listen-profile localhost:12345 [...] + $ curl http://localhost:12345/debug/pprof/heap -o heap.pprof + $ go tool pprof -http localhost:12345 heap.pprof + +The memory usage of restic primarily depends on the index size. To get an idea of the +number of tree and data blobs in the repository, you can use the ``restic stats --mode debug`` command. + +Setting ``GODEBUG=gctrace=1`` prints garbage-collector runs. For a graphical variant +combine this with `gcvis `__. ************* @@ -229,23 +300,6 @@ Example: this is a typical setup for a backup / find scenario rtest.Assert(t, len(lines) == 2, "expected one file, found (%v) in repo", len(lines)-1) -******** -Security -******** - -**Important**: If you discover something that you believe to be a -possible critical security problem, please do *not* open a GitHub issue -but send an email directly to alexander@bumpern.de. If possible, please -encrypt your email using the following PGP key -(`0x91A6868BD3F7A907 `__): - -:: - - pub 4096R/91A6868BD3F7A907 2014-11-01 - Key fingerprint = CF8F 18F2 8445 7597 3F79 D4E1 91A6 868B D3F7 A907 - uid Alexander Neumann - sub 4096R/D5FC2ACF4043FDF1 2014-11-01 - ************* Compatibility *************