Commit Graph

559 Commits (main)

Author SHA1 Message Date
Rowan Bohde a03fdd9566
Avoid MacOS keychain dialog in integration tests (#32813)
Mac's git installation ships with a system wide config that configures
the credential helper `osxkeychain`, which will prompt the user with a
dialog.

```
$ git config list --system 
credential.helper=osxkeychain
```
By setting the environment variable
[`GIT_CONFIG_NOSYSTEM=true`](https://git-scm.com/docs/git-config#ENVIRONMENT),
Git will not load the system wide config, preventing the dialog from
populating.

Closes #26717
2 months ago
wxiaoguang 22bf2ca6ba
Make API "compare" accept commit IDs (#32801) 2 months ago
Kemal Zebari 01b1896bf5
Implement update branch API (#32433)
Resolves #22526.

Builds upon #23061.

---------

Co-authored-by: sillyguodong <33891828+sillyguodong@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
Sebastian T. T. 874b8484aa
Add standard-compliant route to serve outdated R packages (#32783)
The R package repository currently does not have support for older
versions of packages which should be stored in a separate /Archive
router. This PR remedies that by adding a new path router.

I am a member of a group that loves using Gitea and this bug has been
annoying us for a long time. Hope it can be merged in time for Gitea
1.23.0.

Any feedback much appreciated.

Fixes #32782
2 months ago
wxiaoguang 4814f43af7
Fix repo home file list (#32788)
1. use grid instead of table, completely drop "ui table" from that list
2. move some "commit sign" related styles into a new file by the way (no
change) because I need to figure out where `#repo-files-table` is used.
3. move legacy "branch/tag selector" related code into repo-legacy.ts,
now there are 13 `import $` files left.
2 months ago
Blender Defender 18061af490
Rearrange Clone Panel (#31142)
Rearrange the clone panel to use less horizontal space.
The following changes have been made to achieve this:
- Moved everything into the dropdown menu
- Moved the HTTPS/SSH Switch to a separate line
- Moved the "Clone in VS Code"-Button up and added a divider
- Named the dropdown button "Code", added appropriate icon

---------

Co-authored-by: techknowlogick <techknowlogick@gitea.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
Lunny Xiao fbe6d9dc6b
Use batch database operations instead of one by one to optimze api pulls (#32680)
Resolve #31492

The response time for the Pull Requests API has improved significantly,
dropping from over `2000ms` to about `350ms` on my local machine. It's
about `6` times faster.

A key area for further optimization lies in batch-fetching data for
`apiPullRequest.ChangedFiles, apiPullRequest.Additions, and
apiPullRequest.Deletions`.

Tests `TestAPIViewPulls` does exist and new tests added.

- This PR also fixes some bugs in `GetDiff` functions.
- This PR also fixes data inconsistent in test data. For a pull request,
the head branch's reference should be equal to the reference in
`pull/xxx/head`.
2 months ago
Lunny Xiao b32f0cdfa0
Make wiki pages visit fast (#32732) 2 months ago
Illya Marchenko 936665bf85
Issue time estimate, meaningful time tracking (#23113)
Redesign the time tracker side bar, and add "time estimate" support (in "1d 2m" format)

Closes #23112

---------

Co-authored-by: stuzer05 <stuzer05@gmail.com>
Co-authored-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
KN4CK3R 0c3c041c88
Add Arch package registry (#32692)
Close #25037
Close #31037

This PR adds a Arch package registry usable with pacman.

![grafik](https://github.com/user-attachments/assets/81cdb0c2-02f9-4733-bee2-e48af6b45224)

Rewrite of #25396 and #31037. You can follow [this
tutorial](https://wiki.archlinux.org/title/Creating_packages) to build a
package for testing.

Docs PR: https://gitea.com/gitea/docs/pulls/111

Co-authored-by: [d1nch8g@ion.lc](mailto:d1nch8g@ion.lc)
Co-authored-by: @ExplodingDragon

---------

Co-authored-by: dancheg97 <dancheg97@fmnx.su>
Co-authored-by: dragon <ExplodingFKL@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
Lunny Xiao 17053e953f
Fix delete branch perm checking (#32654) 2 months ago
KN4CK3R 136408307c
Add Swift login endpoint (#32693)
Fix #32683

This PR adds the login endpoint and fixes the documentation links.
2 months ago
wxiaoguang def13ece7c
Allow to disable the password-based login (sign-in) form (#32687)
Usually enterprise/organization users would like to only allow OAuth2
login.

This PR adds a new config option to disable the password-based login
form. It is a simple and clear approach and won't block the future
login-system refactoring works.

Fix a TODO in #24821

Replace  #21851

Close #7633 , close #13606
2 months ago
Lunny Xiao 79d593a9be
Split mail sender sub package from mailer service package (#32618)
Move all mail sender related codes into a sub package of
services/mailer. Just move, no code change.
Then we just have dependencies on go-mail package in the new sub
package. We can use other package to replace it because it's
unmaintainable. ref #18664
2 months ago
Zettat123 fd3aa5bedb
Fix a bug in actions artifact test (#32672)
This bug exists in `TestActionsArtifactDownload`.


a1f56f83bf/tests/integration/api_actions_artifact_test.go (L123-L134)

We assert that `listResp.Count` is `2`, so `artifactIdx` could be `0` or `1`.

a1f56f83bf/tests/integration/api_actions_artifact_test.go (L144-L147)

Then we assert that the length of `downloadResp.Value` is `1`. If
`artifactIdx` is `1` at this point, the assertion on Line 147 will throw
an `index out of range` error.
2 months ago
Rowan Bohde 16a7d343d7
Validate OAuth Redirect URIs (#32643)
This fixes a TODO in the code to validate the RedirectURIs when adding
or editing an OAuth application in user settings.

This also includes a refactor of the user settings tests to only create
the DB once per top-level test to avoid reloading fixtures.
2 months ago
Lunny Xiao f62f68cbdd
Move team related functions to service layer (#32537)
There are still some functions under `models` after last big refactor
about `models`. This change will move all team related functions to
service layer with no code change.
2 months ago
6543 846f618716
Add priority to protected branch (#32286)
## Solves

Currently for rules to re-order them you have to alter the creation
date. so you basicly have to delete and recreate them in the right
order. This is more than just inconvinient ...

## Solution

Add a new col for prioritization

## Demo WebUI Video

https://github.com/user-attachments/assets/92182a31-9705-4ac5-b6e3-9bb74108cbd1


---
*Sponsored by Kithara Software GmbH*
2 months ago
wxiaoguang b6ce2d6dc9
Refactor markup render system (#32645)
This PR mainly removes some global variables, moves some code and
renames some functions to make code clearer.

This PR also removes a testing-only option ForceHardLineBreak during
refactoring since the behavior is clear now.
2 months ago
wxiaoguang 9ed768adc4
Improve oauth2 scope token handling (#32633) 2 months ago
Kerwin Bryant 25cacaf0aa
Fixed Issue of Review Menu Shown Behind (#32631)
Fixed #31144 

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
Lunny Xiao 703be6bf30
Add github compatible tarball download API endpoints (#32572)
Fix #29654 
Fix #32481
2 months ago
wxiaoguang 5d57c287fb
Fix sqlite3 test (#32622) 2 months ago
wxiaoguang 633785a5f3
Refactor markup render system (#32612)
This PR removes (almost) all path tricks, and introduces "renderhelper"
package.

Now we can clearly see the rendering behaviors for comment/file/wiki,
more details are in "renderhelper" tests.

Fix #31411 , fix #18592, fix #25632 and maybe more problems. (ps: fix
#32608 by the way)
2 months ago
Lunny Xiao fe49cb0243
Fix get reviewers' bug (#32415)
This PR rewrites `GetReviewer` function and move it to service layer.

Reviewers should not be watchers, so that this PR removed all watchers
from reviewers. When the repository is under an organization, the pull
request unit read permission will be checked to resolve the bug of
#32394

Fix #32394
2 months ago
wxiaoguang c4e27cb27b
Refactor markup render system (#32589)
This PR mainly moves some code and introduces `RenderContext.WithXxx`
functions
2 months ago
Marcell Mars a3881ffa3d
Enhancing Gitea OAuth2 Provider with Granular Scopes for Resource Access (#32573)
Resolve #31609

This PR was initiated following my personal research to find the
lightest possible Single Sign-On solution for self-hosted setups. The
existing solutions often seemed too enterprise-oriented, involving many
moving parts and services, demanding significant resources while
promising planetary-scale capabilities. Others were adequate in
supporting basic OAuth2 flows but lacked proper user management
features, such as a change password UI.

Gitea hits the sweet spot for me, provided it supports more granular
access permissions for resources under users who accept the OAuth2
application.

This PR aims to introduce granularity in handling user resources as
nonintrusively and simply as possible. It allows third parties to inform
users about their intent to not ask for the full access and instead
request a specific, reduced scope. If the provided scopes are **only**
the typical ones for OIDC/OAuth2—`openid`, `profile`, `email`, and
`groups`—everything remains unchanged (currently full access to user's
resources). Additionally, this PR supports processing scopes already
introduced with [personal
tokens](https://docs.gitea.com/development/oauth2-provider#scopes) (e.g.
`read:user`, `write:issue`, `read:group`, `write:repository`...)

Personal tokens define scopes around specific resources: user info,
repositories, issues, packages, organizations, notifications,
miscellaneous, admin, and activitypub, with access delineated by read
and/or write permissions.

The initial case I wanted to address was to have Gitea act as an OAuth2
Identity Provider. To achieve that, with this PR, I would only add
`openid public-only` to provide access token to the third party to
authenticate the Gitea's user but no further access to the API and users
resources.

Another example: if a third party wanted to interact solely with Issues,
it would need to add `read:user` (for authorization) and
`read:issue`/`write:issue` to manage Issues.

My approach is based on my understanding of how scopes can be utilized,
supported by examples like [Sample Use Cases: Scopes and
Claims](https://auth0.com/docs/get-started/apis/scopes/sample-use-cases-scopes-and-claims)
on auth0.com.

I renamed `CheckOAuthAccessToken` to `GetOAuthAccessTokenScopeAndUserID`
so now it returns AccessTokenScope and user's ID. In the case of
additional scopes in `userIDFromToken` the default `all` would be
reduced to whatever was asked via those scopes. The main difference is
the opportunity to reduce the permissions from `all`, as is currently
the case, to what is provided by the additional scopes described above.

Screenshots:

![Screenshot_20241121_121405](https://github.com/user-attachments/assets/29deaed7-4333-4b02-8898-b822e6f2463e)

![Screenshot_20241121_120211](https://github.com/user-attachments/assets/7a4a4ef7-409c-4116-9d5f-2fe00eb37167)

![Screenshot_20241121_120119](https://github.com/user-attachments/assets/aa52c1a2-212d-4e64-bcdf-7122cee49eb6)

![Screenshot_20241121_120018](https://github.com/user-attachments/assets/9eac318c-e381-4ea9-9e2c-3a3f60319e47)
---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
Lunny Xiao a175f9805c
Add line-through for deleted branch on pull request view page (#32500)
Base branch deleted

![图片](https://github.com/user-attachments/assets/bc45aa33-d514-47c8-885a-de9732f2f3d5)

branch deleted comment

![图片](https://github.com/user-attachments/assets/83729bbb-2ee8-4bd3-b6f1-780d2daad3d4)

Head branch deleted

![图片](https://github.com/user-attachments/assets/90120b22-34a9-4387-aae9-5c882e8d421a)
2 months ago
Rowan Bohde 9ac74a1a40
disable gravatar in test (#32529)
When running e2e tests on flaky networks, gravatar can cause a timeout
and test failures. Turn off, and populate avatars on e2e test suite run
to make them reliable.
2 months ago
Lunny Xiao 696fbe6036
Refactor push mirror find and add check for updating push mirror (#32539)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
Lunny Xiao 4f879a00df
Refactor find forks and fix possible bugs that weak permissions check (#32528)
- Move models/GetForks to services/FindForks
- Add doer as a parameter of FindForks to check permissions
- Slight performance optimization for get forks API with batch loading
of repository units
- Add tests for forking repository to organizations

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2 months ago
Lunny Xiao c3dedcffa7
Fix basic auth with webauthn (#32531) 3 months ago
wxiaoguang ecbb03dc6d
Improve testing and try to fix MySQL hanging (#32515)
By some CI fine tunes (`run tests`), SQLite & MSSQL could complete
in about 12~13 minutes (before > 14), MySQL could complete in 18 minutes
(before: about 23 or even > 30)

Major changes:

1. use tmpfs for MySQL storage
1. run `make test-mysql` instead of `make integration-test-coverage`
because the code coverage is not really used at the moment.
1. refactor testlogger to make it more reliable and be able to report
stuck stacktrace
1. do not requeue failed items when a queue is being flushed (failed
items would keep failing and make flush uncompleted)
1. reduce the file sizes for testing
1. use math ChaCha20 random data instead of crypot/rand (for testing
purpose only)
1. no need to `DeleteRepository` in `TestLinguist`
1. other related refactoring to make code easier to maintain
3 months ago
Rowan Bohde 68731c07c5
Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:

* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
3 months ago
wxiaoguang 3f9c3e7bc3
Refactor render system (#32492)
There were too many patches to the Render system, it's really difficult
to make further improvements.

This PR clears the legacy problems and fix TODOs.

1. Rename `RenderContext.Type` to `RenderContext.MarkupType` to clarify
its usage.
2. Use `ContentMode` to replace `meta["mode"]` and `IsWiki`, to clarify
the rendering behaviors.
3. Use "wiki" mode instead of "mode=gfm + wiki=true"
4. Merge `renderByType` and `renderByFile`
5. Add more comments

----

The problem of "mode=document": in many cases it is not set, so many
non-comment places use comment's hard line break incorrectly
3 months ago
wxiaoguang 0aedb03996
Fix LFS route mock, realm, middleware names (#32488)
1. move "internal-lfs" route mock to "common-lfs"
2. fine tune tests
3. fix "realm" strings, according to RFC:
https://datatracker.ietf.org/doc/html/rfc2617:
    * realm       = "realm" "=" realm-value
    * realm-value = quoted-string
4. clarify some names of the middlewares, rename `ignXxx` to `optXxx` to
match `reqXxx`, and rename ambiguous `requireSignIn` to `reqGitSignIn`
3 months ago
wxiaoguang 160ccb5ee2
Fix test fixtures for user2/lfs.git (#32477) 3 months ago
wxiaoguang 580e21dd2e
Refactor LFS SSH and internal routers (#32473)
Gitea instance keeps reporting a lot of errors like "LFS SSH transfer connection denied, pure SSH protocol is disabled". When starting debugging the problem, there are more problems found. Try to address most of them:

* avoid unnecessary server side error logs (change `fail()` to not log them)
* figure out the broken tests/user2/lfs.git (added comments)
* avoid `migratePushMirrors` failure when a repository doesn't exist (ignore them)
* avoid "Authorization" (internal&lfs) header conflicts, remove the tricky "swapAuth" and use "X-Gitea-Internal-Auth"
* make internal token comparing constant time (it wasn't a serous problem because in a real world it's nearly impossible to timing-attack the token, but good to fix and backport)
* avoid duplicate routers (introduce AddOwnerRepoGitLFSRoutes)
* avoid "internal (private)" routes using session/web context (they should use private context)
* fix incorrect "path" usages (use "filepath")
* fix incorrect mocked route point handling (need to check func nil correctly)
* split some tests from "git general tests" to "git misc tests" (to keep "git_general_test.go" simple)

Still no correct result for Git LFS SSH tests. So the code is kept there
(`tests/integration/git_lfs_ssh_test.go`) and a FIXME explains the details.
3 months ago
Calvin K 18aeca5320
Add reviewers selection to new pull request (#32403)
Users could add reviewers when creating new PRs.

---------

Co-authored-by: splitt3r <splitt3r@users.noreply.github.com>
Co-authored-by: Sebastian Sauer <sauer.sebastian@gmail.com>
Co-authored-by: bb-ben <70356237+bboerben@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
3 months ago
Zettat123 35bcd667b2
Fix broken releases when re-pushing tags (#32435)
Fix #32427
3 months ago
Kemal Zebari 7adc4717ec
Include file extension checks in attachment API (#32151)
From testing, I found that issue posters and users with repository write
access are able to edit attachment names in a way that circumvents the
instance-level file extension restrictions using the edit attachment
APIs. This snapshot adds checks for these endpoints.
3 months ago
Bruno Sofiato f64fbd9b74
Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search. 

Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).

In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.

Resolves #32220

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
3 months ago
Lunny Xiao 24b83ff63e
Fix milestone deadline and date related problems (#32339)
Use zero instead of 9999-12-31 for deadline
Fix #32291

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
3 months ago
wxiaoguang 61c35590c7
Refactor RepoRefByType (#32413)
1. clarify the "filepath" could(should) contain "{ref}"
2. remove unclear RepoRefLegacy and RepoRefAny, use RepoRefUnknown to guess
3. by the way, avoid using AppURL
3 months ago
wxiaoguang ec2d1593c2
Refactor tests to prevent from unnecessary preparations (#32398) 3 months ago
Kyle D. 66971e591e
Add artifacts test fixture (#30300)
Closes https://github.com/go-gitea/gitea/issues/30296

- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
3 months ago
wxiaoguang a920fcfd91
Fix db engine (#32351)
Fix #32349
3 months ago
Tim de2ad2e1b1
Make admins adhere to branch protection rules (#32248)
This introduces a new flag `BlockAdminMergeOverride` on the branch
protection rules that prevents admins/repo owners from bypassing branch
protection rules and merging without approvals or failing status checks.

Fixes #17131

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
3 months ago
wxiaoguang afa8dd45af
Make git push options accept short name (#32245)
Just like what most CLI parsers do: `--opt` means `opt=true`

Then users could use `-o force-push` as `-o force-push=true`
4 months ago
Bruno Sofiato 900ac62251
Allow code search by filename (#32210)
This is a large and complex PR, so let me explain in detail its changes.

First, I had to create new index mappings for Bleve and ElasticSerach as
the current ones do not support search by filename. This requires Gitea
to recreate the code search indexes (I do not know if this is a breaking
change, but I feel it deserves a heads-up).

I've used [this
approach](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/analysis-pathhierarchy-tokenizer.html)
to model the filename index. It allows us to efficiently search for both
the full path and the name of a file. Bleve, however, does not support
this out-of-box, so I had to code a brand new [token
filter](https://blevesearch.com/docs/Token-Filters/) to generate the
search terms.

I also did an overhaul in the `indexer_test.go` file. It now asserts the
order of the expected results (this is important since matches based on
the name of a file are more relevant than those based on its content).
I've added new test scenarios that deal with searching by filename. They
use a new repo included in the Gitea fixture.

The screenshot below depicts how Gitea shows the search results. It
shows results based on content in the same way as the current version
does. In matches based on the filename, the first seven lines of the
file contents are shown (BTW, this is how GitHub does it).


![image](https://github.com/user-attachments/assets/9d938d86-1a8d-4f89-8644-1921a473e858)

Resolves #32096

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
4 months ago
Zettat123 0fe5e2b08c
Allow maintainers to view and edit files of private repos when "Allow maintainers to edit" is enabled (#32215)
Fix #31539
4 months ago
wxiaoguang 6029d78ab5
Improve the maintainblity of the reserved username list (#32229) 4 months ago
Lunny Xiao c2217670dd
Move admin routers from /admin to /-/admin (#32189)
Resolve #32181

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
4 months ago
wxiaoguang dd83cfcacc
Refactor CSRF token (#32216) 4 months ago
Lunny Xiao d6d3c96e65
Fix bug when a token is given public only (#32204) 4 months ago
yp05327 5d6d025c9b
Add support for searching users by email (#30908)
Fix #30898

we have an option `SearchByEmail`, so enable it, then we can search user
by email.
Also added a test for it.
4 months ago
Job 7e68bc8823
Fix PR creation on forked repositories (#31863)
Resolves #20475
4 months ago
sommerf-lf 0b1b4030e6
add {{TEST_MINIO_ENDPOINT}} for local testing "with/without" docker + fix pgsql testing doc (#32105)
while testing i found out that testing locally as documented in the
changed README.md for pgsql isn't working because of the minio
dependency. reworked this to by default be still docker, but allow for
for local with only minio in docker and testing on bare metal.
also depending on this: fixed docs for running pgsql test

Closes: #32168 (by changing documentation for pgsql tests)
Closes: #32169 (by changing documentation, Makefile & pgsql.ini.tmpl:
adding {{TEST_MINIO_ENDPOINT}})

sry for the combined pr, but when testing I ran into this issue and
first thought they were related and now finally address the same
problem: not beeing able to run pgsql integration tests as described in
the according README.md
4 months ago
Lunny Xiao 3a4a1bffbe
Make oauth2 code clear. Move oauth2 provider code to their own packages/files (#32148)
Fix #30266
Replace #31533
4 months ago
yp05327 70b7df0e5e
Support repo license (#24872)
Close #278
Close #24076

## Solutions:
- Use
[google/licenseclassifier](https://github.com/google/licenseclassifier/)
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[go-license-detector](https://github.com/go-enry/go-license-detector):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[google/licenseclassifier](https://github.com/google/licenseclassifier/):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1576092178
- Generate License Convert Name List to avoid import license templates
with same contents
Gitea automatically get latest license data from[
spdx/license-list-data](https://github.com/spdx/license-list-data).
But unfortunately, some license templates have same contents. #20915
[click here to see the
list](https://github.com/go-gitea/gitea/pull/24872#issuecomment-1584141684)
So we will generate a list of these license templates with same contents
and create a new file to save the result when using `make
generate-license`. (Need to decide the save path)
- Save License info into a new table `repo_license`
Can easily support searching repo by license in the future.

## Screen shot
Single License:

![image](https://github.com/go-gitea/gitea/assets/18380374/41260bd7-0b4c-4038-8592-508706cffa9f)

Multiple Licenses:

![image](https://github.com/go-gitea/gitea/assets/18380374/34ce2f73-7e18-446b-9b96-ecc4fb61bd70)

Triggers:
- [x] Push commit to default branch
- [x] Create repo
- [x] Mirror repo
- [x] When Default Branch is changed, licenses should be updated

Todo:
- [x] Save Licenses info in to DB when there's a change to license file
in the commit
- [x] DB Migration
- [x] A nominal test?
- [x] Select which library to
use(https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167)
- [x] API Support
- [x] Add repo license table
- ~Select license in settings if there are several licenses(Not
recommended)~
- License board(later, not in this PR)

![image](https://github.com/go-gitea/gitea/assets/18380374/2c3c3bf8-bcc2-4c6d-8ce0-81d1a9733878)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.com>
4 months ago
Zettat123 f4b8f6fc40
Fix the logic of finding the latest pull review commit ID (#32139)
Fix #31423
4 months ago
Jason Song 13283873e9
Ensure `GetCSRF` doesn't return an empty token (#32130)
Since page templates keep changing, some pages that contained forms with
CSRF token no longer have them.

It leads to some calls of `GetCSRF` returning an empty string, which
fails the tests. Like

3269b04d61/tests/integration/attachment_test.go (L62-L63)

The test did try to get the CSRF token and provided it, but it was
empty.
4 months ago
Zettat123 fcedf634d5
Fix bug in getting merged pull request by commit (#32079) 4 months ago
Lunny Xiao 9e0db1b646
Fix rename branch permission bug (#32066)
The previous implementation requires admin permission to rename branches
which should be write permission.

Fix #31993
4 months ago
ChristopherHX b594cec2bd
Fix artifact v4 upload above 8MB (#31664)
Multiple chunks are uploaded with type "block" without using
"appendBlock" and eventually out of order for bigger uploads.
8MB seems to be the chunk size

This change parses the blockList uploaded after all blocks to get the
final artifact size and order them correctly before calculating the
sha256 checksum over all blocks

Fixes #31354
4 months ago
Jamie Schouten d351a42494
Add bin to Composer Metadata (#32099)
This PR addresses the missing `bin` field in Composer metadata, which
currently causes vendor-provided binaries to not be symlinked to
`vendor/bin` during installation.

In the current implementation, running `composer install` does not
publish the binaries, leading to issues where expected binaries are not
available.

By properly declaring the `bin` field, this PR ensures that binaries are
correctly symlinked upon installation, as described in the [Composer
documentation](https://getcomposer.org/doc/articles/vendor-binaries.md).
4 months ago
Lunny Xiao 5de41737a5
Fix upload maven pacakge parallelly (#31851)
Use globallock for maven package uploads.

Thanks @tlusser for the test code.

Depends on ~#31813~
4 months ago
KN4CK3R 08adbc468f
Fix incorrect `/tokens` api (#32085)
Fixes #32078

- Add missing scopes output.
- Disallow empty scope.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
4 months ago
Exploding Dragon 269c630923
Fix: database not update release when using `git push --tags --force` (#32040)
link: https://codeberg.org/forgejo/forgejo/issues/4274

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
4 months ago
wxiaoguang 1fede04b83
Refactor CSRF protector (#32057)
Remove unused CSRF options, decouple "new csrf protector" and "prepare"
logic, do not redirect to home page if CSRF validation falis (it
shouldn't happen in daily usage, if it happens, redirecting to home
doesn't help either but just makes the problem more complex for "fetch")
5 months ago
KN4CK3R 55f1fcf0ad
Add missing comment reply handling (#32050)
Fixes #31937

- Add missing comment reply handling
- Use `onGiteaRun` in the test because the fixtures are not present
otherwise (did this behaviour change?)

Compare without whitespaces.
5 months ago
Kemal Zebari f05d9c98c4
Handle invalid target when creating releases using API (#31841)
A 500 status code was thrown when passing a non-existent target to the
create release API. This snapshot handles this error and instead throws
a 404 status code.

Discovered while working on #31840.
5 months ago
techknowlogick d9a7748cdc
bump to go 1.23 (#31855) 5 months ago
Wim 01dec7577a
Remove SHA1 for support for ssh rsa signing (#31857)
https://github.com/go-fed/httpsig seems to be unmaintained. 

Switch to github.com/42wim/httpsig which has removed deprecated crypto
and default sha256 signing for ssh rsa.

No impact for those that use ed25519 ssh certificates.

This is a breaking change for:
- gitea.com/gitea/tea (go-sdk) - I'll be sending a PR there too
- activitypub using deprecated crypto (is this actually used?)
5 months ago
Lunny Xiao 5c05dddbed
Fix nuget/conan/container packages upload bugs (#31967) 5 months ago
Lunny Xiao ac34449747
Prevent update pull refs manually and will not affect other refs update (#31931)
All refs under `refs/pull` should only be changed from Gitea inside but
not by pushing from outside of Gitea.
This PR will prevent the pull refs update but allow other refs to be
updated on the same pushing with `--mirror` operations.

The main changes are to add checks on `update` hook but not
`pre-receive` because `update` will be invoked by every ref but
`pre-receive` will revert all changes once one ref update fails.
5 months ago
Lunny Xiao 8b92eba21f
Fix agit automerge (#31207) 5 months ago
Lunny Xiao 4f5c96627b
Fix actions notify bug (#31866)
Try to fix
https://github.com/go-gitea/gitea/issues/31757#issuecomment-2295131062
5 months ago
Rowan Bohde 1310649331
render plain text file if the LFS object doesn't exist (#31812)
We had an issue where a repo was using LFS to store a file, but the user
did not push the file. When trying to view the file, Gitea returned a
500 HTTP status code referencing `ErrLFSObjectNotExist`. It appears the
intent was the render this file as plain text, but the conditional was
flipped. I've also added a test to verify that the file is rendered as
plain text.
6 months ago
Jason Song 0470646d46
Show lock owner instead of repo owner on LFS setting page (#31788)
Fix #31784.

Before:

<img width="1648" alt="image"
src="https://github.com/user-attachments/assets/03f32545-4a85-42ed-bafc-2b193a5d8023">

After:

<img width="1653" alt="image"
src="https://github.com/user-attachments/assets/e5bcaf93-49cb-421f-aac1-5122bc488b02">
6 months ago
Jason Song 33cc5837a6
Support compression for Actions logs (#31761)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015

The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.

That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.

This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.

And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.

`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.

<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
6 months ago
Exploding Dragon de175e3b06
Add signature support for the RPM module (#27069)
close  #27031

If the rpm package does not contain a matching gpg signature, the
installation will fail. See (#27031) , now auto-signing rpm uploads.

This option is turned off by default for compatibility.
6 months ago
Jason Song a33e74d40d
Clarify Actions resources ownership (#31724)
Fix #31707.

Also related to #31715.

Some Actions resources could has different types of ownership. It could
be:

- global: all repos and orgs/users can use it.
- org/user level: only the org/user can use it.
- repo level: only the repo can use it.

There are two ways to distinguish org/user level from repo level:
1. `{owner_id: 1, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.
2. `{owner_id: 0, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.

The first way seems more reasonable, but it may not be true. The point
is that although a resource, like a runner, belongs to a repo (it can be
used by the repo), the runner doesn't belong to the repo's org (other
repos in the same org cannot use the runner). So, the second method
makes more sense.

And the first way is not user-friendly to query, we must set the repo id
to zero to avoid wrong results.

So, #31715 should be right. And the most simple way to fix #31707 is
just:

```diff
-	shared.GetRegistrationToken(ctx, ctx.Repo.Repository.OwnerID, ctx.Repo.Repository.ID)
+	shared.GetRegistrationToken(ctx, 0, ctx.Repo.Repository.ID)
```

However, it is quite intuitive to set both owner id and repo id since
the repo belongs to the owner. So I prefer to be compatible with it. If
we get both owner id and repo id not zero when creating or finding, it's
very clear that the caller want one with repo level, but set owner id
accidentally. So it's OK to accept it but fix the owner id to zero.
6 months ago
yp05327 e0a408e6f3
Add permission check when creating PR (#31033)
user should be a collaborator of the base repo to create a PR
6 months ago
Shivaram Lingamneni ecc8f2b047
add `username` to OIDC introspection response (#31688)
This field is specified as optional here:
https://datatracker.ietf.org/doc/html/rfc7662#section-2.2

It's used by some OIDC integrations, e.g.
https://emersion.fr/blog/2022/irc-and-oauth2/

Co-authored-by: Giteabot <teabot@gitea.io>
6 months ago
Shivaram Lingamneni 2f1cb1d289
fix OIDC introspection authentication (#31632)
See discussion on #31561 for some background.

The introspect endpoint was using the OIDC token itself for
authentication. This fixes it to use basic authentication with the
client ID and secret instead:

* Applications with a valid client ID and secret should be able to
  successfully introspect an invalid token, receiving a 200 response
  with JSON data that indicates the token is invalid
* Requests with an invalid client ID and secret should not be able
  to introspect, even if the token itself is valid

Unlike #31561 (which just future-proofed the current behavior against
future changes to `DISABLE_QUERY_AUTH_TOKEN`), this is a potential
compatibility break (some introspection requests without valid client
IDs that would previously succeed will now fail). Affected deployments
must begin sending a valid HTTP basic authentication header with their
introspection requests, with the username set to a valid client ID and
the password set to the corresponding client secret.
6 months ago
Rowan Bohde 416c36f303
allow synchronizing user status from OAuth2 login providers (#31572)
This leverages the existing `sync_external_users` cron job to
synchronize the `IsActive` flag on users who use an OAuth2 provider set
to synchronize. This synchronization is done by checking for expired
access tokens, and using the stored refresh token to request a new
access token. If the response back from the OAuth2 provider is the
`invalid_grant` error code, the user is marked as inactive. However, the
user is able to reactivate their account by logging in the web browser
through their OAuth2 flow.

Also changed to support this is that a linked `ExternalLoginUser` is
always created upon a login or signup via OAuth2.

### Notes on updating permissions
Ideally, we would also refresh permissions from the configured OAuth
provider (e.g., admin, restricted and group mappings) to match the
implementation of LDAP. However, the OAuth library used for this `goth`,
doesn't seem to support issuing a session via refresh tokens. The
interface provides a [`RefreshToken`
method](https://github.com/markbates/goth/blob/master/provider.go#L20),
but the returned `oauth.Token` doesn't implement the `goth.Session` we
would need to call `FetchUser`. Due to specific implementations, we
would need to build a compatibility function for every provider, since
they cast to concrete types (e.g.
[Azure](https://github.com/markbates/goth/blob/master/providers/azureadv2/azureadv2.go#L132))

---------

Co-authored-by: Kyle D <kdumontnu@gmail.com>
7 months ago
Rowan Bohde 1ee59f0fa3
Allow disabling authentication related user features (#31535)
We have some instances that only allow using an external authentication
source for authentication. In this case, users changing their email,
password, or linked OpenID connections will not have any effect, and
we'd like to prevent showing that to them to prevent confusion.

Included in this are several changes to support this:
* A new setting to disable user managed authentication credentials
(email, password & OpenID connections)
* A new setting to disable user managed MFA (2FA codes & WebAuthn)
* Fix an issue where some templates had separate logic for determining
if a feature was disabled since it didn't check the globally disabled
features
* Hide more user setting pages in the navbar when their settings aren't
enabled

---------

Co-authored-by: Kyle D <kdumontnu@gmail.com>
7 months ago
Henry Goodman 12cb1d2998
Allow force push to protected branches (#28086)
Fixes #22722 

### Problem
Currently, it is not possible to force push to a branch with branch
protection rules in place. There are often times where this is necessary
(CI workflows/administrative tasks etc).

The current workaround is to rename/remove the branch protection,
perform the force push, and then reinstate the protections.

### Solution
Provide an additional section in the branch protection rules to allow
users to specify which users with push access can also force push to the
branch. The default value of the rule will be set to `Disabled`, and the
UI is intuitive and very similar to the `Push` section.

It is worth noting in this implementation that allowing force push does
not override regular push access, and both will need to be enabled for a
user to force push.

This applies to manual force push to a remote, and also in Gitea UI
updating a PR by rebase (which requires force push)

This modifies the `BranchProtection` API structs to add:
- `enable_force_push bool`
- `enable_force_push_whitelist bool`
- `force_push_whitelist_usernames string[]`
- `force_push_whitelist_teams string[]`
- `force_push_whitelist_deploy_keys bool`

### Updated Branch Protection UI:

<img width="943" alt="image"
src="https://github.com/go-gitea/gitea/assets/79623665/7491899c-d816-45d5-be84-8512abd156bf">

### Pull Request `Update branch by Rebase` option enabled with source
branch `test` being a protected branch:


![image](https://github.com/go-gitea/gitea/assets/79623665/e018e6e9-b7b2-4bd3-808e-4947d7da35cc)
<img width="1038" alt="image"
src="https://github.com/go-gitea/gitea/assets/79623665/57ead13e-9006-459f-b83c-7079e6f4c654">

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
7 months ago
silverwind 2c92c7c522
Add typescript guideline and typescript-specific eslint plugins and fix issues (#31521)
1. Add some general guidelines how to write our typescript code
2. Add `@typescript-eslint/eslint-plugin`, general typescript rules
3. Add `eslint-plugin-deprecation` to detect deprecated code
4. Fix all new lint issues that came up
7 months ago
silverwind 08579d6cbb
Add initial typescript config and use it for eslint,vitest,playwright (#31186)
This enables eslint to use the typescript parser and resolver which
brings some benefits that eslint rules now have type information
available and a tsconfig.json is required for the upcoming typescript
migration as well. Notable changes done:

- Add typescript parser and resolver
- Move the vue-specific config into the root file
- Enable `vue-scoped-css/enforce-style-type` rule, there was only one
violation and I added a inline disable there.
- Fix new lint errors that were detected because of the parser change
- Update `i/no-unresolved` to remove now-unnecessary workaround for the
resolver
- Disable `i/no-named-as-default` as it seems to raise bogus issues in
the webpack config
- Change vitest config to typescript
- Change playwright config to typescript
- Add `eslint-plugin-playwright` and fix issues
- Add `tsc` linting to `make lint-js`
7 months ago
wxiaoguang f4921b9daa
Simplify 404/500 page (#31409) 7 months ago
wxiaoguang 43c7a2e7b1
Refactor names (#31405)
This PR only does "renaming":

* `Route` should be `Router` (and chi router is also called "router")
* `Params` should be `PathParam` (to distingush it from URL query param, and to match `FormString`)
* Use lower case for private functions to avoid exposing or abusing
8 months ago
wxiaoguang d32648b204
Refactor route path normalization (#31381)
Refactor route path normalization and decouple it from the chi router.
Fix the TODO, fix the legacy strange path behavior.
8 months ago
wxiaoguang 0f09c22663
Improve rubygems package registry (#31357)
To make it work with Bundler:
https://guides.rubygems.org/rubygems-org-compact-index-api/

It only adds 2 new API endpoints and improves some tests, existing logic
is not changed.
8 months ago
Zoupers Zou e25d6960b5
Fix #31185 try fix lfs download from bitbucket failed (#31201)
Fix #31185
8 months ago
wxiaoguang 6106a61eff
Remove sub-path from container registry realm (#31293)
Container registry requires that the "/v2" must be in the root, so the
sub-path in AppURL should be removed
8 months ago
silverwind 816222243a
Add `lint-go-gopls` (#30729)
Uses `gopls check <files>` as a linter. Tested locally and brings up 149
errors currently for me. I don't think I want to fix them in this PR,
but I would like at least to get this analysis running on CI.

List of errors:
```
modules/indexer/code/indexer.go:181:11: impossible condition: nil != nil
routers/private/hook_post_receive.go:120:15: tautological condition: nil == nil
services/auth/source/oauth2/providers.go:185:9: tautological condition: nil == nil
services/convert/issue.go:216:11: tautological condition: non-nil != nil
tests/integration/git_test.go:332:9: impossible condition: nil != nil
services/migrations/migrate.go:179:24-43: unused parameter: ctx
services/repository/transfer.go:288:48-69: unused parameter: doer
tests/integration/api_repo_tags_test.go:75:41-61: unused parameter: session
tests/integration/git_test.go:696:64-74: unused parameter: baseBranch
tests/integration/gpg_git_test.go:265:27-39: unused parameter: t
tests/integration/gpg_git_test.go:284:23-29: unused parameter: tmpDir
tests/integration/gpg_git_test.go:284:31-35: unused parameter: name
tests/integration/gpg_git_test.go:284:37-42: unused parameter: email
```
8 months ago
Thomas Desveaux c888c933a9
Fix NuGet Package API for $filter with Id equality (#31188)
Fixes issue when running `choco info pkgname` where `pkgname` is also a
substring of another package Id.

Relates to #31168

---

This might fix the issue linked, but I'd like to test it with more choco
commands before closing the issue in case I find other problems if
that's ok.

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
8 months ago
Lunny Xiao fb7b743bd0
Azure blob storage support (#30995)
This PR implemented object storages(LFS/Packages/Attachments and etc.)
for Azure Blob Storage. It depends on azure official golang SDK and can
support both the azure blob storage cloud service and azurite mock
server.

Replace #25458
Fix #22527 

- [x] CI Tests
- [x] integration test, MSSQL integration tests will now based on
azureblob
  - [x] unit test 
- [x] CLI Migrate Storage
- [x] Documentation for configuration added

------

TODO (other PRs):
- [ ] Improve performance of `blob download`.

---------

Co-authored-by: yp05327 <576951401@qq.com>
8 months ago