feat(github-lfs): add GitHub authorization for LFS batch endpoint by Zherphy · Pull Request #83 · opensourceways/BigFiles

Zherphy · 2026-05-28T01:35:56Z

Add support for verifying GitHub user permissions on the LFS batch endpoint. A new GITHUB_MODEL config switch selects between the existing Gitee/GitCode authorization+metadata path and the GitHub authorization +metadata path within the same /objects/batch route.

Key changes:

auth: new GitHub auth module with org whitelist and permission check
config: new DefaultGithubToken and GithubModel fields
server: handleBatch dispatches to GitHub or Gitee auth/metadata based on GITHUB_MODEL; IsGithubAuthorized wired through Options
main: register GithubAuth() as IsGithubAuthorized
config.example.yml: document GIT_CODE_SWITCH and GITHUB_MODEL
ignore local coverage and test output artifacts

描述

变更类型

opensourceways-bot · 2026-05-28T01:36:00Z

Welcome To opensourceways Community

Hey @Zherphy , thanks for your contribution to the community.

Bot Usage Manual

I'm the Bot here serving you. You can find the instructions on how to interact with me at Here . That means you can comment below every pull request or issue to trigger Bot Commands.

Contact Guide

If you have any questions, please contact the SIG: infratructure ,
and any of the maintainers: @GeorgeCao-hw, @TangJia025, @pkking, @zhongjun2 ,
and any of the committers: @Goalina, @Zherphy, @tfhddd .

opensourceways-bot · 2026-05-28T01:36:06Z

CLA Signature Pass

Zherphy, thanks for your pull request. All authors of the commits have signed the CLA. 👍

opensourceways-bot · 2026-05-28T01:36:09Z

Linking Issue Notice

@Zherphy , the pull request must be linked to at least one issue.
If an issue has already been linked, but the needs-issue label remains, you can remove the label by commenting /check-issue .

gemini-code-assist

Code Review

This pull request upgrades the project to Go 1.24.0, introduces a new GitHub LFS Batch endpoint with a dedicated authentication module, fixes several security vulnerabilities (G706, G304), and establishes an AI Agent workflow infrastructure. The reviewer feedback identifies critical issues regarding error handling and classification: a double-write bug in the batch handler when metadata insertion fails, fragile prefix-based error matching for HTTP status codes, and redundant API calls when collaborator checks fail due to unauthorized tokens.

gemini-code-assist · 2026-05-28T01:38:14Z

+		if _, err = auth.CheckRepoOwner(userInRepo); req.Operation == "upload" || err != nil {
+			err := s.dealWithAuthError(userInRepo, w, r)
+			if err != nil {
+				return
+			}
+		}

-	resp := s.handleRequestObject(req)
+		resp = s.handleRequestObject(req)

-	// 添加元数据
-	addMetaData(req, w, userInRepo)
+		// 添加元数据
+		addMetaData(req, w, userInRepo)
+	}

 	must(json.NewEncoder(w).Encode(resp))
 }


If addMetaData fails, it writes a 500 Internal Server Error response to the client and returns. However, since addMetaData does not return an error, handleBatch continues execution and attempts to write a successful response via json.NewEncoder(w).Encode(resp). This results in a double-write bug, which corrupts the HTTP response. Refactoring addMetaData to return an error and handling it here prevents this issue.

if (_, err = auth.CheckRepoOwner(userInRepo); req.Operation == "upload" || err != nil { err := s.dealWithAuthError(userInRepo, w, r) if err != nil { return } } resp = s.handleRequestObject(req) // 添加元数据 if err := addMetaData(req, w, userInRepo); err != nil { return } } must(json.NewEncoder(w).Encode(resp)) }

gemini-code-assist · 2026-05-28T01:38:14Z

+	if err != nil {
+		v := err.Error()
+		w.Header().Set("LFS-Authenticate", `Basic realm="Git LFS"`)
+		switch {
+		case strings.HasPrefix(v, "unauthorized") || strings.HasPrefix(v, "not_found"):
+			w.WriteHeader(401)
+		case strings.HasPrefix(v, "forbidden"):
+			w.WriteHeader(403)
+		default:
+			w.WriteHeader(500)
+		}


The error message returned by getParsedResponse starts with the HTTP status (e.g., "401 Unauthorized: ..."), meaning strings.HasPrefix(v, "unauthorized") will evaluate to false. This causes unauthorized requests to fall through to the default case and return a 500 Internal Server Error instead of 401 Unauthorized. Using case-insensitive substring matching makes the error classification much more robust.

Suggested change

if err != nil {

v := err.Error()

w.Header().Set("LFS-Authenticate", `Basic realm="Git LFS"`)

switch {

case strings.HasPrefix(v, "unauthorized") || strings.HasPrefix(v, "not_found"):

w.WriteHeader(401)

case strings.HasPrefix(v, "forbidden"):

w.WriteHeader(403)

default:

w.WriteHeader(500)

}

if err != nil {

v := strings.ToLower(err.Error())

w.Header().Set("LFS-Authenticate", "Basic realm=\"Git LFS\"")

switch {

case strings.Contains(v, "unauthorized") || strings.Contains(v, "not_found") || strings.Contains(v, "401"):

w.WriteHeader(401)

case strings.Contains(v, "forbidden") || strings.Contains(v, "403"):

w.WriteHeader(403)

default:

w.WriteHeader(500)

}

gemini-code-assist · 2026-05-28T01:38:14Z

+func verifyGithubUpload(userInRepo UserInRepo, headers http.Header) error {
+	perm, err := getGithubCollaboratorPermission(userInRepo, headers)
+	if err != nil {
+		msg := err.Error() + ": verify github user permission failed"
+		logrus.Error(fmt.Sprintf(formatLogString, verifyLog, msg))
+		return errors.New(msg)
+	}


Prefix the returned error with "unauthorized: " if the collaborator permission check failed due to an unauthorized token. This ensures that dealWithGithubAuthError correctly classifies the error and returns a 401 Unauthorized status code instead of a 500 or 403.

func verifyGithubUpload(userInRepo UserInRepo, headers http.Header) error { perm, err := getGithubCollaboratorPermission(userInRepo, headers) if err != nil { prefix := "forbidden: " if strings.Contains(strings.ToLower(err.Error()), "unauthorized") { prefix = "unauthorized: " } msg := prefix + err.Error() + ": verify github user permission failed" logrus.Error(fmt.Sprintf(formatLogString, verifyLog, msg)) return errors.New(msg) }

gemini-code-assist · 2026-05-28T01:38:14Z

+func verifyGithubDownload(userInRepo UserInRepo, headers http.Header) error {
+	perm, err := getGithubCollaboratorPermission(userInRepo, headers)
+	if err != nil {
+		// collaborator API requires write/maintain/admin; for public repos it returns 404
+		// fall back to checking repo accessibility
+		path := fmt.Sprintf("https://api.github.com/repos/%s/%s",
+			userInRepo.Owner, userInRepo.Repo)
+		repo := new(githubRepo)
+		if repoErr := getParsedResponse("GET", path, headers, nil, repo); repoErr != nil {
+			// propagate unauthorized as-is so dealWithGithubAuthError returns 401
+			if strings.HasPrefix(repoErr.Error(), "unauthorized") {
+				return repoErr
+			}
+			msg := fmt.Sprintf("forbidden: user %s has no permission to download", userInRepo.Username)
+			logrus.Error(fmt.Sprintf(formatLogString, verifyLog, msg))
+			return errors.New(msg)
+		}
+		return nil
+	}


If the collaborator permission check fails with an unauthorized error (e.g., invalid or expired token), we should abort immediately instead of making a redundant fallback API call to check repository accessibility. Additionally, prefix the error with "unauthorized: " or "forbidden: " to ensure correct HTTP status routing.

func verifyGithubDownload(userInRepo UserInRepo, headers http.Header) error { perm, err := getGithubCollaboratorPermission(userInRepo, headers) if err != nil { if strings.Contains(strings.ToLower(err.Error()), "unauthorized") { return fmt.Errorf("unauthorized: %w", err) } // collaborator API requires write/maintain/admin; for public repos it returns 404 // fall back to checking repo accessibility path := fmt.Sprintf("https://api.github.com/repos/%s/%s", userInRepo.Owner, userInRepo.Repo) repo := new(githubRepo) if repoErr := getParsedResponse("GET", path, headers, nil, repo); repoErr != nil { prefix := "forbidden: " if strings.Contains(strings.ToLower(repoErr.Error()), "unauthorized") { prefix = "unauthorized: " } msg := prefix + fmt.Sprintf("user %s has no permission to download", userInRepo.Username) logrus.Error(fmt.Sprintf(formatLogString, verifyLog, msg)) return errors.New(msg) } return nil }

gemini-code-assist · 2026-05-28T01:38:14Z

+func verifyGithubDelete(userInRepo UserInRepo, headers http.Header) error {
+	perm, err := getGithubCollaboratorPermission(userInRepo, headers)
+	if err != nil {
+		msg := err.Error() + ": unauthorized: github token is invalid or expired, please re-authenticate"
+		return errors.New(msg)
+	}


Ensure the returned error is prefixed with "unauthorized: " or "forbidden: " depending on the collaborator permission check failure reason. This guarantees that dealWithGithubAuthError correctly returns a 401 or 403 status code.

func verifyGithubDelete(userInRepo UserInRepo, headers http.Header) error { perm, err := getGithubCollaboratorPermission(userInRepo, headers) if err != nil { prefix := "forbidden: " if strings.Contains(strings.ToLower(err.Error()), "unauthorized") { prefix = "unauthorized: " } msg := prefix + err.Error() + ": github token is invalid or expired, please re-authenticate" return errors.New(msg) }

opensourceways-bot · 2026-05-28T01:38:50Z

检查项	状态
敏感信息扫描	❌
安全编码扫描	❌
漏洞扫描	❌
Check代码检查	❌
开源license合规扫描	✅
UT测试覆盖率	❌
开发阶段设计文档检查	❌
流水线链接	点击跳转查看日志

opensourceways-bot · 2026-05-28T01:40:06Z

CLA Signature Pass

Zherphy, thanks for your pull request. All authors of the commits have signed the CLA. 👍

opensourceways-bot · 2026-05-28T01:43:29Z

检查项	状态
敏感信息扫描	❌
安全编码扫描	❌
漏洞扫描	❌
Check代码检查	❌
开源license合规扫描	✅
UT测试覆盖率	❌
开发阶段设计文档检查	❌
流水线链接	点击跳转查看日志

opensourceways-bot · 2026-05-28T02:04:01Z

CLA Signature Pass

Zherphy, thanks for your pull request. All authors of the commits have signed the CLA. 👍

opensourceways-bot · 2026-05-28T02:07:13Z

检查项	状态
敏感信息扫描	❌
安全编码扫描	❌
漏洞扫描	❌
Check代码检查	❌
开源license合规扫描	✅
UT测试覆盖率	❌
开发阶段设计文档检查	❌
流水线链接	点击跳转查看日志

Add support for verifying GitHub user permissions on the LFS batch endpoint. A new GITHUB_MODEL config switch selects between the existing Gitee/GitCode authorization+metadata path and the GitHub authorization +metadata path within the same /objects/batch route. Key changes: - auth: new GitHub auth module with org whitelist and permission check - config: new DefaultGithubToken and GithubModel fields - server: handleBatch dispatches to GitHub or Gitee auth/metadata based on GITHUB_MODEL; IsGithubAuthorized wired through Options - main: register GithubAuth() as IsGithubAuthorized - config.example.yml: document GIT_CODE_SWITCH and GITHUB_MODEL - ignore local coverage and test output artifacts

opensourceways-bot · 2026-06-03T00:32:25Z

CLA Signature Pass

Zherphy, thanks for your pull request. All authors of the commits have signed the CLA. 👍

opensourceways-bot · 2026-06-03T00:36:29Z

检查项	状态
敏感信息扫描	❌
安全编码扫描	❌
漏洞扫描	❌
Check代码检查	❌
开源license合规扫描	✅
UT测试覆盖率	❌
开发阶段设计文档检查	❌
流水线链接	点击跳转查看日志

opensourceways-bot added sig/infratructure opensourceways-cla/yes labels May 28, 2026

opensourceways-bot added the needs-issue label May 28, 2026

gemini-code-assist Bot reviewed May 28, 2026

View reviewed changes

Zherphy force-pushed the feature/add-github-lfs branch from 64f207b to a5d4f57 Compare May 28, 2026 01:39

Zherphy force-pushed the feature/add-github-lfs branch from a5d4f57 to a6ea029 Compare May 28, 2026 02:03

Zherphy force-pushed the feature/add-github-lfs branch from a6ea029 to 44db7ce Compare June 3, 2026 00:32

Uh oh!

Conversation

Zherphy commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

描述

相关 Issue

变更类型

Uh oh!

opensourceways-bot commented May 28, 2026

Welcome To opensourceways Community

Bot Usage Manual

Contact Guide

Uh oh!

opensourceways-bot commented May 28, 2026

CLA Signature Pass

Uh oh!

opensourceways-bot commented May 28, 2026

Linking Issue Notice

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

opensourceways-bot commented May 28, 2026

Uh oh!

opensourceways-bot commented May 28, 2026

CLA Signature Pass

Uh oh!

opensourceways-bot commented May 28, 2026

Uh oh!

opensourceways-bot commented May 28, 2026

CLA Signature Pass

Uh oh!

opensourceways-bot commented May 28, 2026

Uh oh!

opensourceways-bot commented Jun 3, 2026

CLA Signature Pass

Uh oh!

opensourceways-bot commented Jun 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Zherphy commented May 28, 2026 •

edited

Loading