Ensure _git_event_log is initialized before use in _LogMessage. This
avoids crashes when _git_event_log is accessed before it's set, such as
during repo info.
Bug: 435317391
Change-Id: I3adc32d6a9377558e852bbb43f9cf82041fcf1bc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498521
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
When checkout errors occurred in interleaved sync, they were wrapped in
a SyncError with no message, causing blank lines in the final summary.
Refactor _SyncResult to hold a list of exceptions, ensuring the original
error messages are propagated correctly.
Bug: 438178765
Change-Id: Ic25e515068959829cb6290cfd9e4c2d3963bbbea
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498342
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Failures in deferred sync actions were not recorded because `_Later.Run`
discarded the `GitError` exception. Record the specific error using
`syncbuf.fail()` and propagate it for proper error aggregation and
reporting.
Bug: 438178765
Change-Id: Iad59e389f9677bd6b8d873ee1ea2aa6ce44c86fa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498141
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
A race condition occurs when sync redirects sys.stderr to capture worker output, while a background progress thread simultaneously calls fileno() on it. This causes an io.UnsupportedOperation error. Fix by caching the original sys.stderr for all progress bar IO.
Change-Id: Idb1f45d707596d31238a19fd373cac3bf669c405
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Keep track of finished projects, not just successful ones, when deciding
which projects still need to be synced. Also project errors are already
reported by sync workers so stall detection doesn't need to add failed
projects to the error list.
Bug: 438178765
Change-Id: Ibf15aad009ba7295e70c8df2ff158215085e9732
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498062
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
_ProcessSyncInterleavedResults currently only shows stderr_text if
verbose. Show it if a sync worker fails, regardless of verbosity.
Bug: 438178765
Change-Id: If24dcb10fb5d6857386782d371e3f9c6844dece9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498061
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
The _CheckForImmutableRevision method was modified in commit 0e776a58 to
include upstream branch validation for superproject scenarios. However,
this change inadvertently broke shallow clones when both clone-depth and
upstream attributes are specified in regular (non-superproject)
manifests.
Issue: When upstream is present, _CheckForImmutableRevision performs two
additional checks: 1. git rev-list on the upstream reference 2. git
merge-base --is-ancestor between revision and upstream
In shallow clones, the upstream branch history may not be available
locally, causing these checks to fail. This triggers the retry mechanism
that removes depth limitations, effectively converting shallow clones to
full clones, resulting in excessive disk usage.
Fix: Make upstream validation conditional on superproject usage. This
preserves the original superproject fix while restoring the method's
original behavior for regular scenarios - checking only if the immutable
revision (SHA1/tag) exists locally.
Note: The SetRevisionId method from the same commit 0e776a58 is left
unchanged as it only stores upstream information (no git operations),
which is beneficial for preserving branch context for commands like
'repo start' without causing fetch-related issues.
The fix ensures that manifests with both clone-depth and upstream work
correctly in non-superproject scenarios, maintaining shallow clone
efficiency and reducing disk usage.
Bug: b/427093249
Change-Id: I00acd4c61b179cd2abf796c2fecb7a2f38016a18
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/493883
Tested-by: Krzysztof Wesolowski <krzysztof.wesolowski@volvocars.com>
Commit-Queue: Krzysztof Wesolowski <krzysztof.wesolowski@volvocars.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Kamaljeet Maini <kamaljeet@google.com>
Reviewed-by: Xin Li <delphij@google.com>
git rev-parse fails on invalid HEAD, e.g. after incomplete sync, causing
NoManifestException. Fall back to v2.56's direct file reading when
rev-parse fails.
Bug: 435045466
Change-Id: Ia14560335110c00d80408b2a93595a84446f8a57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/495181
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Skip hook if its project is not present on disk.
Bug: 434232630
Change-Id: I09a8b412d078af7a068d533f7be320d5b02327be
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/494441
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
The switch to git rev-parse caused GetHead() to return the literal
string 'HEAD' when in a detached state. This broke repo prune, which
expects a commit SHA.
Bug: 434077990
Change-Id: I80b7d5965749096b59e854f61e913aa74c857b99
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/494401
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Interleaved sync should not try checkout out a project if it's a mirror.
Change-Id: I2549faab197a3202d79a10e44b449b68d53e3fe7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/492942
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
The previous default, "phased" sync (separate network and checkout
phases), can now be selected with `--no-interleaved`.
Bug: 421935613
Bug: 432082000
Change-Id: Ia8624daa609a28ea2f87f8ea4b42138d8b3e9269
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/489681
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This fixes two issues:
1. the progress bar could show a count greater than the total if new projects were discovered mid-sync. Update the progress bar total dynamically
2. Make "Stall detected" error message more actionable
Bug: 432206932
Change-Id: Ie2a4ada5b1770cae0302fb06590641c522cbb7e7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/491941
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This fixes the double "done" text on successful interleaved sync.
Bug: 421935613
Change-Id: I4f01418cb0340129a8f0a2a5835f7e3fa6a6b119
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/487081
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Add support for a new hook type "post-sync" declared in the manifest using
<repo-hooks>. This allows executing a script automatically after a successful
`repo sync`.
This is useful for initializing developer environments, installing project-wide
Git hooks, generating configs, and other post-sync automation tasks.
Example manifest usage:
<project name="myorg/repo-hooks" path="hooks" revision="main" />
<repo-hooks in-project="myorg/repo-hooks" enabled-list="post-sync">
<hook name="post-sync" />
</repo-hooks>
The hook script must be named `post-sync.py` and located at the root of the
hook project.
The post-sync hook does not block `repo sync`; if the script fails, the sync
still completes successfully with a warning.
Test: Added `post-sync.py` in hook project and verified it runs after `repo sync`
Bug: b/421694721
Change-Id: I69f3158f0fc319d73a85028d6e90fea02c1dc8c8
Signed-off-by: Kenny Cheng <chao.shun.cheng.tw@gmail.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/480581
Reviewed-by: Scott Lee <ddoman@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Convenient way to always enable or disable git-lfs without having to
remember to put on the command line.
Useful if you want to ALWAYS have git-lfs enabled on your system when
you 'init' a new project.
Also useful if you are using the Jenkins repo plugin as it doesn't
provide an option for enabling git-lfs in its UI.
Change-Id: Ieb1bbe83de9c21523ab69b30fc5047c257d02731
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437661
Commit-Queue: Scott Lee <ddoman@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Fatahillah Wk <fatahillahwkwk@gmail.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Matt Moeller <moeller.matt@gmail.com>
Reviewed-by: Yingchun Li <sword.l.dragon@gmail.com>
Dedupe error reporting logic for phased and interleaved sync modes by
extracting it into _ReportErrors.
Error reporting will now distinguish between network and local failures
and lists the specific repos that failed in each phase.
Bug: 421935613
Change-Id: I4604a83943dbbd71d979158d7a1c4b8c243347d2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/484541
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Extract the manifest update loop from _SyncPhased into a new
_UpdateManifestLists method and use it in both sync types.
Bug: 421935613
Change-Id: If499a3ce4a0bbb3c4641dba52ca5c1c82b11f16f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/484341
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
The logic for checking for repo self-updates lives in _FetchMain, which
is part of the "phased" sync path.
Extract this logic into a new _UpdateRepoProject helper method. Call
this common helper from _ExecuteHelper before either sync mode begins,
so the repo self-update check is always performed.
Bug: 421935613
Change-Id: I9a804f43fbf6239c4146be446040be531f12fc8a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/484041
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
--jobs-network and --jobs-checkout are ignored with --interleaved.
Bug: 421935613
Change-Id: Ib69413993c4f970b385bd09318972716e5ac3324
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/485021
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
For each assigned project, the worker sequentially calls
Sync_NetworkHalf and Sync_LocalHalf, respecting --local-only and
--network-only flags. To prevent scrambled progress bars, all stderr
output from the checkout phase is captured (shown with --verbose).
Result objects now carry status and timing information from the worker
for state updates.
Bug: 421935613
Change-Id: I398602e08a375e974a8914e5fa48ffae673dda9b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/483301
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Introduce the parallel orchestration framework for `repo sync
--interleaved`.
The new logic respects project dependencies by processing them in
hierarchical levels. Projects sharing a git object directory are grouped
and processed serially. Also reuse the familiar fetch progress bar UX.
Bug: 421935613
Change-Id: Ia388a231fa96b3220e343f952f07021bc9817d19
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/483281
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Upload gets a FileNotFoundError if not using superproject because it
tries to access the superproject's repo_id before checking if
superproject was actually enabled.
Reorder the logic to check use_superproject first.
Change-Id: I65cd2adab481e799dd7bb75e1a83553ad6e34d8d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/484401
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Prepare for an interleaved fetch and checkout mode for `repo sync`. The
goal of the new mode is to significantly speed up syncs by running fetch
and checkout operations in parallel for different projects, rather than
waiting for all fetches to complete before starting any checkouts.
Bug: 421935613
Change-Id: I8c66d1e790c7bba6280e409b95238c5e4e61a9c8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/482821
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Bug: 416589884
Change-Id: Icbaade585932f0cbb51367e07925ef606f089697
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/482762
Commit-Queue: Scott Lee <ddoman@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Lint: Scott Lee <ddoman@google.com>
Tested-by: Scott Lee <ddoman@google.com>
Add an internal check to make sure we always follow the API we've
documented for external authors. Since the internal call is a bit
ad-hoc, it can be easy to miss a call site.
Change-Id: Ie8cd298d1fc34f10f3c5eb353512a3e881f42252
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/481721
Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
It was released in python 3.8, and repo still supports 3.6.
Bug: 422226033
Change-Id: I6bdd2cdbb074766ecfb1492d842c847781c4b264
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/481201
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Bug: 416589884
Change-Id: I5d1c709518d76d777a7f07c4c774569773c5a265
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/478205
Lint: Scott Lee <ddoman@google.com>
Tested-by: Scott Lee <ddoman@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Scott Lee <ddoman@google.com>
Add a test to enforce this too.
Change-Id: I80b5cf567aa33db9c24b53428c66d69f9c1d8d74
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/478481
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
When uploading, add the root superproject repo as a push option in the
format `-o custom-keyed-value=rootRepo:$HOST/$PROJECT`.
Bug: b/401147338
Change-Id: I00230256eb7ae307b03840bb4090c28dc8a0505e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/472601
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This will make it easier to add more formats without exploding the
common --xxx space and checking a large set of boolean flags.
Also fill out the test coverage while we're here.
Bug: b/412725063
Change-Id: I754013dc6cb3445f8a0979cefec599d55dafdcff
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/471941
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Make sure we run this for local devs.
Change-Id: I472b7c347086d54649dd9d5778eea4737447b353
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/471921
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
People often forget to regen when making interface changes.
We skip the test if help2man isn't installed since it's not common,
and it's not available on our CI bots currently.
Change-Id: Ib4911a0e3fa1294ad90e4ac8afc047a0b7c2b66d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/469741
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
.venv is by convention a very common place for venvs and
is the default in some tools, for example like "Astral uv".
The third-party packages installed there should not be linted.
Change-Id: I3278d90c2fdfc8a34a2488e82d4df8e836111ce1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/469941
Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This change should be a noop from a functional point of view.
Exceptions inheriting directly from BaseException (KeyboardInterrupt,
SystemExit) are not caught by "except Exception", they will instead
continue raising upwards the stack, so there is no need to explicitly
catch and re-raise them.
Change-Id: Ic10764af4a6c05d1162f8b21651e7864ed742286
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/469601
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com>
In sha1 mode, when depth is enabled, syncing the revision from
upstream may not work because some servers only allow fetching
named refs. Fetching a specific sha1 may result in an error like
'server does not allow request for unadvertised object'. In this
case, attempt a full sync with depth disabled.
Bug: 410825502
Change-Id: If51bcf18b877cd9491706f5bc3d6fd13c0c3d4f3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/468282
Commit-Queue: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Some users are reporting a "curl: (26) .netrc error: no such file"
message on sync caused by an change to curl behavior.
See https://github.com/curl/curl/issues/16163.
Use --netrc-optional which was introduced in curl version 7.9.8
released in 2002.
Bug: 409354839
Change-Id: I8365c6e806968a4ee765a7e023b4bced30489c20
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/467026
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Minor fix, but just in case, provides properly quoted commands for
people to copy & paste.
Change-Id: Ia9fce5c0df9f51cbed9d49861adcf6821251e46f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/466821
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Warn users if the effective job count specified via `-j`,
`--jobs-network`, or `--jobs-checkout` exceeds a threshold
(currently 100). This encourages users to use more reasonable
values.
Bug: 406868778
Bug: 254914814
Change-Id: I116e2bbaf3dc824c04d1b2fbe52cf9ca5be77b9a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/466801
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
If you try running this from a subdir, then most of the tests fail
because they assume they're running from the top of the source tree.
Change all the tests to actually run there.
For example: cd docs && ../run_tests
Change-Id: I92e17476393a108e56b58e049193b9fd72c5b7ba
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/464841
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
repo attempts to create /etc/.repo_gitconfig.json file, and fails if
root file system is mounted read-only. Removing non-existing file on
read-only filesystem results in EROFS instead of ENOENT.
Bug: 401018409
Change-Id: I64edc0567fb88649f3fd8cacb65a8780744640d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/458821
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Egor Duda <egor.duda@gmail.com>
Commit-Queue: Egor Duda <egor.duda@gmail.com>
We use global when we need to write to a variable, not read it.
This function only reads, so drop the keyword.
Change-Id: Iee91998fba67fd3e8ebaf2f4a79f95032f70b1c0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/464501
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Since we require Python 3.6 now in the launcher, swap out our custom
RunError class for the standard subprocess one.
Change-Id: Id0ca17c40e22ece03e06366a263ad340963f979d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/464401
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
