superproject: Don't exit if superproject tag doesn't exist in manifest.
Don't exit if there are missing commit ids in superproject.
This change implements the following suggestion from delphij@:
"we should note the event (so we know that --use-superproject but there
were some errors, e.g. manifest didn't specify commit id for some
reason, or if there is no superproject but --use-superproject is
used), print out a message telling the use that this is not support,
but continue as if --no-use-superproject was specified?"
Changes:
superproject:
+ Added git_trace2_event_log as an argument to the constructor.
+ Sync method returns SyncResult a NamedTuple of
++ success - True if sync of superproject is successful, or False.
++ fatal - True if caller should exit, Or False.
+ UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of
++ manifest_path - path name of the overriding manifest file instead
of None
++ fatal - True if caller should exit, Or False
+ _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of
++ commit_ids - a dictionary with the projects/commit ids on success,
otherwise None
++ fatal - True if caller should exit, Or False
+ Added _SkipUpdatingProjectRevisionId a helper function to see if a
project's revision id needs to be updated or not. This function is
used to exclude projects from local manifest file.
+ Added the following error events into git_trace2_event_log
++ If superproject is missing in a manifest
++ If there are missing commit ids for projects.
command.py:
+ Deleted unused import - platform
+ Added git_trace2_event_log as a member so all subcmds can log error
events.
main.py:
+ Initialized git_trace2_event_log as a member of command object.
init.py:
+ Deleted unused import - optparse
init.py:
+ Called sys.exit only if Sync returns exit=True
sync.py:
+ Called sys.exit only if Superproject's UpdateProjectsRevisionId returns
exit=True
+ Reloaded the manifest only if manifest path is returned by
UpdateProjectsRevisionId. If not, fall back to the old way of doing
repo sync.
test_git_superproject:
+ Added code to verify error events are being logged.
+ Added a test for no superproject tag
+ Added test for UpdateProjectsRevisionId not updating the revision id
with the commit ids.
Tested the code with the following commands.
+ Positive test case with aosp-master.
$ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
.../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
Your identity is: Raman Tenneti <rtenneti@google.com>
If you want to change this, please re-run 'repo init' with --config-name
repo has been initialized in .../android/aosp
$ repo_dev sync -j40 --use-superproject
remote: Total 12 (delta 4), reused 12 (delta 4)
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
.../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
...
repo sync has finished successfully.
+ Negative test case without superproject tag.
$ repo_dev sync -j40 --use-superproject
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml
error: Cannot get project commit ids from manifest
error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
...
Checking out: 100% (1022/1022), done in 3.589s
repo sync has finished successfully.
+ Test for missing commit_id for a project.
$ repo_dev sync -j40 --use-superproject
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
.../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint']
error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
...
Checking out: 100% (1022/1022), done in 3.364s
repo sync has finished successfully.
$ ./run_tests -v
...
...== 164 passed in 2.87s ==...
Bug: [google internal] b/189371541
Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
This commit is contained in:
Raman Tenneti
parent
b8c84483a5
commit
784e16f3aa
6 changed files with 225 additions and 61 deletions
|
|
@ -19,12 +19,13 @@ https://en.wikibooks.org/wiki/Git/Submodules_and_Superprojects
|
|||
|
||||
Examples:
|
||||
superproject = Superproject()
|
||||
project_commit_ids = superproject.UpdateProjectsRevisionId(projects)
|
||||
UpdateProjectsResult = superproject.UpdateProjectsRevisionId(projects)
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import sys
|
||||
from typing import NamedTuple
|
||||
|
||||
from git_command import git_require, GitCommand
|
||||
from git_refs import R_HEADS
|
||||
|
|
@ -34,6 +35,33 @@ _SUPERPROJECT_GIT_NAME = 'superproject.git'
|
|||
_SUPERPROJECT_MANIFEST_NAME = 'superproject_override.xml'
|
||||
|
||||
|
||||
class SyncResult(NamedTuple):
|
||||
"""Return the status of sync and whether caller should exit."""
|
||||
|
||||
# Whether the superproject sync was successful.
|
||||
success: bool
|
||||
# Whether the caller should exit.
|
||||
fatal: bool
|
||||
|
||||
|
||||
class CommitIdsResult(NamedTuple):
|
||||
"""Return the commit ids and whether caller should exit."""
|
||||
|
||||
# A dictionary with the projects/commit ids on success, otherwise None.
|
||||
commit_ids: dict
|
||||
# Whether the caller should exit.
|
||||
fatal: bool
|
||||
|
||||
|
||||
class UpdateProjectsResult(NamedTuple):
|
||||
"""Return the overriding manifest file and whether caller should exit."""
|
||||
|
||||
# Path name of the overriding manfiest file if successful, otherwise None.
|
||||
manifest_path: str
|
||||
# Whether the caller should exit.
|
||||
fatal: bool
|
||||
|
||||
|
||||
class Superproject(object):
|
||||
"""Get commit ids from superproject.
|
||||
|
||||
|
|
@ -41,19 +69,21 @@ class Superproject(object):
|
|||
lookup of commit ids for all projects. It contains _project_commit_ids which
|
||||
is a dictionary with project/commit id entries.
|
||||
"""
|
||||
def __init__(self, manifest, repodir, superproject_dir='exp-superproject',
|
||||
quiet=False):
|
||||
def __init__(self, manifest, repodir, git_event_log,
|
||||
superproject_dir='exp-superproject', quiet=False):
|
||||
"""Initializes superproject.
|
||||
|
||||
Args:
|
||||
manifest: A Manifest object that is to be written to a file.
|
||||
repodir: Path to the .repo/ dir for holding all internal checkout state.
|
||||
It must be in the top directory of the repo client checkout.
|
||||
git_event_log: A git trace2 event log to log events.
|
||||
superproject_dir: Relative path under |repodir| to checkout superproject.
|
||||
quiet: If True then only print the progress messages.
|
||||
"""
|
||||
self._project_commit_ids = None
|
||||
self._manifest = manifest
|
||||
self._git_event_log = git_event_log
|
||||
self._quiet = quiet
|
||||
self._branch = self._GetBranch()
|
||||
self._repodir = os.path.abspath(repodir)
|
||||
|
|
@ -172,44 +202,48 @@ class Superproject(object):
|
|||
"""Gets a local copy of a superproject for the manifest.
|
||||
|
||||
Returns:
|
||||
True if sync of superproject is successful, or False.
|
||||
SyncResult
|
||||
"""
|
||||
print('NOTICE: --use-superproject is in beta; report any issues to the '
|
||||
'address described in `repo version`', file=sys.stderr)
|
||||
|
||||
if not self._manifest.superproject:
|
||||
print('error: superproject tag is not defined in manifest',
|
||||
file=sys.stderr)
|
||||
return False
|
||||
msg = (f'repo error: superproject tag is not defined in manifest: '
|
||||
f'{self._manifest.manifestFile}')
|
||||
print(msg, file=sys.stderr)
|
||||
self._git_event_log.ErrorEvent(msg, '')
|
||||
return SyncResult(False, False)
|
||||
|
||||
should_exit = True
|
||||
url = self._manifest.superproject['remote'].url
|
||||
if not url:
|
||||
print('error: superproject URL is not defined in manifest',
|
||||
file=sys.stderr)
|
||||
return False
|
||||
return SyncResult(False, should_exit)
|
||||
|
||||
if not self._Init():
|
||||
return False
|
||||
return SyncResult(False, should_exit)
|
||||
if not self._Fetch(url):
|
||||
return False
|
||||
return SyncResult(False, should_exit)
|
||||
if not self._quiet:
|
||||
print('%s: Initial setup for superproject completed.' % self._work_git)
|
||||
return True
|
||||
return SyncResult(True, False)
|
||||
|
||||
def _GetAllProjectsCommitIds(self):
|
||||
"""Get commit ids for all projects from superproject and save them in _project_commit_ids.
|
||||
|
||||
Returns:
|
||||
A dictionary with the projects/commit ids on success, otherwise None.
|
||||
CommitIdsResult
|
||||
"""
|
||||
if not self.Sync():
|
||||
return None
|
||||
sync_result = self.Sync()
|
||||
if not sync_result.success:
|
||||
return CommitIdsResult(None, sync_result.fatal)
|
||||
|
||||
data = self._LsTree()
|
||||
if not data:
|
||||
print('error: git ls-tree failed to return data for superproject',
|
||||
file=sys.stderr)
|
||||
return None
|
||||
return CommitIdsResult(None, True)
|
||||
|
||||
# Parse lines like the following to select lines starting with '160000' and
|
||||
# build a dictionary with project path (last element) and its commit id (3rd element).
|
||||
|
|
@ -225,7 +259,7 @@ class Superproject(object):
|
|||
commit_ids[ls_data[3]] = ls_data[2]
|
||||
|
||||
self._project_commit_ids = commit_ids
|
||||
return commit_ids
|
||||
return CommitIdsResult(commit_ids, False)
|
||||
|
||||
def _WriteManfiestFile(self):
|
||||
"""Writes manifest to a file.
|
||||
|
|
@ -250,6 +284,23 @@ class Superproject(object):
|
|||
return None
|
||||
return manifest_path
|
||||
|
||||
def _SkipUpdatingProjectRevisionId(self, project):
|
||||
"""Checks if a project's revision id needs to be updated or not.
|
||||
|
||||
Revision id for projects from local manifest will not be updated.
|
||||
|
||||
Args:
|
||||
project: project whose revision id is being updated.
|
||||
|
||||
Returns:
|
||||
True if a project's revision id should not be updated, or False,
|
||||
"""
|
||||
path = project.relpath
|
||||
if not path:
|
||||
return True
|
||||
# Skip the project if it comes from the local manifest.
|
||||
return any(s.startswith(LOCAL_MANIFEST_GROUP_PREFIX) for s in project.groups)
|
||||
|
||||
def UpdateProjectsRevisionId(self, projects):
|
||||
"""Update revisionId of every project in projects with the commit id.
|
||||
|
||||
|
|
@ -257,30 +308,35 @@ class Superproject(object):
|
|||
projects: List of projects whose revisionId needs to be updated.
|
||||
|
||||
Returns:
|
||||
manifest_path: Path name of the overriding manfiest file instead of None.
|
||||
UpdateProjectsResult
|
||||
"""
|
||||
commit_ids = self._GetAllProjectsCommitIds()
|
||||
commit_ids_result = self._GetAllProjectsCommitIds()
|
||||
commit_ids = commit_ids_result.commit_ids
|
||||
if not commit_ids:
|
||||
print('error: Cannot get project commit ids from manifest', file=sys.stderr)
|
||||
return None
|
||||
return UpdateProjectsResult(None, commit_ids_result.fatal)
|
||||
|
||||
projects_missing_commit_ids = []
|
||||
for project in projects:
|
||||
if self._SkipUpdatingProjectRevisionId(project):
|
||||
continue
|
||||
path = project.relpath
|
||||
if not path:
|
||||
continue
|
||||
# Skip the project if it comes from local manifest.
|
||||
if any(s.startswith(LOCAL_MANIFEST_GROUP_PREFIX) for s in project.groups):
|
||||
continue
|
||||
commit_id = commit_ids.get(path)
|
||||
if commit_id:
|
||||
project.SetRevisionId(commit_id)
|
||||
else:
|
||||
if not commit_id:
|
||||
projects_missing_commit_ids.append(path)
|
||||
|
||||
# If superproject doesn't have a commit id for a project, then report an
|
||||
# error event and continue as if do not use superproject is specified.
|
||||
if projects_missing_commit_ids:
|
||||
print('error: please file a bug using %s to report missing commit_ids for: %s' %
|
||||
(self._manifest.contactinfo.bugurl, projects_missing_commit_ids), file=sys.stderr)
|
||||
return None
|
||||
msg = (f'error: please file a bug using {self._manifest.contactinfo.bugurl} '
|
||||
f'to report missing commit_ids for: {projects_missing_commit_ids}')
|
||||
print(msg, file=sys.stderr)
|
||||
self._git_event_log.ErrorEvent(msg, '')
|
||||
return UpdateProjectsResult(None, False)
|
||||
|
||||
for project in projects:
|
||||
if not self._SkipUpdatingProjectRevisionId(project):
|
||||
project.SetRevisionId(commit_ids.get(project.relpath))
|
||||
|
||||
manifest_path = self._WriteManfiestFile()
|
||||
return manifest_path
|
||||
return UpdateProjectsResult(manifest_path, False)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue