Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unresolvable names from buildreq_cache #563

Closed
wants to merge 3 commits into from
Closed

Remove unresolvable names from buildreq_cache #563

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bfb2c9c
Add helper functions to remove deps from global sets
phmccarty Jan 11, 2020
7eb5926
Start using the new dep removal helper functions
phmccarty Jan 11, 2020
182dc05
Remove unresolvable names from buildreq_cache
phmccarty Jan 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion autospec/build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,10 @@ def parse_buildroot_log(filename, returncode):
for line in loglines:
match = missing_pat.match(line)
if match is not None:
util.print_fatal("Cannot resolve dependency name: {}".format(match.group(1)))
name = match.group(1)
util.print_fatal("Cannot resolve dependency name: {}".format(name))
# Avoid adding the dependency name to the buildreq cache
buildreq.remove_buildreq(name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm so this removes it from the buildreq and rebuildreq_cache. Ideally we want to see if this is from the buildreq_cache or the buildreq_add and if it is in the cache just remove it and retry the build. If it is from the buildreq_add then it is fatal and the build should halt. Unfortunately we don't really keep the original file contents themselves around and just the consolidated sets.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I had not considered the interaction between buildreq_add and buildreq_cache here..

So, the cases to handle are:

  1. Unresolvable name in buildreq_add only -> halt build
  2. Unresolvable name in buildreq_cache only -> eject from cache and start new build round
  3. Unresolvable name in both files -> eject from cache and halt build

Does that look right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately we don't really keep the original file contents themselves around and just the consolidated sets.

I didn't understand what you meant here at first, but after testing, it's very clear now :-P

I would like to solve that problem, too, but not sure how to go about it yet.

is_clean = False

return is_clean
Expand Down
29 changes: 29 additions & 0 deletions autospec/buildreq.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,23 @@ def add_buildreq(req, cache=False):
return new


def remove_buildreq(req):
"""Remove req from the global buildreqs and buildreqs_cache sets."""
global buildreqs
global buildreqs_cache

req = req.strip()

try:
buildreqs.remove(req)
except KeyError:
pass
try:
buildreqs_cache.remove(req)
except KeyError:
pass


def add_requires(req, override=False):
"""Add req to the global requires set if it is present in buildreqs and os_packages and is not banned."""
global buildreqs
Expand Down Expand Up @@ -118,6 +135,18 @@ def add_requires(req, override=False):
return new


def remove_requires(req):
"""Remove req from the global requires set."""
global requires

req = req.strip()

try:
requires.remove(req)
except KeyError:
pass


def add_pkgconfig_buildreq(preq, cache=False):
"""Format preq as pkgconfig req and add to buildreqs."""
if config.config_opts['32bit']:
Expand Down
8 changes: 3 additions & 5 deletions autospec/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -748,22 +748,20 @@ def write_default_conf_file(name, description):
for banned in content:
print("Banning build requirement: %s." % banned)
buildreq.banned_buildreqs.add(banned)
buildreq.buildreqs.discard(banned)
buildreq.buildreqs_cache.discard(banned)
buildreq.remove_buildreq(banned)

content = read_conf_file(os.path.join(path, "pkgconfig_ban"))
for banned in content:
banned = "pkgconfig(%s)" % banned
print("Banning build requirement: %s." % banned)
buildreq.banned_buildreqs.add(banned)
buildreq.buildreqs.discard(banned)
buildreq.buildreqs_cache.discard(banned)
buildreq.remove_buildreq(banned)

content = read_conf_file(os.path.join(path, "requires_ban"))
for banned in content:
print("Banning runtime requirement: %s." % banned)
buildreq.banned_requires.add(banned)
buildreq.requires.discard(banned)
buildreq.remove_requires(banned)

content = read_conf_file(os.path.join(path, "buildreq_add"))
for extra in content:
Expand Down