Skip to content

Commit

Permalink
Work around for atomic_write() data race exception issue
Browse files Browse the repository at this point in the history
It is not unusual that we observed the atomic_write() has data race
in multipul threads and can raise exception to stop build as below:
PermissionError: [WinError 5] Access is denied.

Even the %LOCALAPPDATA% suggestion in below links still cannot
totally fix the exceptions in our CI build system
frerich#334 (comment)
frerich#342

So, simply retry the atomic_write() several times first and then just
pass through if still fails. The save() work around looks only impact
the statistics data accurate. Since The build expection hang is always
show stopper issue for us with top priority, it's OK for us get harden
stable compiler cache with cost of inaccurate statistics info.
  • Loading branch information
shijunjing committed Mar 15, 2019
1 parent 8dbd89f commit 941e63a
Showing 1 changed file with 28 additions and 4 deletions.
32 changes: 28 additions & 4 deletions clcache/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
from atomicwrites import atomic_write
import configparser

VERSION = "4.2.0-dev-edk2-20190315-v3"
VERSION = "4.2.0-dev-edk2-20190315-v4"

HashAlgorithm = hashlib.md5

Expand Down Expand Up @@ -657,8 +657,32 @@ def __init__(self, fileName):

def save(self):
if self._dirty:
with atomic_write(self._fileName, overwrite=True) as f:
json.dump(self._dict, f, sort_keys=True, indent=4)
#
# It is not unusual that we observed the atomic_write() has data race
# in multipul threads and can raise exception to stop build as below:
# PermissionError: [WinError 5] Access is denied.
# Even the %LOCALAPPDATA% suggestion in below links still cannot
# totally fix the exceptions in our CI build system
# https://github.com/frerich/clcache/pull/334#issuecomment-452475038
# https://github.com/frerich/clcache/issues/342
#
# So, simply retry the atomic_write() several times first and then just
# pass through if still fails. The save() work around looks only impact
# the statistics data accurate. Since The build expection hang is always
# show stopper issue for us with top priority, it's OK for us get harden
# stable compiler cache with cost of inaccurate statistics info.
# https://stackoverflow.com/questions/2083987/how-to-retry-after-exception
#
for attempt in range(3):
try:
with atomic_write(self._fileName, overwrite=True) as f:
json.dump(self._dict, f, sort_keys=True, indent=4)
except:
printErrStr("clcache: atomic_write %s fail with exception" % self._fileName)
else:
break
else:
pass

def __setitem__(self, key, value):
self._dict[key] = value
Expand All @@ -675,7 +699,7 @@ def __eq__(self, other):


class Configuration:
_defaultValues = {"MaximumCacheSize": 5368709120} # 5 GiB
_defaultValues = {"MaximumCacheSize": 10*1204*1024*1024} # 10 GiB

def __init__(self, configurationFile):
self._configurationFile = configurationFile
Expand Down

0 comments on commit 941e63a

Please sign in to comment.