Skip to content

Commit

Permalink
PS-5363 (Merge MySQL 8.0.17): Merging mysql-8.0.17
Browse files Browse the repository at this point in the history
https://jira.percona.com/browse/PS-5363

- Mark keyring_vault pluging as PLUGIN_OPT_ALLOW_EARLY, as the keyring
  plugins should be after WL#12935 [6].

- Revert our fix for PS-5126 due to upstream fix for bug 93544
  (3bb6832, 68e1a95).

- Reconcile our fix for PS-5485 with upstream fix for bug 64665 [2] by
  removing undo-specific checks in i_s_dict_fill_innodb_tablespaces so
  that the upstream fix handles them, and reverting our additions to
  innodb.innodb_undo_basic test due to slightly different fix
  behavior.

- Revert our fix PS-5221 due to upstream fix for bug 93708. Note that
  upstream reported performance regression caused by using std::chrono
  library. Thus, replaced the parts of our fix that are not touched by
  upstream (i.e. LRU manager sleeps and "failure to get a free page"
  printouts) with ut_time_monotonic_ms too. Part of the fix [3]
  removes ut_usectime/ut_time_us, thus replace them with
  ut_time_monotonic_ms/ut_time_monotonic_us too in other our
  patches. At the same time use ut_time_monotonic_us_t type as
  appropriate.

- Resolve interaction between our fix for PS-1689 / bug 72466 and
  os_event changes in upstream [3]. The upstream introduced a
  condition variable attribute that must be initialized before any
  condition variable. And our patch forced some of the mutex events to
  be created at startup time. The combination of these changes
  suffers from C++ initialization order fiasco. Observe that
  startup-time created mutexes are still unusable and that subsequent
  mutex_create does placement new, which constructs them again. Thus
  fix by avoiding the startup-time construction but keep the static
  memory allocation for them by introducing a new type
  ib_uninitialized_mutex_t, which holds memory for the mutex and later
  becomes ib_mutex_t. Convert all global and static variable InnoDB
  mutexes to ib_uninitialized_mutex_t. As a consequence, since
  mutex_own macro was converted to an inline function, fix upstream
  bug 96401 (No-op mutex_own calls instead of ut_ad(mutex_own)).

- Convert LRU manager and redo log tracking thread to use the new
  InnoDB background thread framework [5].

- Replace alloc_root calls with MEM_ROOT::Alloc in accordance with
  upstream [1].

- Replace some uses of uint64 with uint64_t due to upstream migrating
  towards the same.

- Replace trx_sys_update_mysql_binlog_position call with
  trx_sys_write_binlog_position due to upstream [4].

- Restore ReadView::up_limit_id() method, which was removed by [4],
  because it is used by our INFORMATION_SCHEMA.XTRADB_READ_VIEW table.

- Update TravisCI and CircleCI configs to use clang-format version 8
  instead of 5 due to upstream [7].

- Renamed rocksdb.foo and tokudb.foo MTR suites to rocksdb_foo and
  tokudb_foo due to stricter MTR checking which points out that dots
  are not allowed inside suite names.

- Cleanup MTR disabled.def to remove missing tests and to fix up
  suites of disabled and then moved tests. This has been caused by
  stricted disabled.def validation by the upstream.

[1]:

commit 5001463
Author: Steinar H. Gunderson <[email protected]>
Date:   Mon Feb 11 16:14:21 2019 +0100

    Bug #26781567: REMOVE C LEGACY [noclose]

    Remove all uses of the obsolete wrappers init_alloc() and sql_alloc().

    This is done 99% mechanically, with Perl and clang-format; there was
    one case in opt_range.cc that required manual intervention (a ternary
    expression used for the MEM_ROOT), and of course the removals themselves.

    Change-Id: Iaa11deadbded0253a6b5e0dd3a417c650a4692fa

[2]:

commit ab2817b
Author: Mayank Prasad <[email protected]>
Date:   Mon Apr 22 12:43:17 2019 +0530

    Bug#29492911 : ENABLING UNDO-TABLESPACE ENCRYPTION DOESN'T MARK TABLESPACE
                   ENCRYPTION FLAG

    Issue:
     - Encryption information displayed in INFORMATION_SCHEMA.INNODB_TABLESPACES
       is being picked from DD metadata.
     - Change in encryption of UNDO Tablespaces is governed by server variable
       innodb_undo_log_encryption instead of a DDL.
     - Thus this DD metadata is not changed when UNDO encryption (server variable)
       is changed.
     - Thus changes of UNDO tablespace encryption is not updated in
       INFORMATION_SHCEMA.INNODB_TABLESPACES.

    Fix:
     - A complete fix would be to alter UNDO Tablespace Encryption using a DDL.
       Once that is done, DD metadata will be updated and Encryption information
       in INNODB_TABLESPACES will also be reflected correctly.
     - This patch if a workaroung fix.
     - As of now, UNDO Tablespace Encryption is governed by srv_undo_log_encrypt
       global variable. So while populating INFORMATION_SCHEMA.INNODB_TABLESPACES
       for UNDO Tablespace Encryption information, set it according to
       srv_undo_log_encrypt.

    Reviewed by : Kevin Lewis <[email protected]>
    RB : 21933

[3]:

commit 0fddbfa
Author: Pawe<C5><82> Olchawa <[email protected]>
Date:   Wed Mar 27 04:36:41 2019 +0100

    BUG#29138644 PAGE CLEANER WILL SLEEP FOR LONG TIME IF CLOCK CHANGES

    1. We use pthread_condattr_setclock() to configure CLOCK_MONOTONIC
       as a clock used in pthread_cond_timedwait (in os_event).

    2. We check if we succeeded to setup the CLOCK_MONOTONIC and fallback
       to the old way if we failed (emitting a warning).

    3. We rely on clock_gettime() in the os_event::wait_time_low()
       if we succeeded to setup the CLOCK_MONOTONIC.

    4. Usages of functions that return seconds/milliseconds/microseconds
       since epoch, became replaced with usages of functions that return
       monotonic number of seconds/ms/us since epoch. These new functions
       are based on usage of std::chrono::steady_clock.

    There are still some usages of ut_time (returns system time) in cases
    where the time is exposed to the user (e.g. when trx was started).
    I left them as they were.

    Keep in mind that some OS could still have their own minor issues with
    the CLOCK_MONOTONIC, in which case you might want to update the OS.

    If one day, we switched from direct usages of pthread_cond to usages of
    std::condition_variable (in os_event), we could then delete all usages of
    the clock_gettime. That's why I put that code very close to os_event.
    There are no other usages. What's more, because of that I could remove
    implementation of the my_gettimeofday for Windows.

    RB: 21796

[4]:

commit 1efd16d
Author: Debarun Banerjee <[email protected]>
Date:   Fri May 17 18:11:07 2019 +0530

        Backport 8.0 wl#11636 InnoDB: Clone Remote provisioning
        RB 21407
        Reviewed-by: Pawel Olchawa <[email protected]>
                     Annamalai Gurusami <[email protected]>

        Backport 8.0 wl#9211 InnoDB: Clone Replication Coordinates
        RB 21417
        Reviewed-by: Pawel Olchawa <[email protected]>
                     Pedro Figueiredo <[email protected]>

        Backport 8.0 wl#9682  InnoDB: Support cloning encrypted and compressed database
        RB 21409
        Reviewed-by: Mayank Prasad <[email protected]>
                     Kevin Lewis <[email protected]>

[5]:

commit cd6fab0
Author: Pawe<C5><82> Olchawa <[email protected]>
Date:   Thu Apr 11 13:33:16 2019 +0200

    BUG#29417503 MYSQL COULD HANG DURING SHUTDOWN BECAUSE OF ZOMBIE IN MYSQL_TRX_LIST

    Proper and efficient mechanism to check if threads are gone is delivered.
    Based on that the code is fixed for all background threads that are started
    in InnoDB.

    Also, shutdown in InnoDB was not precise enough regarding when the master
    thread should exit. That is fixed - a new intermediate state of shutdown
    is introduced, which is placed between CLEANUP and FLUSH_PHASE and marks
    the point since which master should be closed.

    RB: 21627

[6]:

commit a37f91a
Author: Georgi Kodinov <[email protected]>
Date:   Thu Feb 7 17:25:43 2019 +0100

    WL#12935: Control what plugins can be passed to --early-plugin-load

    RB#21450

[7]:

commit e0981d9
Author: Anonymous Reformatter <matt.reform@invalid>
Date:   Tue Apr 2 11:03:07 2019 +0200

    Bug #29588949: UPGRADE TO CLANG-FORMAT 8

    Upgrade clang-format settings from 5.0 to 8, and rerun it over the entire code base.

    The real committer is Steinar H. Gunderson <[email protected]>,
    but the author has been set to Anonymous Reformatter so that git blame won't
    show myself as author of all existing code. To ignore whitespace changes caused
    by this commit, please use git blame -w. For ignoring larger changes, you can
    run blame on the commit immediately preceding this, as follows:

      git blame clang-format-8-base^ -- path/to/file.cc

    Change-Id: Ia6fe3de72ab6e760b678305175204f216185fa89
  • Loading branch information
percona-ysorokin committed Oct 15, 2019
2 parents 629210c + 4869291 commit 992ef63
Show file tree
Hide file tree
Showing 5,915 changed files with 328,712 additions and 99,954 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
6 changes: 3 additions & 3 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ jobs:
name: Clang-format test
command: |
curl -sSL "http://apt.llvm.org/llvm-snapshot.gpg.key" | sudo -E apt-key add -
echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-5.0 main" | sudo tee -a /etc/apt/sources.list > /dev/null
echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-8 main" | sudo tee -a /etc/apt/sources.list > /dev/null
sudo -E apt-get -yq update >> ~/apt-get-update.log 2>&1
sudo -E apt-get -yq --no-install-suggests --no-install-recommends install clang-format-5.0
sudo -E apt-get -yq --no-install-suggests --no-install-recommends install clang-format-8
wget https://llvm.org/svn/llvm-project/cfe/trunk/tools/clang-format/clang-format-diff.py
chmod a+x clang-format-diff.py
git diff -U0 --no-color HEAD^1 *.c *.cc *.cpp *.h *.hpp *.i *.ic *.ih | ./clang-format-diff.py -binary=clang-format-5.0 -style=file -p1 >_GIT_DIFF
git diff -U0 --no-color HEAD^1 *.c *.cc *.cpp *.h *.hpp *.i *.ic *.ih | ./clang-format-diff.py -binary=clang-format-8 -style=file -p1 >_GIT_DIFF
if [ ! -s _GIT_DIFF ]; then
echo The last git commit is clang-formatted;
else
Expand Down
52 changes: 48 additions & 4 deletions .clang-format
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA

# This is the output of clang-format-5.0 --style=google --dump-config,
# This is the output of clang-format-8 --style=google --dump-config,
# except for changes mentioned below. We lock the style so that any newer
# version of clang-format will give the same result; as time goes, we may
# update this list, requiring newer versions of clang-format.

---
Language: Cpp
# BasedOnStyle: Google
AccessModifierOffset: -1
Expand All @@ -43,7 +44,7 @@ AllowShortLoopsOnASingleLine: true
AlwaysBreakAfterDefinitionReturnType: None
AlwaysBreakAfterReturnType: None
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: true
AlwaysBreakTemplateDeclarations: Yes
BinPackArguments: true
BinPackParameters: true
BraceWrapping:
Expand All @@ -55,6 +56,7 @@ BraceWrapping:
AfterObjCDeclaration: false
AfterStruct: false
AfterUnion: false
AfterExternBlock: false
BeforeCatch: false
BeforeElse: false
IndentBraces: false
Expand All @@ -64,6 +66,7 @@ BraceWrapping:
BreakBeforeBinaryOperators: None
BreakBeforeBraces: Attach
BreakBeforeInheritanceComma: false
BreakInheritanceList: BeforeColon
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: false
BreakConstructorInitializers: BeforeColon
Expand All @@ -83,7 +86,10 @@ ForEachMacros:
- foreach
- Q_FOREACH
- BOOST_FOREACH
IncludeBlocks: Preserve
IncludeCategories:
- Regex: '^<ext/.*\.h>'
Priority: 2
- Regex: '^<.*\.h>'
Priority: 1
- Regex: '^<.*'
Expand All @@ -92,6 +98,7 @@ IncludeCategories:
Priority: 3
IncludeIsMainRegex: '([-_](test|unittest))?$'
IndentCaseLabels: true
IndentPPDirectives: None
IndentWidth: 2
IndentWrappedFunctionNames: false
JavaScriptQuotes: Leave
Expand All @@ -101,30 +108,67 @@ MacroBlockBegin: ''
MacroBlockEnd: ''
MaxEmptyLinesToKeep: 1
NamespaceIndentation: None
ObjCBinPackProtocolList: Never
ObjCBlockIndentWidth: 2
ObjCSpaceAfterProperty: false
ObjCSpaceBeforeProtocolList: false
ObjCSpaceBeforeProtocolList: true
PenaltyBreakAssignment: 2
PenaltyBreakBeforeFirstCallParameter: 1
PenaltyBreakComment: 300
PenaltyBreakFirstLessLess: 120
PenaltyBreakString: 1000
PenaltyBreakTemplateDeclaration: 10
PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 200
RawStringFormats:
- Language: Cpp
Delimiters:
- cc
- CC
- cpp
- Cpp
- CPP
- 'c++'
- 'C++'
CanonicalDelimiter: ''
BasedOnStyle: google
- Language: TextProto
Delimiters:
- pb
- PB
- proto
- PROTO
EnclosingFunctions:
- EqualsProto
- EquivToProto
- PARSE_PARTIAL_TEXT_PROTO
- PARSE_TEST_PROTO
- PARSE_TEXT_PROTO
- ParseTextOrDie
- ParseTextProtoOrDie
CanonicalDelimiter: ''
BasedOnStyle: google
ReflowComments: true
SortIncludes: true
SortUsingDeclarations: true
SpaceAfterCStyleCast: false
SpaceAfterTemplateKeyword: true
SpaceBeforeAssignmentOperators: true
SpaceBeforeCpp11BracedList: false
SpaceBeforeCtorInitializerColon: true
SpaceBeforeInheritanceColon: true
SpaceBeforeParens: ControlStatements
SpaceBeforeRangeBasedForLoopColon: true
SpaceInEmptyParentheses: false
SpacesBeforeTrailingComments: 2
SpacesInAngles: false
SpacesInContainerLiterals: true
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesInSquareBrackets: false
StatementMacros:
- Q_UNUSED
- QT_REQUIRE_VERSION
TabWidth: 8
UseTab: Never

Expand All @@ -133,5 +177,5 @@ UseTab: Never
DerivePointerAlignment: false
PointerAlignment: Right

# MySQL source code is allowed to use C++11 features.
# MySQL source code is allowed to use C++11 (and C++14) features.
Standard: Cpp11
6 changes: 3 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ matrix:
- env: COMMAND=clang-test
script:
- curl -sSL "http://apt.llvm.org/llvm-snapshot.gpg.key" | sudo -E apt-key add -;
- echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-5.0 main" | sudo tee -a /etc/apt/sources.list > /dev/null;
- echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-8 main" | sudo tee -a /etc/apt/sources.list > /dev/null;
- sudo -E apt-get -yq update >> ~/apt-get-update.log 2>&1;
- sudo -E apt-get -yq --no-install-suggests --no-install-recommends install diffstat clang-format-5.0 || travis_terminate 1
- sudo -E apt-get -yq --no-install-suggests --no-install-recommends install diffstat clang-format-8 || travis_terminate 1
- wget https://llvm.org/svn/llvm-project/cfe/trunk/tools/clang-format/clang-format-diff.py || travis_terminate 1
- chmod a+x clang-format-diff.py
- git diff -U0 --no-color HEAD^1 *.c *.cc *.cpp *.h *.hpp *.i *.ic *.ih | ./clang-format-diff.py -binary=clang-format-5.0 -style=file -p1 >_GIT_DIFF
- git diff -U0 --no-color HEAD^1 *.c *.cc *.cpp *.h *.hpp *.i *.ic *.ih | ./clang-format-diff.py -binary=clang-format-8 -style=file -p1 >_GIT_DIFF
- '[ ! -s _GIT_DIFF ] && { echo The last git commit is clang-formatted; travis_terminate 0; } || { diffstat _GIT_DIFF; echo; cat _GIT_DIFF; travis_terminate 1; }'


Expand Down
Loading

0 comments on commit 992ef63

Please sign in to comment.