Commit 943a8989 authored by Hans Wennborg's avatar Hans Wennborg
Browse files

Merging r229421: 

------------------------------------------------------------------------
r229421 | dexonsmith | 2015-02-16 11:18:01 -0800 (Mon, 16 Feb 2015) | 33 lines

Bitcode: Fix major regression: large files w/ debug info

The metadata/value split introduced a major regression reading large
bitcode files that contain debug info (or other cyclic (non-self
reference) metadata graphs).  For the first time in a while, I dropped
from libLTO.dylib down to `llvm-lto` with a non-trivial bitcode file
(~350MB), and I hit this when reading the result of ld64's `-save-temps`
in `llvm-lto`.

Here's pseudo-code for what was going on:

    read-main-metadata-block:
      for each md:
        if has-fwd-ref: // Only true for cyclic graphs.
          any-fwd-refs <- true
      if any-fwd-refs:
        foreach md:
          resolve-cycles(md) // Handle cycles.

    foreach function:
      read-function-metadata-block: // Such as !alias, !loop
        if any-fwd-refs:
          foreach md: // (all metadata, not just this block)
            resolve-cycles(md) // A no-op, but the loop is expensive!!

This commit resets the `AnyFwdRefs` flag to `false`.  This on its own
was enough to change my Release+Asserts `llvm-lto` time for reading this
bitcode from over 20 minutes (I gave up on it) to 20 seconds.  I've gone
further by tracking the min/max metadata forward-references in a
metadata block.  This protects against a schema that has lots of
functions that each reference their own metadata cycle.

Unfortunately, this regression is in the 3.6 branch as well.
------------------------------------------------------------------------

llvm-svn: 229545
parent 557bc8e4

llvm/lib/Bitcode/Reader/BitcodeReader.cpp

+15 −3
Original line number Diff line number Diff line
@@ -538,9 +538,17 @@ Metadata *BitcodeReaderMDValueList::getValueFwdRef(unsigned Idx) { 
  if (Metadata *MD = MDValuePtrs[Idx])

    return MD;



  // Create and return a placeholder, which will later be RAUW'd.

  // Track forward refs to be resolved later.

  if (AnyFwdRefs) {

    MinFwdRef = std::min(MinFwdRef, Idx);

    MaxFwdRef = std::max(MaxFwdRef, Idx);

  } else {

    AnyFwdRefs = true;

    MinFwdRef = MaxFwdRef = Idx;

  }

  ++NumFwdRefs;



  // Create and return a placeholder, which will later be RAUW'd.

  Metadata *MD = MDNode::getTemporary(Context, None);

  MDValuePtrs[Idx].reset(MD);

  return MD;
@@ -556,11 +564,15 @@ void BitcodeReaderMDValueList::tryToResolveCycles() { 
    return;



  // Resolve any cycles.

  for (auto &MD : MDValuePtrs) {

  for (unsigned I = MinFwdRef, E = MaxFwdRef + 1; I != E; ++I) {

    auto &MD = MDValuePtrs[I];

    assert(!(MD && isa<MDNodeFwdDecl>(MD)) && "Unexpected forward reference");

    if (auto *N = dyn_cast_or_null<UniquableMDNode>(MD))

      N->resolveCycles();

  }



  // Make sure we return early again until there's another forward ref.

  AnyFwdRefs = false;

}



Type *BitcodeReader::getTypeByID(unsigned ID) {

llvm/lib/Bitcode/Reader/BitcodeReader.h

+2 −0
Original line number Diff line number Diff line
@@ -99,6 +99,8 @@ public: 
class BitcodeReaderMDValueList {

  unsigned NumFwdRefs;

  bool AnyFwdRefs;

  unsigned MinFwdRef;

  unsigned MaxFwdRef;

  std::vector<TrackingMDRef> MDValuePtrs;



  LLVMContext &Context;