Missed function specialization + partial devirtualization

v8:
 1. Rebase to master with Martin's static function (r280043) comments merge.
    Boostrap/testsuite/SPEC2017 tested pass on Power8-LE.
 2. TODO:
    2.1. C++ devirt for multiple speculative call targets.
    2.2. ipa-icf ipa_merge_profiles refine with COMDAT inline testcase.

This patch aims to fix PR69678 caused by PGO indirect call profiling
performance issues.
The bug that profiling data is never working was fixed by Martin's pull
back of topN patches, performance got GEOMEAN ~1% improvement(+24% for
511.povray_r specifically).
Still, currently the default profile only generates SINGLE indirect target
that called more than 75%.  This patch leverages MULTIPLE indirect
targets use in LTO-WPA and LTO-LTRANS stage, as a result, function
specialization, profiling, partial devirtualization, inlining and
cloning could be done successfully based on it.
Performance can get improved from 0.70 sec to 0.38 sec on simple tests.
Details are:
  1.  PGO with topn is enabled by default now, but only one indirect
  target edge will be generated in ipa-profile pass, so add variables to enable
  multiple speculative edges through passes, speculative_id will record the
  direct edge index bind to the indirect edge, indirect_call_targets length
  records how many direct edges owned by the indirect edge, postpone gimple_ic
  to ipa-profile like default as inline pass will decide whether it is benefit
  to transform indirect call.
  2.  Use speculative_id to track and search the reference node matched
  with the direct edge's callee for multiple targets.  Actually, it is the
  caller's responsibility to handle the direct edges mapped to same indirect
  edge.  speculative_call_info will return one of the direct edge specified,
  this will leverage current IPA edge process framework mostly.
  3.  Enable LTO WPA/LTRANS stage multiple indirect call targets analysis for
  profile full support in ipa passes and cgraph_edge functions.  speculative_id
  can be set by make_speculative id when multiple targets are binded to
  one indirect edge, and cloned if new edge is cloned.  speculative_id
  is streamed out and stream int by lto like lto_stmt_uid.
  4.  Create and duplicate all speculative direct edge's call summary
  in ipa-fnsummary.c with auto_vec.
  5.  Add 1 in module testcase and 2 cross module testcases.
  6.  Bootstrap and regression test passed on Power8-LE.  No function
  and performance regression for SPEC2017.

gcc/ChangeLog

	2020-01-14  Xiong Hu Luo  <luoxhu@linux.ibm.com>

	PR ipa/69678
	* cgraph.c (symbol_table::create_edge): Init speculative_id and
	target_prob.
	(cgraph_edge::make_speculative): Add param for setting speculative_id
	and target_prob.
	(cgraph_edge::speculative_call_info): Update comments and find reference
	by speculative_id for multiple indirect targets.
	(cgraph_edge::resolve_speculation): Decrease the speculations
	for indirect edge, drop it's speculative if not direct target
	left. Update comments.
	(cgraph_edge::redirect_call_stmt_to_callee): Likewise.
	(cgraph_node::dump): Print num_speculative_call_targets.
	(cgraph_node::verify_node): Don't report error if speculative
	edge not include statement.
	(cgraph_edge::num_speculative_call_targets_p): New function.
	* cgraph.h (int common_target_id): Remove.
	(int common_target_probability): Remove.
	(num_speculative_call_targets): New variable.
	(make_speculative): Add param for setting speculative_id.
	(cgraph_edge::num_speculative_call_targets_p): New declare.
	(target_prob): New variable.
	(speculative_id): New variable.
	* ipa-fnsummary.c (analyze_function_body): Create and duplicate
	  call summaries for multiple speculative call targets.
	* cgraphclones.c (cgraph_node::create_clone): Clone speculative_id.
	* ipa-profile.c (struct speculative_call_target): New struct.
	(class speculative_call_summary): New class.
	(class speculative_call_summaries): New class.
	(call_sums): New variable.
	(ipa_profile_generate_summary): Generate indirect multiple targets summaries.
	(ipa_profile_write_edge_summary): New function.
	(ipa_profile_write_summary): Stream out indirect multiple targets summaries.
	(ipa_profile_dump_all_summaries): New function.
	(ipa_profile_read_edge_summary): New function.
	(ipa_profile_read_summary_section): New function.
	(ipa_profile_read_summary): Stream in indirect multiple targets summaries.
	(ipa_profile): Generate num_speculative_call_targets from
	profile summaries.
	* ipa-ref.h (speculative_id): New variable.
	* ipa-utils.c (ipa_merge_profiles): Update with target_prob.
	* lto-cgraph.c (lto_output_edge): Remove indirect common_target_id and
	common_target_probability.   Stream out speculative_id and
	num_speculative_call_targets.
	(input_edge): Likewise.
	* predict.c (dump_prediction): Remove edges count assert to be
	precise.
	* symtab.c (symtab_node::create_reference): Init speculative_id.
	(symtab_node::clone_references): Clone speculative_id.
	(symtab_node::clone_referring): Clone speculative_id.
	(symtab_node::clone_reference): Clone speculative_id.
	(symtab_node::clear_stmts_in_references): Clear speculative_id.
	* tree-inline.c (copy_bb): Duplicate all the speculative edges
	if indirect call contains multiple speculative targets.
	* value-prof.h  (check_ic_target): Remove.
	* value-prof.c  (gimple_value_profile_transformations):
	Use void function gimple_ic_transform.
	* value-prof.c  (gimple_ic_transform): Handle topn case.
	Fix comment typos.  Change it to a void function.

gcc/testsuite/ChangeLog

	2020-01-14  Xiong Hu Luo  <luoxhu@linux.ibm.com>

	PR ipa/69678
	* gcc.dg/tree-prof/indir-call-prof-topn.c: New testcase.
	* gcc.dg/tree-prof/crossmodule-indir-call-topn-1.c: New testcase.
	* gcc.dg/tree-prof/crossmodule-indir-call-topn-1a.c: New testcase.
	* gcc.dg/tree-prof/crossmodule-indir-call-topn-2.c: New testcase.
	* lib/scandump.exp: Dump executable file name.
	* lib/scanwpaipa.exp: New scan-pgo-wap-ipa-dump.

21 files changed, 934 insertions, 169 deletions

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 4664c320805..07e5bebe909 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,64 @@
+2020-01-14  Xiong Hu Luo  <luoxhu@linux.ibm.com>
+
+	PR ipa/69678
+	* cgraph.c (symbol_table::create_edge): Init speculative_id and
+	target_prob.
+	(cgraph_edge::make_speculative): Add param for setting speculative_id
+	and target_prob.
+	(cgraph_edge::speculative_call_info): Update comments and find reference
+	by speculative_id for multiple indirect targets.
+	(cgraph_edge::resolve_speculation): Decrease the speculations
+	for indirect edge, drop it's speculative if not direct target
+	left. Update comments.
+	(cgraph_edge::redirect_call_stmt_to_callee): Likewise.
+	(cgraph_node::dump): Print num_speculative_call_targets.
+	(cgraph_node::verify_node): Don't report error if speculative
+	edge not include statement.
+	(cgraph_edge::num_speculative_call_targets_p): New function.
+	* cgraph.h (int common_target_id): Remove.
+	(int common_target_probability): Remove.
+	(num_speculative_call_targets): New variable.
+	(make_speculative): Add param for setting speculative_id.
+	(cgraph_edge::num_speculative_call_targets_p): New declare.
+	(target_prob): New variable.
+	(speculative_id): New variable.
+	* ipa-fnsummary.c (analyze_function_body): Create and duplicate
+	  call summaries for multiple speculative call targets.
+	* cgraphclones.c (cgraph_node::create_clone): Clone speculative_id.
+	* ipa-profile.c (struct speculative_call_target): New struct.
+	(class speculative_call_summary): New class.
+	(class speculative_call_summaries): New class.
+	(call_sums): New variable.
+	(ipa_profile_generate_summary): Generate indirect multiple targets summaries.
+	(ipa_profile_write_edge_summary): New function.
+	(ipa_profile_write_summary): Stream out indirect multiple targets summaries.
+	(ipa_profile_dump_all_summaries): New function.
+	(ipa_profile_read_edge_summary): New function.
+	(ipa_profile_read_summary_section): New function.
+	(ipa_profile_read_summary): Stream in indirect multiple targets summaries.
+	(ipa_profile): Generate num_speculative_call_targets from
+	profile summaries.
+	* ipa-ref.h (speculative_id): New variable.
+	* ipa-utils.c (ipa_merge_profiles): Update with target_prob.
+	* lto-cgraph.c (lto_output_edge): Remove indirect common_target_id and
+	common_target_probability.   Stream out speculative_id and
+	num_speculative_call_targets.
+	(input_edge): Likewise.
+	* predict.c (dump_prediction): Remove edges count assert to be
+	precise.
+	* symtab.c (symtab_node::create_reference): Init speculative_id.
+	(symtab_node::clone_references): Clone speculative_id.
+	(symtab_node::clone_referring): Clone speculative_id.
+	(symtab_node::clone_reference): Clone speculative_id.
+	(symtab_node::clear_stmts_in_references): Clear speculative_id.
+	* tree-inline.c (copy_bb): Duplicate all the speculative edges
+	if indirect call contains multiple speculative targets.
+	* value-prof.h  (check_ic_target): Remove.
+	* value-prof.c  (gimple_value_profile_transformations):
+	Use void function gimple_ic_transform.
+	* value-prof.c  (gimple_ic_transform): Handle topn case.
+	Fix comment typos.  Change it to a void function.
+
 2020-01-13  Andrew Pinski  <apinski@marvell.com>
 
 	* config/aarch64/aarch64-cores.def (octeontx2): New define.
diff --git a/gcc/cgraph.c b/gcc/cgraph.c
index fe3f06726d4..95b523d6be5 100644
--- a/gcc/cgraph.c
+++ b/gcc/cgraph.c
@@ -858,6 +858,8 @@ symbol_table::create_edge (cgraph_node *caller, cgraph_node *callee,
   edge->prev_callee = NULL;
   edge->next_callee = NULL;
   edge->lto_stmt_uid = 0;
+  edge->target_prob = 0;
+  edge->speculative_id = 0;
 
   edge->count = count;
   edge->call_stmt = call_stmt;
@@ -1044,10 +1046,16 @@ cgraph_edge::remove (cgraph_edge *edge)
    the reference representing the if conditional and attaches
    them all to the original indirect call statement.  
 
+   speculative_id is used to link direct calls with their corresponding
+   IPA_REF_ADDR references when representing speculative calls.
+
+   target_prob is the probability of the speculative call.
+
    Return direct edge created.  */
 
 cgraph_edge *
-cgraph_edge::make_speculative (cgraph_node *n2, profile_count direct_count)
+cgraph_edge::make_speculative (cgraph_node *n2, profile_count direct_count,
+			       unsigned int speculative_id, int target_prob)
 {
   cgraph_node *n = caller;
   ipa_ref *ref = NULL;
@@ -1065,24 +1073,53 @@ cgraph_edge::make_speculative (cgraph_node *n2, profile_count direct_count)
   else
     e2->can_throw_external = can_throw_external;
   e2->lto_stmt_uid = lto_stmt_uid;
+  e2->speculative_id = speculative_id;
+  e2->target_prob = target_prob;
   e2->in_polymorphic_cdtor = in_polymorphic_cdtor;
   count -= e2->count;
   symtab->call_edge_duplication_hooks (this, e2);
   ref = n->create_reference (n2, IPA_REF_ADDR, call_stmt);
   ref->lto_stmt_uid = lto_stmt_uid;
+  ref->speculative_id = speculative_id;
   ref->speculative = speculative;
   n2->mark_address_taken ();
   return e2;
 }
 
-/* Speculative call consist of three components:
-   1) an indirect edge representing the original call
-   2) an direct edge representing the new call
-   3) ADDR_EXPR reference representing the speculative check.
-   All three components are attached to single statement (the indirect
-   call) and if one of them exists, all of them must exist.
+/* Speculative calls represent a transformation of indirect calls
+   which may be later inserted into gimple in the following form:
+
+   if (call_dest == target1)
+   target1 ();
+   else if (call_dest == target2)
+   target2 ();
+   else
+   call_dest ();
+
+   This is a win in the case when target1 and target2 are common values for
+   call_dest as determined by ipa-devirt or indirect call profiling.
+   In particular this may enable inlining and other optimizations.
+
+   Speculative call consists of the following main components:
+
+   1) One or more "speculative" direct call (num_speculative_call_targets is
+   speculative direct call count belongs to the speculative indirect call)
+   2) One or more IPA_REF_ADDR references (representing the fact that code above
+   takes address of target1 and target2)
+   3) The fallback "speculative" indirect call
 
-   Given speculative call edge, return all three components.
+   Direct calls and corresponding references are linked by
+   speculative_id.
+
+   speculative_call_info returns triple
+   (direct_call, indirect call, IPA_REF_ADDR reference)
+   when called on one edge participating in the speculative call:
+
+   1) If called on direct call, its corresponding IPA_REF_ADDR and related
+   indirect call are returned.
+
+   2) If called on indirect call, it will return one of direct edges and its
+   matching IPA_REF_ADDR.
  */
 
 void
@@ -1122,7 +1159,7 @@ cgraph_edge::speculative_call_info (cgraph_edge *&direct,
 
   reference = NULL;
   for (i = 0; e->caller->iterate_reference (i, ref); i++)
-    if (ref->speculative
+    if (ref->speculative && ref->speculative_id == e->speculative_id
 	&& ((ref->stmt && ref->stmt == e->call_stmt)
 	    || (!ref->stmt && ref->lto_stmt_uid == e->lto_stmt_uid)))
       {
@@ -1138,9 +1175,18 @@ cgraph_edge::speculative_call_info (cgraph_edge *&direct,
 
 /* Speculative call EDGE turned out to be direct call to CALLEE_DECL.  Remove
    the speculative call sequence and return edge representing the call, the
-   original EDGE can be removed and deallocated.  It is up to caller to
-   redirect the call as appropriate.  Return the edge that now represents the
-   call.  */
+   original EDGE can be removed and deallocated.  Return the edge that now
+   represents the call.
+
+   For "speculative" indirect call that contains multiple "speculative"
+   targets (i.e. edge->indirect_info->num_speculative_call_targets > 1),
+   decrease the count and only remove current direct edge.
+
+   If no speculative direct call left to the speculative indirect call, remove
+   the speculative of both the indirect call and corresponding direct edge.
+
+   It is up to caller to iteratively resolve each "speculative" direct call and
+   redirect the call as appropriate.  */
 
 cgraph_edge *
 cgraph_edge::resolve_speculation (cgraph_edge *edge, tree callee_decl)
@@ -1184,7 +1230,16 @@ cgraph_edge::resolve_speculation (cgraph_edge *edge, tree callee_decl)
          in the functions inlined through it.  */
     }
   edge->count += e2->count;
-  edge->speculative = false;
+  if (edge->num_speculative_call_targets_p ())
+    {
+      /* The indirect edge has multiple speculative targets, don't remove
+	 speculative until all related direct edges are resolved.  */
+      edge->indirect_info->num_speculative_call_targets--;
+      if (!edge->indirect_info->num_speculative_call_targets)
+	edge->speculative = false;
+    }
+  else
+    edge->speculative = false;
   e2->speculative = false;
   ref->remove_reference ();
   if (e2->indirect_unknown_callee || e2->inline_failed)
@@ -1244,7 +1299,17 @@ cgraph_edge::make_direct (cgraph_edge *edge, cgraph_node *callee)
 
 /* If necessary, change the function declaration in the call statement
    associated with E so that it corresponds to the edge callee.  Speculations
-   can be resolved in the process and EDGE can be removed and deallocated.  */
+   can be resolved in the process and EDGE can be removed and deallocated.
+
+   The edge could be one of speculative direct call generated from speculative
+   indirect call.  In this circumstance, decrease the speculative targets
+   count (i.e. num_speculative_call_targets) and redirect call stmt to the
+   corresponding i-th target.  If no speculative direct call left to the
+   speculative indirect call, remove "speculative" of the indirect call and
+   also redirect stmt to it's final direct target.
+
+   It is up to caller to iteratively transform each "speculative"
+   direct call as appropriate.  */
 
 gimple *
 cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e)
@@ -1290,7 +1355,17 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e)
 	  e->caller->set_call_stmt_including_clones (e->call_stmt, new_stmt,
 						     false);
 	  e->count = gimple_bb (e->call_stmt)->count;
-	  e2->speculative = false;
+	  if (e2->num_speculative_call_targets_p ())
+	    {
+	      /* The indirect edge has multiple speculative targets, don't
+		 remove speculative until all related direct edges are
+		 redirected.  */
+	      e2->indirect_info->num_speculative_call_targets--;
+	      if (!e2->indirect_info->num_speculative_call_targets)
+		e2->speculative = false;
+	    }
+	  else
+	    e2->speculative = false;
 	  e2->count = gimple_bb (e2->call_stmt)->count;
 	  ref->speculative = false;
 	  ref->stmt = NULL;
@@ -2103,6 +2178,8 @@ cgraph_node::dump (FILE *f)
 	  if (edge->indirect_info->vptr_changed)
 	    fprintf (f, " (vptr maybe changed)");
 	}
+      fprintf (f, " Num speculative call targets: %i",
+	       edge->indirect_info->num_speculative_call_targets);
       fprintf (f, "\n");
       if (edge->indirect_info->polymorphic)
 	edge->indirect_info->context.dump (f);
@@ -3393,7 +3470,7 @@ cgraph_node::verify_node (void)
 
       for (e = callees; e; e = e->next_callee)
 	{
-	  if (!e->aux)
+	  if (!e->aux && !e->speculative)
 	    {
 	      error ("edge %s->%s has no corresponding call_stmt",
 		     identifier_to_locale (e->caller->name ()),
@@ -3732,6 +3809,14 @@ cgraph_edge::possibly_call_in_translation_unit_p (void)
   return node->get_availability () >= AVAIL_INTERPOSABLE;
 }
 
+/* Return num_speculative_targets of this edge.  */
+
+int
+cgraph_edge::num_speculative_call_targets_p (void)
+{
+  return indirect_info ? indirect_info->num_speculative_call_targets : 0;
+}
+
 /* A stashed copy of "symtab" for use by selftest::symbol_table_test.
    This needs to be a global so that it can be a GC root, and thus
    prevent the stashed copy from being garbage-collected if the GC runs
diff --git a/gcc/cgraph.h b/gcc/cgraph.h
index 71cd902312e..0ace13df1f9 100644
--- a/gcc/cgraph.h
+++ b/gcc/cgraph.h
@@ -1703,10 +1703,9 @@ public:
   int param_index;
   /* ECF flags determined from the caller.  */
   int ecf_flags;
-  /* Profile_id of common target obtained from profile.  */
-  int common_target_id;
-  /* Probability that call will land in function with COMMON_TARGET_ID.  */
-  int common_target_probability;
+
+  /* Number of speculative call targets, it's less than GCOV_TOPN_VALUES.  */
+  unsigned num_speculative_call_targets : 16;
 
   /* Set when the call is a virtual call with the parameter being the
      associated object pointer rather than a simple direct call.  */
@@ -1764,10 +1763,14 @@ public:
 
   /* Turn edge into speculative call calling N2. Update
      the profile so the direct call is taken COUNT times
-     with FREQUENCY.  */
-  cgraph_edge *make_speculative (cgraph_node *n2, profile_count direct_count);
-
-   /* Given speculative call edge, return all three components.  */
+     with FREQUENCY.  speculative_id is used to link direct calls with their
+     corresponding IPA_REF_ADDR references when representing speculative calls.
+     target_prob is the probability of the speculative call.  */
+  cgraph_edge *make_speculative (cgraph_node *n2, profile_count direct_count,
+				 unsigned int speculative_id = 0,
+				 int target_prob = 0);
+
+  /* Given speculative call edge, return all three components.  */
   void speculative_call_info (cgraph_edge *&direct, cgraph_edge *&indirect,
 			      ipa_ref *&reference);
 
@@ -1775,14 +1778,34 @@ public:
      the speculative call sequence and return edge representing the call, the
      original EDGE can be removed and deallocated.  It is up to caller to
      redirect the call as appropriate.  Return the edge that now represents the
-     call.  */
+     call.
+
+     For "speculative" indirect call that contains multiple "speculative"
+     targets (i.e. edge->indirect_info->num_speculative_call_targets > 1),
+     decrease the count and only remove current direct edge.
+
+     If no speculative direct call left to the speculative indirect call, remove
+     the speculative of both the indirect call and corresponding direct edge.
+
+     It is up to caller to iteratively resolve each "speculative" direct call
+     and redirect the call as appropriate.  */
   static cgraph_edge *resolve_speculation (cgraph_edge *edge,
 					   tree callee_decl = NULL);
 
   /* If necessary, change the function declaration in the call statement
      associated with edge E so that it corresponds to the edge callee.
      Speculations can be resolved in the process and EDGE can be removed and
-     deallocated.  */
+     deallocated.
+
+     The edge could be one of speculative direct call generated from speculative
+     indirect call.  In this circumstance, decrease the speculative targets
+     count (i.e. num_speculative_call_targets) and redirect call stmt to the
+     corresponding i-th target.  If no speculative direct call left to the
+     speculative indirect call, remove "speculative" of the indirect call and
+     also redirect stmt to it's final direct target.
+
+     It is up to caller to iteratively transform each "speculative"
+     direct call as appropriate.  */
   static gimple *redirect_call_stmt_to_callee (cgraph_edge *e);
 
   /* Create clone of edge in the node N represented
@@ -1829,6 +1852,9 @@ public:
      be internal to the current translation unit.  */
   bool possibly_call_in_translation_unit_p (void);
 
+  /* Return num_speculative_targets of this edge.  */
+  int num_speculative_call_targets_p (void);
+
   /* Expected number of executions: calculated in profile.c.  */
   profile_count count;
   cgraph_node *caller;
@@ -1848,6 +1874,11 @@ public:
   /* The stmt_uid of call_stmt.  This is used by LTO to recover the call_stmt
      when the function is serialized in.  */
   unsigned int lto_stmt_uid;
+  /*  target_prob is the probability of the speculative call.  */
+  unsigned int target_prob;
+  /* speculative id is used to link direct calls with their corresponding
+     IPA_REF_ADDR references when representing speculative calls.  */
+  unsigned int speculative_id : 16;
   /* Whether this edge was made direct by indirect inlining.  */
   unsigned int indirect_inlining_edge : 1;
   /* Whether this edge describes an indirect call with an undetermined
diff --git a/gcc/cgraphclones.c b/gcc/cgraphclones.c
index c186fb98cf5..e73e0696b91 100644
--- a/gcc/cgraphclones.c
+++ b/gcc/cgraphclones.c
@@ -133,6 +133,7 @@ cgraph_edge::clone (cgraph_node *n, gcall *call_stmt, unsigned stmt_uid,
   new_edge->inline_failed = inline_failed;
   new_edge->indirect_inlining_edge = indirect_inlining_edge;
   new_edge->lto_stmt_uid = stmt_uid;
+  new_edge->speculative_id = speculative_id;
   /* Clone flags that depend on call_stmt availability manually.  */
   new_edge->can_throw_external = can_throw_external;
   new_edge->call_stmt_cannot_inline_p = call_stmt_cannot_inline_p;
diff --git a/gcc/ipa-fnsummary.c b/gcc/ipa-fnsummary.c
index efe42641b6b..4e1c587b101 100644
--- a/gcc/ipa-fnsummary.c
+++ b/gcc/ipa-fnsummary.c
@@ -2604,7 +2604,7 @@ analyze_function_body (struct cgraph_node *node, bool early)
 	      edge_set_predicate (edge, &bb_predicate);
 	      if (edge->speculative)
 		{
-		  cgraph_edge *direct, *indirect;
+		  cgraph_edge *direct, *indirect, *next_direct;
 		  ipa_ref *ref;
 		  edge->speculative_call_info (direct, indirect, ref);
 		  gcc_assert (direct == edge);
@@ -2612,6 +2612,26 @@ analyze_function_body (struct cgraph_node *node, bool early)
 			 = ipa_call_summaries->get_create (indirect);
 		  ipa_call_summaries->duplicate (edge, indirect,
 						 es, es2);
+
+		  /* Create and duplicate call summaries for multiple
+		     speculative call targets.  */
+		  int num_specs = indirect->num_speculative_call_targets_p ();
+		  if (num_specs > 1)
+		    for (next_direct = edge->next_callee;
+			 next_direct && --num_specs;
+			 next_direct = next_direct->next_callee)
+		      {
+			next_direct->speculative_call_info (direct, indirect,
+							    ref);
+			if (direct == next_direct && next_direct->speculative
+			    && edge->call_stmt == stmt)
+			  {
+			    ipa_call_summary *es3
+			      = ipa_call_summaries->get_create (next_direct);
+			    ipa_call_summaries->duplicate (edge, next_direct,
+							   es, es3);
+			  }
+		      }
 		}
 	    }
 
diff --git a/gcc/ipa-profile.c b/gcc/ipa-profile.c
index 017f63e80a3..fc231c916b7 100644
--- a/gcc/ipa-profile.c
+++ b/gcc/ipa-profile.c
@@ -159,7 +159,99 @@ dump_histogram (FILE *file, vec<histogram_entry *> histogram)
    }
 }
 
-/* Collect histogram from CFG profiles.  */
+/* Structure containing speculative target information from profile.  */
+
+struct speculative_call_target
+{
+  speculative_call_target (unsigned int id = 0, int prob = 0)
+    : target_id (id), target_probability (prob)
+  {
+  }
+
+  /* Profile_id of target obtained from profile.  */
+  unsigned int target_id;
+  /* Probability that call will land in function with target_id.  */
+  unsigned int target_probability;
+};
+
+class speculative_call_summary
+{
+public:
+  speculative_call_summary () : speculative_call_targets ()
+  {}
+
+  auto_vec<speculative_call_target> speculative_call_targets;
+
+  void dump (FILE *f);
+
+};
+
+  /* Class to manage call summaries.  */
+
+class ipa_profile_call_summaries
+  : public call_summary<speculative_call_summary *>
+{
+public:
+  ipa_profile_call_summaries (symbol_table *table)
+    : call_summary<speculative_call_summary *> (table)
+  {}
+
+  /* Duplicate info when an edge is cloned.  */
+  virtual void duplicate (cgraph_edge *, cgraph_edge *,
+			  speculative_call_summary *old_sum,
+			  speculative_call_summary *new_sum);
+};
+
+static ipa_profile_call_summaries *call_sums = NULL;
+
+/* Dump all information in speculative call summary to F.  */
+
+void
+speculative_call_summary::dump (FILE *f)
+{
+  cgraph_node *n2;
+
+  unsigned spec_count = speculative_call_targets.length ();
+  for (unsigned i = 0; i < spec_count; i++)
+    {
+      speculative_call_target item = speculative_call_targets[i];
+      n2 = find_func_by_profile_id (item.target_id);
+      if (n2)
+	fprintf (f, "    The %i speculative target is %s with prob %3.2f\n", i,
+		 n2->dump_name (),
+		 item.target_probability / (float) REG_BR_PROB_BASE);
+      else
+	fprintf (f, "    The %i speculative target is %u with prob %3.2f\n", i,
+		 item.target_id,
+		 item.target_probability / (float) REG_BR_PROB_BASE);
+    }
+}
+
+/* Duplicate info when an edge is cloned.  */
+
+void
+ipa_profile_call_summaries::duplicate (cgraph_edge *, cgraph_edge *,
+				       speculative_call_summary *old_sum,
+				       speculative_call_summary *new_sum)
+{
+  if (!old_sum)
+    return;
+
+  unsigned old_count = old_sum->speculative_call_targets.length ();
+  if (!old_count)
+    return;
+
+  new_sum->speculative_call_targets.reserve_exact (old_count);
+  new_sum->speculative_call_targets.quick_grow_cleared (old_count);
+
+  for (unsigned i = 0; i < old_count; i++)
+    {
+      new_sum->speculative_call_targets[i]
+	= old_sum->speculative_call_targets[i];
+    }
+}
+
+/* Collect histogram and speculative target summaries from CFG profiles.  */
 
 static void
 ipa_profile_generate_summary (void)
@@ -169,7 +261,10 @@ ipa_profile_generate_summary (void)
   basic_block bb;
 
   hash_table<histogram_hash> hashtable (10);
-  
+
+  gcc_checking_assert (!call_sums);
+  call_sums = new ipa_profile_call_summaries (symtab);
+
   FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
     if (ENTRY_BLOCK_PTR_FOR_FN (DECL_STRUCT_FUNCTION (node->decl))->count.ipa_p ())
       FOR_EACH_BB_FN (bb, DECL_STRUCT_FUNCTION (node->decl))
@@ -191,23 +286,35 @@ ipa_profile_generate_summary (void)
 		  if (h)
 		    {
 		      gcov_type val, count, all;
-		      if (get_nth_most_common_value (NULL, "indirect call", h,
-						     &val, &count, &all))
+		      struct cgraph_edge *e = node->get_edge (stmt);
+		      if (e && !e->indirect_unknown_callee)
+			continue;
+
+		      speculative_call_summary *csum
+			= call_sums->get_create (e);
+
+		      for (unsigned j = 0; j < GCOV_TOPN_VALUES; j++)
 			{
-			  struct cgraph_edge * e = node->get_edge (stmt);
-			  if (e && !e->indirect_unknown_callee)
+			  if (!get_nth_most_common_value (NULL, "indirect call",
+							  h, &val, &count, &all,
+							  j))
+			    continue;
+
+			  if (val == 0)
 			    continue;
 
-			  e->indirect_info->common_target_id = val;
-			  e->indirect_info->common_target_probability
-			    = GCOV_COMPUTE_SCALE (count, all);
-			  if (e->indirect_info->common_target_probability > REG_BR_PROB_BASE)
+			  speculative_call_target item (
+			    val, GCOV_COMPUTE_SCALE (count, all));
+			  if (item.target_probability > REG_BR_PROB_BASE)
 			    {
 			      if (dump_file)
-				fprintf (dump_file, "Probability capped to 1\n");
-			      e->indirect_info->common_target_probability = REG_BR_PROB_BASE;
+				fprintf (dump_file,
+					 "Probability capped to 1\n");
+			      item.target_probability = REG_BR_PROB_BASE;
 			    }
+			  csum->speculative_call_targets.safe_push (item);
 			}
+
 		      gimple_remove_histogram_value (DECL_STRUCT_FUNCTION (node->decl),
 						      stmt, h);
 		    }
@@ -222,6 +329,33 @@ ipa_profile_generate_summary (void)
   histogram.qsort (cmp_counts);
 }
 
+/* Serialize the speculative summary info for LTO.  */
+
+static void
+ipa_profile_write_edge_summary (lto_simple_output_block *ob,
+				speculative_call_summary *csum)
+{
+  unsigned len = 0;
+
+  len = csum->speculative_call_targets.length ();
+
+  gcc_assert (len <= GCOV_TOPN_VALUES);
+
+  streamer_write_hwi_stream (ob->main_stream, len);
+
+  if (len)
+    {
+      unsigned spec_count = csum->speculative_call_targets.length ();
+      for (unsigned i = 0; i < spec_count; i++)
+	{
+	  speculative_call_target item = csum->speculative_call_targets[i];
+	  gcc_assert (item.target_id);
+	  streamer_write_hwi_stream (ob->main_stream, item.target_id);
+	  streamer_write_hwi_stream (ob->main_stream, item.target_probability);
+	}
+    }
+}
+
 /* Serialize the ipa info for lto.  */
 
 static void
@@ -238,10 +372,122 @@ ipa_profile_write_summary (void)
       streamer_write_uhwi_stream (ob->main_stream, histogram[i]->time);
       streamer_write_uhwi_stream (ob->main_stream, histogram[i]->size);
     }
+
+  if (!call_sums)
+    return;
+
+  /* Serialize speculative targets information.  */
+  unsigned int count = 0;
+  lto_symtab_encoder_t encoder = ob->decl_state->symtab_node_encoder;
+  lto_symtab_encoder_iterator lsei;
+  cgraph_node *node;
+
+  for (lsei = lsei_start_function_in_partition (encoder); !lsei_end_p (lsei);
+       lsei_next_function_in_partition (&lsei))
+    {
+      node = lsei_cgraph_node (lsei);
+      if (node->definition && node->has_gimple_body_p ()
+	  && node->indirect_calls)
+	count++;
+    }
+
+  streamer_write_uhwi_stream (ob->main_stream, count);
+
+  /* Process all of the functions.  */
+  for (lsei = lsei_start_function_in_partition (encoder);
+       !lsei_end_p (lsei) && count; lsei_next_function_in_partition (&lsei))
+    {
+      cgraph_node *node = lsei_cgraph_node (lsei);
+      if (node->definition && node->has_gimple_body_p ()
+	  && node->indirect_calls)
+	{
+	  int node_ref = lto_symtab_encoder_encode (encoder, node);
+	  streamer_write_uhwi_stream (ob->main_stream, node_ref);
+
+	  for (cgraph_edge *e = node->indirect_calls; e; e = e->next_callee)
+	    {
+	      speculative_call_summary *csum = call_sums->get_create (e);
+	      ipa_profile_write_edge_summary (ob, csum);
+	    }
+      }
+    }
+
   lto_destroy_simple_output_block (ob);
 }
 
-/* Deserialize the ipa info for lto.  */
+/* Dump all profile summary data for all cgraph nodes and edges to file F.  */
+
+static void
+ipa_profile_dump_all_summaries (FILE *f)
+{
+  fprintf (dump_file,
+	   "\n========== IPA-profile speculative targets: ==========\n");
+  cgraph_node *node;
+  FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
+    {
+      fprintf (f, "\nSummary for node %s:\n", node->dump_name ());
+      for (cgraph_edge *e = node->indirect_calls; e; e = e->next_callee)
+	{
+	  fprintf (f, "  Summary for %s of indirect edge %d:\n",
+		   e->caller->dump_name (), e->lto_stmt_uid);
+	  speculative_call_summary *csum = call_sums->get_create (e);
+	  csum->dump (f);
+	}
+    }
+  fprintf (f, "\n\n");
+}
+
+/* Read speculative targets information about edge for LTO WPA.  */
+
+static void
+ipa_profile_read_edge_summary (class lto_input_block *ib, cgraph_edge *edge)
+{
+  unsigned i, len;
+
+  len = streamer_read_hwi (ib);
+  gcc_assert (len <= GCOV_TOPN_VALUES);
+
+  speculative_call_summary *csum = call_sums->get_create (edge);
+
+  for (i = 0; i < len; i++)
+  {
+    speculative_call_target item (streamer_read_hwi (ib),
+	streamer_read_hwi (ib));
+    csum->speculative_call_targets.safe_push (item);
+  }
+}
+
+/* Read profile speculative targets section information for LTO WPA.  */
+
+static void
+ipa_profile_read_summary_section (struct lto_file_decl_data *file_data,
+				  class lto_input_block *ib)
+{
+  if (!ib)
+    return;
+
+  lto_symtab_encoder_t encoder = file_data->symtab_node_encoder;
+
+  unsigned int count = streamer_read_uhwi (ib);
+
+  unsigned int i;
+  unsigned int index;
+  cgraph_node * node;
+
+  for (i = 0; i < count; i++)
+    {
+      index = streamer_read_uhwi (ib);
+      encoder = file_data->symtab_node_encoder;
+      node
+	= dyn_cast<cgraph_node *> (lto_symtab_encoder_deref (encoder, index));
+
+      for (cgraph_edge *e = node->indirect_calls; e; e = e->next_callee)
+	ipa_profile_read_edge_summary (ib, e);
+    }
+}
+
+/* Deserialize the IPA histogram and speculative targets summary info for LTO.
+   */
 
 static void
 ipa_profile_read_summary (void)
@@ -253,6 +499,9 @@ ipa_profile_read_summary (void)
 
   hash_table<histogram_hash> hashtable (10);
 
+  gcc_checking_assert (!call_sums);
+  call_sums = new ipa_profile_call_summaries (symtab);
+
   while ((file_data = file_data_vec[j++]))
     {
       const char *data;
@@ -273,6 +522,9 @@ ipa_profile_read_summary (void)
 	      account_time_size (&hashtable, histogram,
 				 count, time, size);
 	    }
+
+	  ipa_profile_read_summary_section (file_data, ib);
+
 	  lto_destroy_simple_input_block (file_data,
 					  LTO_section_ipa_profile,
 					  ib, data, len);
@@ -512,6 +764,7 @@ ipa_profile (void)
   int nindirect = 0, ncommon = 0, nunknown = 0, nuseless = 0, nconverted = 0;
   int nmismatch = 0, nimpossible = 0;
   bool node_map_initialized = false;
+  gcov_type threshold;
 
   if (dump_file)
     dump_histogram (dump_file, histogram);
@@ -520,14 +773,12 @@ ipa_profile (void)
       overall_time += histogram[i]->count * histogram[i]->time;
       overall_size += histogram[i]->size;
     }
+  threshold = 0;
   if (overall_time)
     {
-      gcov_type threshold;
-
       gcc_assert (overall_size);
 
       cutoff = (overall_time * param_hot_bb_count_ws_permille + 500) / 1000;
-      threshold = 0;
       for (i = 0; cumulated < cutoff; i++)
 	{
 	  cumulated += histogram[i]->count * histogram[i]->time;
@@ -563,10 +814,21 @@ ipa_profile (void)
   histogram.release ();
   histogram_pool.release ();
 
-  /* Produce speculative calls: we saved common target from porfiling into
-     e->common_target_id.  Now, at link time, we can look up corresponding
+  /* Produce speculative calls: we saved common target from profiling into
+     e->target_id.  Now, at link time, we can look up corresponding
      function node and produce speculative call.  */
 
+  gcc_checking_assert (call_sums);
+
+  if (dump_file)
+    {
+      if (!node_map_initialized)
+	init_node_map (false);
+      node_map_initialized = true;
+
+      ipa_profile_dump_all_summaries (dump_file);
+    }
+
   FOR_EACH_DEFINED_FUNCTION (n)
     {
       bool update = false;
@@ -578,13 +840,35 @@ ipa_profile (void)
 	{
 	  if (n->count.initialized_p ())
 	    nindirect++;
-	  if (e->indirect_info->common_target_id)
+
+	  speculative_call_summary *csum = call_sums->get_create (e);
+	  unsigned spec_count = csum->speculative_call_targets.length ();
+	  if (spec_count)
 	    {
 	      if (!node_map_initialized)
-	        init_node_map (false);
+		init_node_map (false);
 	      node_map_initialized = true;
 	      ncommon++;
-	      n2 = find_func_by_profile_id (e->indirect_info->common_target_id);
+
+	      if (in_lto_p)
+		{
+		  if (dump_file)
+		    {
+		      fprintf (dump_file,
+			       "Updating hotness threshold in LTO mode.\n");
+		      fprintf (dump_file, "Updated min count: %" PRId64 "\n",
+			       (int64_t) threshold / spec_count);
+		    }
+		  set_hot_bb_threshold (threshold / spec_count);
+		}
+
+	      unsigned speculative_id = 0;
+	      bool speculative_found = false;
+	      for (unsigned i = 0; i < spec_count; i++)
+	      {
+	      speculative_call_target item
+		= csum->speculative_call_targets[i];
+	      n2 = find_func_by_profile_id (item.target_id);
 	      if (n2)
 		{
 		  if (dump_file)
@@ -593,11 +877,10 @@ ipa_profile (void)
 			       " other module %s => %s, prob %3.2f\n",
 			       n->dump_name (),
 			       n2->dump_name (),
-			       e->indirect_info->common_target_probability
-			       / (float)REG_BR_PROB_BASE);
+			       item.target_probability
+				 / (float) REG_BR_PROB_BASE);
 		    }
-		  if (e->indirect_info->common_target_probability
-		      < REG_BR_PROB_BASE / 2)
+		  if (item.target_probability < REG_BR_PROB_BASE / 2)
 		    {
 		      nuseless++;
 		      if (dump_file)
@@ -653,20 +936,26 @@ ipa_profile (void)
 			    n2 = alias;
 			}
 		      nconverted++;
-		      e->make_speculative
-			(n2,
-			 e->count.apply_probability
-				     (e->indirect_info->common_target_probability));
+		      e->make_speculative (n2,
+					   e->count.apply_probability (
+					     item.target_probability),
+					   speculative_id,
+					   item.target_probability);
 		      update = true;
+		      speculative_id++;
+		      speculative_found = true;
 		    }
 		}
 	      else
 		{
 		  if (dump_file)
 		    fprintf (dump_file, "Function with profile-id %i not found.\n",
-			     e->indirect_info->common_target_id);
+			     item.target_id);
 		  nunknown++;
 		}
+	       }
+	     if (speculative_found)
+	       e->indirect_info->num_speculative_call_targets = speculative_id;
 	    }
 	 }
        if (update)
@@ -729,6 +1018,10 @@ ipa_profile (void)
 	}
     }
   free (order);
+
+  if (dump_file && (dump_flags & TDF_DETAILS))
+    symtab->dump (dump_file);
+
   return 0;
 }
 
diff --git a/gcc/ipa-ref.h b/gcc/ipa-ref.h
index f4ea7945338..1de5bd34b82 100644
--- a/gcc/ipa-ref.h
+++ b/gcc/ipa-ref.h
@@ -59,6 +59,9 @@ public:
   symtab_node *referred;
   gimple *stmt;
   unsigned int lto_stmt_uid;
+  /* speculative id is used to link direct calls with their corresponding
+     IPA_REF_ADDR references when representing speculative calls.  */
+  unsigned int speculative_id : 16;
   unsigned int referred_index;
   ENUM_BITFIELD (ipa_ref_use) use:3;
   unsigned int speculative:1;
diff --git a/gcc/ipa-utils.c b/gcc/ipa-utils.c
index 2e8298045fa..587ad5c6360 100644
--- a/gcc/ipa-utils.c
+++ b/gcc/ipa-utils.c
@@ -675,68 +675,133 @@ ipa_merge_profiles (struct cgraph_node *dst,
 	   e2 = (e2 ? e2->next_callee : NULL), e = e->next_callee)
 	{
 	  profile_count count = gimple_bb (e->call_stmt)->count;
+	  /* Below code are introduced by r279373 of "Fix merging of common
+	     traget info.".
+
+	     ipa-icf runs after ipa-profile, common_target_id with
+	     common_target_probablity are useless in ipa-icf since they are
+	     moved from cgraph.h to ipa-profile.c and processed already.
+	     Need double circulation to find out each mapped direct speculative
+	     edge and do prob merge.  Not easy to construct a case to cover all
+	     circumstances here.  For src and dst both have multiple speculative
+	     targets, only N:N maps are implemented, 2:0, 2:1, 0:2, 1:2 are not
+	     implemented yet as too complicated and no test cases to cover.  */
 	  if (copy_counts)
 	    {
-	      e->indirect_info->common_target_id
-		      = e2->indirect_info->common_target_id;
-	      e->indirect_info->common_target_probability
-		      = e2->indirect_info->common_target_probability;
+	      /* copy if both e and e2 have same num_speculative_call_targets.
+	       */
+	      if (e->num_speculative_call_targets_p ()
+		  == e2->num_speculative_call_targets_p ())
+		{
+		  int num_specs = e->num_speculative_call_targets_p ();
+		  cgraph_edge *direct, *indirect, *next_direct;
+		  cgraph_edge *direct2, *indirect2, *next_direct2;
+		  ipa_ref *ref;
+		  for (next_direct = e; next_direct && num_specs--;
+		       next_direct = direct->next_callee)
+		    {
+		      next_direct->speculative_call_info (direct, indirect,
+							  ref);
+
+		      int num_specs2 = e2->num_speculative_call_targets_p ();
+		      for (next_direct2 = e2; next_direct2 && num_specs2--;
+			   next_direct2 = direct2->next_callee)
+			{
+			  if (e2 && e2->speculative)
+			    next_direct2->speculative_call_info (direct2,
+								 indirect2,
+								 ref);
+			  if (direct->speculative_id == direct2->speculative_id
+			      && direct->lto_stmt_uid == direct2->lto_stmt_uid)
+			    {
+			      direct->target_prob = direct2->target_prob;
+			      break;
+			    }
+			}
+		    }
+		}
+	      else
+		gcc_assert (e->num_speculative_call_targets_p ()
+			    && e->num_speculative_call_targets_p ());
 	    }
-	  else if (e->indirect_info->common_target_id
-		   || e2->indirect_info->common_target_id)
+	  else if (e->num_speculative_call_targets_p ()
+		   || e2->num_speculative_call_targets_p ())
 	    {
-	      sreal scale1
-		 = e->count.ipa().to_sreal_scale (count);
-	      sreal scale2
-		 = e2->count.ipa().to_sreal_scale (count);
-
-	      if (scale1 == 0 && scale2 == 0)
-		scale1 = scale2 = 1;
-	      sreal sum = scale1 + scale2;
-	      int scaled_probability1
-		      = ((sreal)e->indirect_info->common_target_probability
-			* scale1 / sum).to_int ();
-	      int scaled_probability2
-		      = ((sreal)e2->indirect_info->common_target_probability
-			 * scale2 / sum).to_int ();
-	      if (symtab->dump_file)
+	      if (e->num_speculative_call_targets_p ()
+		  == e2->num_speculative_call_targets_p ())
 		{
-		  fprintf (symtab->dump_file,
-			   "Merging common targets %i prob %i"
-			   " and %i prob %i with scales %f %f\n",
-			   e->indirect_info->common_target_id,
-			   e->indirect_info->common_target_probability,
-			   e2->indirect_info->common_target_id,
-			   e2->indirect_info->common_target_probability,
-			   scale1.to_double (),
-			   scale2.to_double ());
-		  fprintf (symtab->dump_file, "Combined BB count ");
-		  count.dump (symtab->dump_file);
-		  fprintf (symtab->dump_file, " dst edge count ");
-		  e->count.dump (symtab->dump_file);
-		  fprintf (symtab->dump_file, " src edge count ");
-		  e2->count.dump (symtab->dump_file);
-		  fprintf (symtab->dump_file, "\n");
+		  int num_specs = e->num_speculative_call_targets_p ();
+		  cgraph_edge *direct, *indirect, *next_direct;
+		  cgraph_edge *direct2, *indirect2, *next_direct2;
+		  ipa_ref *ref;
+		  for (next_direct = e; next_direct && num_specs--;
+		       next_direct = direct->next_callee)
+		    {
+		      next_direct->speculative_call_info (direct, indirect,
+							  ref);
+
+		      int num_specs2 = e2->num_speculative_call_targets_p ();
+		      for (next_direct2 = e2; next_direct2 && num_specs2--;
+			   next_direct2 = direct2->next_callee)
+			{
+			  if (e2 && e2->speculative)
+			    next_direct2->speculative_call_info (direct2,
+								 indirect2,
+								 ref);
+			  if (direct->speculative_id == direct2->speculative_id
+			      && direct->lto_stmt_uid == direct2->lto_stmt_uid)
+			    {
+			      sreal scale1
+				= e->count.ipa ().to_sreal_scale (count);
+			      sreal scale2
+				= e2->count.ipa ().to_sreal_scale (count);
+
+			      if (scale1 == 0 && scale2 == 0)
+				scale1 = scale2 = 1;
+			      sreal sum = scale1 + scale2;
+			      int scaled_prob1
+				= (((sreal)direct->target_prob)
+				   * scale1 / sum).to_int ();
+			      int scaled_prob2
+				= (((sreal)direct2->target_prob)
+				   * scale2 / sum).to_int ();
+			      if (symtab->dump_file)
+				{
+				  fprintf (
+				    symtab->dump_file,
+				    "Merging speculative id %i prob %i"
+				    " and %i prob %i with scales %f %f\n",
+				    direct->speculative_id, direct->target_prob,
+				    direct2->speculative_id,
+				    direct2->target_prob, scale1.to_double (),
+				    scale2.to_double ());
+				  fprintf (symtab->dump_file,
+					   "Combined BB count ");
+				  count.dump (symtab->dump_file);
+				  fprintf (symtab->dump_file,
+					   " dst edge count ");
+				  e->count.dump (symtab->dump_file);
+				  fprintf (symtab->dump_file,
+					   " src edge count ");
+				  e2->count.dump (symtab->dump_file);
+				  fprintf (symtab->dump_file, "\n");
+				}
+			      direct->target_prob = scaled_prob1 + scaled_prob2;
+			      break;
+			    }
+			}
+		    }
 		}
-	      if (e->indirect_info->common_target_id
-		  == e2->indirect_info->common_target_id)
-		e->indirect_info->common_target_probability
-		       	= scaled_probability1 + scaled_probability2;
-	      else if (!e2->indirect_info->common_target_id
-		       || scaled_probability1 > scaled_probability2)
-		e->indirect_info->common_target_probability
-		       	= scaled_probability1;
-	      else 
+	      else if (e->num_speculative_call_targets_p ())
 		{
-		  e->indirect_info->common_target_id
-			  = e2->indirect_info->common_target_id;
-		  e->indirect_info->common_target_probability
-			  = scaled_probability2;
+		  /* Process if only dst is speculative.  */
+		  gcc_assert (!e->num_speculative_call_targets_p ());
+		}
+	      else if (e2->num_speculative_call_targets_p ())
+		{
+		  /* Process if only src is speculative.  */
+		  gcc_assert (!e2->num_speculative_call_targets_p ());
 		}
-	      if (symtab->dump_file)
-		fprintf (symtab->dump_file, "Merged as %i prob %i\n",
-			 e->indirect_info->common_target_id,
-			 e->indirect_info->common_target_probability);
 	    }
 
 	  /* When call is speculative, we need to re-distribute probabilities
diff --git a/gcc/lto-cgraph.c b/gcc/lto-cgraph.c
index dd605f65918..621607499e1 100644
--- a/gcc/lto-cgraph.c
+++ b/gcc/lto-cgraph.c
@@ -262,6 +262,7 @@ lto_output_edge (struct lto_simple_output_block *ob, struct cgraph_edge *edge,
   bp_pack_enum (&bp, cgraph_inline_failed_t,
 	        CIF_N_REASONS, edge->inline_failed);
   bp_pack_var_len_unsigned (&bp, uid);
+  bp_pack_value (&bp, edge->speculative_id, 16);
   bp_pack_value (&bp, edge->indirect_inlining_edge, 1);
   bp_pack_value (&bp, edge->speculative, 1);
   bp_pack_value (&bp, edge->call_stmt_cannot_inline_p, 1);
@@ -284,16 +285,11 @@ lto_output_edge (struct lto_simple_output_block *ob, struct cgraph_edge *edge,
 			     | ECF_SIBCALL
 			     | ECF_LEAF
 			     | ECF_NOVOPS)));
+
+      bp_pack_value (&bp, edge->indirect_info->num_speculative_call_targets,
+		     16);
     }
   streamer_write_bitpack (&bp);
-  if (edge->indirect_unknown_callee)
-    {
-      streamer_write_hwi_stream (ob->main_stream,
-			         edge->indirect_info->common_target_id);
-      if (edge->indirect_info->common_target_id)
-	streamer_write_hwi_stream
-	   (ob->main_stream, edge->indirect_info->common_target_probability);
-    }
 }
 
 /* Return if NODE contain references from other partitions.  */
@@ -690,6 +686,8 @@ lto_output_ref (struct lto_simple_output_block *ob, struct ipa_ref *ref,
       if (ref->stmt)
 	uid = gimple_uid (ref->stmt) + 1;
       streamer_write_hwi_stream (ob->main_stream, uid);
+      bp_pack_value (&bp, ref->speculative_id, 16);
+      streamer_write_bitpack (&bp);
     }
 }
 
@@ -1428,7 +1426,11 @@ input_ref (class lto_input_block *ib,
   ref = referring_node->create_reference (node, use);
   ref->speculative = speculative;
   if (is_a <cgraph_node *> (referring_node))
-    ref->lto_stmt_uid = streamer_read_hwi (ib);
+    {
+      ref->lto_stmt_uid = streamer_read_hwi (ib);
+      bp = streamer_read_bitpack (ib);
+      ref->speculative_id = bp_unpack_value (&bp, 16);
+    }
 }
 
 /* Read an edge from IB.  NODES points to a vector of previously read nodes for
@@ -1442,7 +1444,7 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes,
 {
   struct cgraph_node *caller, *callee;
   struct cgraph_edge *edge;
-  unsigned int stmt_id;
+  unsigned int stmt_id, speculative_id;
   profile_count count;
   cgraph_inline_failed_t inline_failed;
   struct bitpack_d bp;
@@ -1466,6 +1468,7 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes,
   bp = streamer_read_bitpack (ib);
   inline_failed = bp_unpack_enum (&bp, cgraph_inline_failed_t, CIF_N_REASONS);
   stmt_id = bp_unpack_var_len_unsigned (&bp);
+  speculative_id = bp_unpack_value (&bp, 16);
 
   if (indirect)
     edge = caller->create_indirect_edge (NULL, 0, count);
@@ -1475,6 +1478,7 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes,
   edge->indirect_inlining_edge = bp_unpack_value (&bp, 1);
   edge->speculative = bp_unpack_value (&bp, 1);
   edge->lto_stmt_uid = stmt_id;
+  edge->speculative_id = speculative_id;
   edge->inline_failed = inline_failed;
   edge->call_stmt_cannot_inline_p = bp_unpack_value (&bp, 1);
   edge->can_throw_external = bp_unpack_value (&bp, 1);
@@ -1494,9 +1498,9 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes,
       if (bp_unpack_value (&bp, 1))
 	ecf_flags |= ECF_RETURNS_TWICE;
       edge->indirect_info->ecf_flags = ecf_flags;
-      edge->indirect_info->common_target_id = streamer_read_hwi (ib);
-      if (edge->indirect_info->common_target_id)
-        edge->indirect_info->common_target_probability = streamer_read_hwi (ib);
+
+      edge->indirect_info->num_speculative_call_targets
+	= bp_unpack_value (&bp, 16);
     }
 }
 
diff --git a/gcc/predict.c b/gcc/predict.c
index a134deb9b3f..02c5fe0667d 100644
--- a/gcc/predict.c
+++ b/gcc/predict.c
@@ -761,7 +761,6 @@ dump_prediction (FILE *file, enum br_predictor predictor, int probability,
       && bb->count.precise_p ()
       && reason == REASON_NONE)
     {
-      gcc_assert (e->count ().precise_p ());
       fprintf (file, ";;heuristics;%s;%" PRId64 ";%" PRId64 ";%.1f;\n",
 	       predictor_info[predictor].name,
 	       bb->count.to_gcov_type (), e->count ().to_gcov_type (),
diff --git a/gcc/symtab.c b/gcc/symtab.c
index 1b1b0ee409d..f141200a452 100644
--- a/gcc/symtab.c
+++ b/gcc/symtab.c
@@ -605,6 +605,7 @@ symtab_node::create_reference (symtab_node *referred_node,
   ref->referred = referred_node;
   ref->stmt = stmt;
   ref->lto_stmt_uid = 0;
+  ref->speculative_id = 0;
   ref->use = use_type;
   ref->speculative = 0;
 
@@ -662,6 +663,7 @@ symtab_node::clone_references (symtab_node *node)
       ref2 = create_reference (ref->referred, ref->use, ref->stmt);
       ref2->speculative = speculative;
       ref2->lto_stmt_uid = stmt_uid;
+      ref2->speculative_id = ref->speculative_id;
     }
 }
 
@@ -680,6 +682,7 @@ symtab_node::clone_referring (symtab_node *node)
       ref2 = ref->referring->create_reference (this, ref->use, ref->stmt);
       ref2->speculative = speculative;
       ref2->lto_stmt_uid = stmt_uid;
+      ref2->speculative_id = ref->speculative_id;
     }
 }
 
@@ -695,6 +698,7 @@ symtab_node::clone_reference (ipa_ref *ref, gimple *stmt)
   ref2 = create_reference (ref->referred, ref->use, stmt);
   ref2->speculative = speculative;
   ref2->lto_stmt_uid = stmt_uid;
+  ref2->speculative_id = ref->speculative_id;
   return ref2;
 }
 
@@ -749,6 +753,7 @@ symtab_node::clear_stmts_in_references (void)
       {
 	r->stmt = NULL;
 	r->lto_stmt_uid = 0;
+	r->speculative_id = 0;
       }
 }
 
diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog
index 8fc0bfd7c5a..86e954fb84d 100644
--- a/gcc/testsuite/ChangeLog
+++ b/gcc/testsuite/ChangeLog
@@ -1,3 +1,13 @@
+2020-01-14  Xiong Hu Luo  <luoxhu@linux.ibm.com>
+
+	PR ipa/69678
+	* gcc.dg/tree-prof/indir-call-prof-topn.c: New testcase.
+	* gcc.dg/tree-prof/crossmodule-indir-call-topn-1.c: New testcase.
+	* gcc.dg/tree-prof/crossmodule-indir-call-topn-1a.c: New testcase.
+	* gcc.dg/tree-prof/crossmodule-indir-call-topn-2.c: New testcase.
+	* lib/scandump.exp: Dump executable file name.
+	* lib/scanwpaipa.exp: New scan-pgo-wap-ipa-dump.
+
 2020-01-13  David Malcolm  <dmalcolm@redhat.com>
 
 	* gcc.dg/plugin/diagnostic-test-paths-2.c: Remove unused dg-line
diff --git a/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-1.c b/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-1.c
new file mode 100644
index 00000000000..a13b08cd60e
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-1.c
@@ -0,0 +1,33 @@
+/* { dg-require-effective-target lto } */
+/* { dg-additional-sources "crossmodule-indir-call-topn-1a.c" } */
+/* { dg-require-profiling "-fprofile-generate" } */
+/* { dg-options "-O2 -flto -DDOJOB=1 -fdump-ipa-profile_estimate" } */
+
+#include <stdio.h>
+
+typedef int (*fptr) (int);
+int
+one (int a);
+
+int
+two (int a);
+
+fptr table[] = {&one, &two};
+
+int
+main()
+{
+  int i, x;
+  fptr p = &one;
+
+  x = one (3);
+
+  for (i = 0; i < 350000000; i++)
+    {
+      x = (*p) (3);
+      p = table[x];
+    }
+  printf ("done:%d\n", x);
+}
+
+/* { dg-final-use-not-autofdo { scan-pgo-wpa-ipa-dump "2 \\(200.00%\\) speculations produced." "profile_estimate" } } */
diff --git a/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-1a.c b/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-1a.c
new file mode 100644
index 00000000000..a8c6e365fb9
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-1a.c
@@ -0,0 +1,22 @@
+/* It seems there is no way to avoid the other source of mulitple
+   source testcase from being compiled independently.  Just avoid
+   error.  */
+#ifdef DOJOB
+int
+one (int a)
+{
+  return 1;
+}
+
+int
+two (int a)
+{
+  return 0;
+}
+#else
+int
+main()
+{
+  return 0;
+}
+#endif
diff --git a/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-2.c b/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-2.c
new file mode 100644
index 00000000000..9b996fcf0ed
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/tree-prof/crossmodule-indir-call-topn-2.c
@@ -0,0 +1,40 @@
+/* { dg-require-effective-target lto } */
+/* { dg-additional-sources "crossmodule-indir-call-topn-1a.c" } */
+/* { dg-require-profiling "-fprofile-generate" } */
+/* { dg-options "-O2 -flto -DDOJOB=1 -fdump-ipa-profile_estimate" } */
+
+#include <stdio.h>
+
+typedef int (*fptr) (int);
+int
+one (int a);
+
+int
+two (int a);
+
+fptr table[] = {&one, &two};
+
+int foo ()
+{
+  int i, x;
+  fptr p = &one;
+
+  x = one (3);
+
+  for (i = 0; i < 350000000; i++)
+    {
+      x = (*p) (3);
+      p = table[x];
+    }
+  return x;
+}
+
+int
+main()
+{
+  int x = foo ();
+  printf ("done:%d\n", x);
+}
+
+/* { dg-final-use-not-autofdo { scan-pgo-wpa-ipa-dump "2 \\(200.00%\\) speculations produced." "profile_estimate" } } */
+
diff --git a/gcc/testsuite/gcc.dg/tree-prof/indir-call-prof-topn.c b/gcc/testsuite/gcc.dg/tree-prof/indir-call-prof-topn.c
new file mode 100644
index 00000000000..063996c71df
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/tree-prof/indir-call-prof-topn.c
@@ -0,0 +1,37 @@
+/* { dg-require-profiling "-fprofile-generate" } */
+/* { dg-options "-O2 -fdump-ipa-profile_estimate" } */
+
+#include <stdio.h>
+
+typedef int (*fptr) (int);
+int
+one (int a)
+{
+  return 1;
+}
+
+int
+two (int a)
+{
+  return 0;
+}
+
+fptr table[] = {&one, &two};
+
+int
+main()
+{
+  int i, x;
+  fptr p = &one;
+
+  one (3);
+
+  for (i = 0; i < 350000000; i++)
+    {
+      x = (*p) (3);
+      p = table[x];
+    }
+  printf ("done:%d\n", x);
+}
+
+/* { dg-final-use-not-autofdo { scan-ipa-dump "2 \\(200.00%\\) speculations produced." "profile_estimate" } } */
diff --git a/gcc/testsuite/lib/scandump.exp b/gcc/testsuite/lib/scandump.exp
index 4c7d904cf77..d6ba350acc8 100644
--- a/gcc/testsuite/lib/scandump.exp
+++ b/gcc/testsuite/lib/scandump.exp
@@ -70,6 +70,7 @@ proc scan-dump { args } {
     set output_file "[glob -nocomplain $dumpbase.[lindex $args 2]]"
     if { $output_file == "" } {
 	verbose -log "$testcase: dump file does not exist"
+	verbose -log "dump file: $dumpbase.$suf"
 	unresolved "$testname"
 	return
     }
diff --git a/gcc/testsuite/lib/scanwpaipa.exp b/gcc/testsuite/lib/scanwpaipa.exp
index d00b400dd42..2f58823a4f3 100644
--- a/gcc/testsuite/lib/scanwpaipa.exp
+++ b/gcc/testsuite/lib/scanwpaipa.exp
@@ -45,6 +45,29 @@ proc scan-wpa-ipa-dump { args } {
     }
 }
 
+# Argument 0 is the regexp to match
+# Argument 1 is the name of the dumped ipa pass
+# Argument 2 handles expected failures and the like
+proc scan-pgo-wpa-ipa-dump { args } {
+
+    if { [llength $args] < 2 } {
+	error "scan-pgo-wpa-ipa-dump: too few arguments"
+	return
+    }
+    if { [llength $args] > 3 } {
+	error "scan-pgo-wpa-ipa-dump: too many arguments"
+	return
+    }
+    if { [llength $args] >= 3 } {
+	scan-dump "pgo-wpa-ipa" [lindex $args 0] \
+		  "\[0-9\]\[0-9\]\[0-9\]i.[lindex $args 1]" ".x02.wpa" \
+		  [lindex $args 2]
+    } else {
+	scan-dump "pgo-wpa-ipa" [lindex $args 0] \
+		  "\[0-9\]\[0-9\]\[0-9\]i.[lindex $args 1]" ".x02.wpa"
+    }
+}
+
 # Call pass if pattern is present given number of times, otherwise fail.
 # Argument 0 is the regexp to match
 # Argument 1 is number of times the regexp must be found
diff --git a/gcc/tree-inline.c b/gcc/tree-inline.c
index 51aa06566b1..2197769bf17 100644
--- a/gcc/tree-inline.c
+++ b/gcc/tree-inline.c
@@ -2187,9 +2187,10 @@ copy_bb (copy_body_data *id, basic_block bb,
 					  num, den,
 					  true);
 
-		      /* Speculative calls consist of two edges - direct and
-			 indirect.  Duplicate the whole thing and distribute
-			 frequencies accordingly.  */
+		      /* A speculative call is consist of edges - indirect edge
+			 and direct edges (one indirect edeg may has multiple
+			 direct edges).  Duplicate the whole thing and
+			 distribute frequencies accordingly.  */
 		      if (edge->speculative)
 			{
 			  struct cgraph_edge *direct, *indirect;
@@ -2197,8 +2198,33 @@ copy_bb (copy_body_data *id, basic_block bb,
 
 			  gcc_assert (!edge->indirect_unknown_callee);
 			  old_edge->speculative_call_info (direct, indirect, ref);
+			  while (old_edge->next_callee
+				 && old_edge->next_callee->speculative
+				 && indirect->num_speculative_call_targets_p ()
+				      > 1)
+			    {
+			      id->dst_node->clone_reference (ref, stmt);
+
+			      edge = old_edge->next_callee;
+			      edge = edge->clone (id->dst_node, call_stmt,
+						  gimple_uid (stmt), num, den,
+						  true);
+			      old_edge = old_edge->next_callee;
+			      gcc_assert (!edge->indirect_unknown_callee);
+
+			      /* If the indirect edge has multiple speculative
+				 calls, iterate through all direct calls
+				 associated to the speculative call and clone
+				 all related direct edges before cloning the
+				 related indirect edge.  */
+			      old_edge->speculative_call_info (direct, indirect,
+							       ref);
+			    }
 
 			  profile_count indir_cnt = indirect->count;
+
+			  /* Duplicate the indirect edge after all direct edges
+			     cloned.  */
 			  indirect = indirect->clone (id->dst_node, call_stmt,
 						      gimple_uid (stmt),
 						      num, den,
diff --git a/gcc/value-prof.c b/gcc/value-prof.c
index 81994964148..9a2c46252c6 100644
--- a/gcc/value-prof.c
+++ b/gcc/value-prof.c
@@ -106,7 +106,7 @@ static bool gimple_divmod_fixed_value_transform (gimple_stmt_iterator *);
 static bool gimple_mod_pow2_value_transform (gimple_stmt_iterator *);
 static bool gimple_mod_subtract_transform (gimple_stmt_iterator *);
 static bool gimple_stringops_transform (gimple_stmt_iterator *);
-static bool gimple_ic_transform (gimple_stmt_iterator *);
+static void gimple_ic_transform (gimple_stmt_iterator *);
 
 /* Allocate histogram value.  */
 
@@ -616,8 +616,7 @@ gimple_value_profile_transformations (void)
 	  if (gimple_mod_subtract_transform (&gsi)
 	      || gimple_divmod_fixed_value_transform (&gsi)
 	      || gimple_mod_pow2_value_transform (&gsi)
-	      || gimple_stringops_transform (&gsi)
-	      || gimple_ic_transform (&gsi))
+	      || gimple_stringops_transform (&gsi))
 	    {
 	      stmt = gsi_stmt (gsi);
 	      changed = true;
@@ -628,6 +627,9 @@ gimple_value_profile_transformations (void)
 		  gsi = gsi_for_stmt (stmt);
 		}
 	    }
+
+	  /* The function never thansforms a GIMPLE statement.  */
+	  gimple_ic_transform (&gsi);
         }
     }
 
@@ -1386,13 +1388,12 @@ gimple_ic (gcall *icall_stmt, struct cgraph_node *direct_call,
   return dcall_stmt;
 }
 
-/*
-  For every checked indirect/virtual call determine if most common pid of
-  function/class method has probability more than 50%. If yes modify code of
-  this call to:
- */
+/* There maybe multiple indirect targets in histogram.  Check every
+   indirect/virtual call if callee function exists, if not exist, leave it to
+   LTO stage for later process.  Modify code of this indirect call to an if-else
+   structure in ipa-profile finally.  */
 
-static bool
+static void
 gimple_ic_transform (gimple_stmt_iterator *gsi)
 {
   gcall *stmt;
@@ -1402,52 +1403,58 @@ gimple_ic_transform (gimple_stmt_iterator *gsi)
 
   stmt = dyn_cast <gcall *> (gsi_stmt (*gsi));
   if (!stmt)
-    return false;
+    return;
 
   if (gimple_call_fndecl (stmt) != NULL_TREE)
-    return false;
+    return;
 
   if (gimple_call_internal_p (stmt))
-    return false;
+    return;
 
   histogram = gimple_histogram_value_of_type (cfun, stmt, HIST_TYPE_INDIR_CALL);
   if (!histogram)
-    return false;
+    return;
 
-  if (!get_nth_most_common_value (NULL, "indirect call", histogram, &val,
-				  &count, &all))
-    return false;
+  count = 0;
+  all = histogram->hvalue.counters[0];
 
-  if (4 * count <= 3 * all)
-    return false;
+  for (unsigned j = 0; j < GCOV_TOPN_VALUES; j++)
+    {
+      if (!get_nth_most_common_value (NULL, "indirect call", histogram, &val,
+				      &count, &all, j))
+	return;
 
-  direct_call = find_func_by_profile_id ((int)val);
+      /* Minimum probability.  should be higher than 25%.  */
+      if (4 * count <= all)
+	return;
 
-  if (direct_call == NULL)
-    {
-      if (val)
+      direct_call = find_func_by_profile_id ((int) val);
+
+      if (direct_call == NULL)
 	{
-	  if (dump_enabled_p ())
-	    dump_printf_loc (MSG_MISSED_OPTIMIZATION, stmt,
-			     "Indirect call -> direct call from other "
-			     "module %T=> %i (will resolve only with LTO)\n",
-			     gimple_call_fn (stmt), (int)val);
+	  if (val)
+	    {
+	      if (dump_enabled_p ())
+		dump_printf_loc (
+		  MSG_MISSED_OPTIMIZATION, stmt,
+		  "Indirect call -> direct call from other "
+		  "module %T=> %i (will resolve only with LTO)\n",
+		  gimple_call_fn (stmt), (int) val);
+	    }
+	  return;
 	}
-      return false;
-    }
 
-  if (dump_enabled_p ())
-    {
-      dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, stmt,
-		       "Indirect call -> direct call "
-		       "%T => %T transformation on insn postponed\n",
-		       gimple_call_fn (stmt), direct_call->decl);
-      dump_printf_loc (MSG_NOTE, stmt,
-		       "hist->count %" PRId64
-		       " hist->all %" PRId64"\n", count, all);
+      if (dump_enabled_p ())
+	{
+	  dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, stmt,
+			   "Indirect call -> direct call "
+			   "%T => %T transformation on insn postponed\n",
+			   gimple_call_fn (stmt), direct_call->decl);
+	  dump_printf_loc (MSG_NOTE, stmt,
+			   "hist->count %" PRId64 " hist->all %" PRId64 "\n",
+			   count, all);
+	}
     }
-
-  return true;
 }
 
 /* Return true if the stringop CALL shall be profiled.  SIZE_ARG be
diff --git a/gcc/value-prof.h b/gcc/value-prof.h
index bf024a3854c..81c615dadec 100644
--- a/gcc/value-prof.h
+++ b/gcc/value-prof.h
@@ -89,7 +89,6 @@ void verify_histograms (void);
 void free_histograms (function *);
 void stringop_block_profile (gimple *, unsigned int *, HOST_WIDE_INT *);
 gcall *gimple_ic (gcall *, struct cgraph_node *, profile_probability);
-bool check_ic_target (gcall *, struct cgraph_node *);
 bool get_nth_most_common_value (gimple *stmt, const char *counter_type,
 				histogram_value hist, gcov_type *value,
 				gcov_type *count, gcov_type *all,