Remove vec_perm_const optab

One of the changes needed for variable-length VEC_PERM_EXPRs -- and for
long fixed-length VEC_PERM_EXPRs -- is the ability to use constant
selectors that wouldn't fit in the vectors being permuted.  E.g. a
permute on two V256QIs can't be done using a V256QI selector.

At the moment constant permutes use two interfaces:
targetm.vectorizer.vec_perm_const_ok for testing whether a permute is
valid and the vec_perm_const optab for actually emitting the permute.
The former gets passed a vec<> selector and the latter an rtx selector.
Most ports share a lot of code between the hook and the optab, with a
wrapper function for each interface.

We could try to keep that interface and require ports to define wider
vector modes that could be attached to the CONST_VECTOR (e.g. V256HI or
V256SI in the example above).  But building a CONST_VECTOR rtx seems a bit
pointless here, since the expand code only creates the CONST_VECTOR in
order to call the optab, and the first thing the target does is take
the CONST_VECTOR apart again.

The easiest approach therefore seemed to be to remove the optab and
reuse the target hook to emit the code.  One potential drawback is that
it's no longer possible to use match_operand predicates to force
operands into the required form, but in practice all targets want
register operands anyway.

The patch also changes vec_perm_indices into a class that provides
some simple routines for handling permutations.  A later patch will
flesh this out and get rid of auto_vec_perm_indices, but I didn't
want to do all that in this patch and make it more complicated than
it already is.

2018-01-02  Richard Sandiford  <richard.sandiford@linaro.org>

gcc/
	* Makefile.in (OBJS): Add vec-perm-indices.o.
	* vec-perm-indices.h: New file.
	* vec-perm-indices.c: Likewise.
	* target.h (vec_perm_indices): Replace with a forward class
	declaration.
	(auto_vec_perm_indices): Move to vec-perm-indices.h.
	* optabs.h: Include vec-perm-indices.h.
	(expand_vec_perm): Delete.
	(selector_fits_mode_p, expand_vec_perm_var): Declare.
	(expand_vec_perm_const): Declare.
	* target.def (vec_perm_const_ok): Replace with...
	(vec_perm_const): ...this new hook.
	* doc/tm.texi.in (TARGET_VECTORIZE_VEC_PERM_CONST_OK): Replace with...
	(TARGET_VECTORIZE_VEC_PERM_CONST): ...this new hook.
	* doc/tm.texi: Regenerate.
	* optabs.def (vec_perm_const): Delete.
	* doc/md.texi (vec_perm_const): Likewise.
	(vec_perm): Refer to TARGET_VECTORIZE_VEC_PERM_CONST.
	* expr.c (expand_expr_real_2): Use expand_vec_perm_const rather than
	expand_vec_perm for constant permutation vectors.  Assert that
	the mode of variable permutation vectors is the integer equivalent
	of the mode that is being permuted.
	* optabs-query.h (selector_fits_mode_p): Declare.
	* optabs-query.c: Include vec-perm-indices.h.
	(selector_fits_mode_p): New function.
	(can_vec_perm_const_p): Check whether targetm.vectorize.vec_perm_const
	is defined, instead of checking whether the vec_perm_const_optab
	exists.  Use targetm.vectorize.vec_perm_const instead of
	targetm.vectorize.vec_perm_const_ok.  Check whether the indices
	fit in the vector mode before using a variable permute.
	* optabs.c (shift_amt_for_vec_perm_mask): Take a mode and a
	vec_perm_indices instead of an rtx.
	(expand_vec_perm): Replace with...
	(expand_vec_perm_const): ...this new function.  Take the selector
	as a vec_perm_indices rather than an rtx.  Also take the mode of
	the selector.  Update call to shift_amt_for_vec_perm_mask.
	Use targetm.vectorize.vec_perm_const instead of vec_perm_const_optab.
	Use vec_perm_indices::new_expanded_vector to expand the original
	selector into bytes.  Check whether the indices fit in the vector
	mode before using a variable permute.
	(expand_vec_perm_var): Make global.
	(expand_mult_highpart): Use expand_vec_perm_const.
	* fold-const.c: Includes vec-perm-indices.h.
	* tree-ssa-forwprop.c: Likewise.
	* tree-vect-data-refs.c: Likewise.
	* tree-vect-generic.c: Likewise.
	* tree-vect-loop.c: Likewise.
	* tree-vect-slp.c: Likewise.
	* tree-vect-stmts.c: Likewise.
	* config/aarch64/aarch64-protos.h (aarch64_expand_vec_perm_const):
	Delete.
	* config/aarch64/aarch64-simd.md (vec_perm_const<mode>): Delete.
	* config/aarch64/aarch64.c (aarch64_expand_vec_perm_const)
	(aarch64_vectorize_vec_perm_const_ok): Fuse into...
	(aarch64_vectorize_vec_perm_const): ...this new function.
	(TARGET_VECTORIZE_VEC_PERM_CONST_OK): Delete.
	(TARGET_VECTORIZE_VEC_PERM_CONST): Redefine.
	* config/arm/arm-protos.h (arm_expand_vec_perm_const): Delete.
	* config/arm/vec-common.md (vec_perm_const<mode>): Delete.
	* config/arm/arm.c (TARGET_VECTORIZE_VEC_PERM_CONST_OK): Delete.
	(TARGET_VECTORIZE_VEC_PERM_CONST): Redefine.
	(arm_expand_vec_perm_const, arm_vectorize_vec_perm_const_ok): Merge
	into...
	(arm_vectorize_vec_perm_const): ...this new function.  Explicitly
	check for NEON modes.
	* config/i386/i386-protos.h (ix86_expand_vec_perm_const): Delete.
	* config/i386/sse.md (VEC_PERM_CONST, vec_perm_const<mode>): Delete.
	* config/i386/i386.c (ix86_expand_vec_perm_const_1): Update comment.
	(ix86_expand_vec_perm_const, ix86_vectorize_vec_perm_const_ok): Merge
	into...
	(ix86_vectorize_vec_perm_const): ...this new function.  Incorporate
	the old VEC_PERM_CONST conditions.
	* config/ia64/ia64-protos.h (ia64_expand_vec_perm_const): Delete.
	* config/ia64/vect.md (vec_perm_const<mode>): Delete.
	* config/ia64/ia64.c (ia64_expand_vec_perm_const)
	(ia64_vectorize_vec_perm_const_ok): Merge into...
	(ia64_vectorize_vec_perm_const): ...this new function.
	* config/mips/loongson.md (vec_perm_const<mode>): Delete.
	* config/mips/mips-msa.md (vec_perm_const<mode>): Delete.
	* config/mips/mips-ps-3d.md (vec_perm_constv2sf): Delete.
	* config/mips/mips-protos.h (mips_expand_vec_perm_const): Delete.
	* config/mips/mips.c (mips_expand_vec_perm_const)
	(mips_vectorize_vec_perm_const_ok): Merge into...
	(mips_vectorize_vec_perm_const): ...this new function.
	* config/powerpcspe/altivec.md (vec_perm_constv16qi): Delete.
	* config/powerpcspe/paired.md (vec_perm_constv2sf): Delete.
	* config/powerpcspe/spe.md (vec_perm_constv2si): Delete.
	* config/powerpcspe/vsx.md (vec_perm_const<mode>): Delete.
	* config/powerpcspe/powerpcspe-protos.h (altivec_expand_vec_perm_const)
	(rs6000_expand_vec_perm_const): Delete.
	* config/powerpcspe/powerpcspe.c (TARGET_VECTORIZE_VEC_PERM_CONST_OK):
	Delete.
	(TARGET_VECTORIZE_VEC_PERM_CONST): Redefine.
	(altivec_expand_vec_perm_const_le): Take each operand individually.
	Operate on constant selectors rather than rtxes.
	(altivec_expand_vec_perm_const): Likewise.  Update call to
	altivec_expand_vec_perm_const_le.
	(rs6000_expand_vec_perm_const): Delete.
	(rs6000_vectorize_vec_perm_const_ok): Delete.
	(rs6000_vectorize_vec_perm_const): New function.
	(rs6000_do_expand_vec_perm): Take a vec_perm_builder instead of
	an element count and rtx array.
	(rs6000_expand_extract_even): Update call accordingly.
	(rs6000_expand_interleave): Likewise.
	* config/rs6000/altivec.md (vec_perm_constv16qi): Delete.
	* config/rs6000/paired.md (vec_perm_constv2sf): Delete.
	* config/rs6000/vsx.md (vec_perm_const<mode>): Delete.
	* config/rs6000/rs6000-protos.h (altivec_expand_vec_perm_const)
	(rs6000_expand_vec_perm_const): Delete.
	* config/rs6000/rs6000.c (TARGET_VECTORIZE_VEC_PERM_CONST_OK): Delete.
	(TARGET_VECTORIZE_VEC_PERM_CONST): Redefine.
	(altivec_expand_vec_perm_const_le): Take each operand individually.
	Operate on constant selectors rather than rtxes.
	(altivec_expand_vec_perm_const): Likewise.  Update call to
	altivec_expand_vec_perm_const_le.
	(rs6000_expand_vec_perm_const): Delete.
	(rs6000_vectorize_vec_perm_const_ok): Delete.
	(rs6000_vectorize_vec_perm_const): New function.  Remove stray
	reference to the SPE evmerge intructions.
	(rs6000_do_expand_vec_perm): Take a vec_perm_builder instead of
	an element count and rtx array.
	(rs6000_expand_extract_even): Update call accordingly.
	(rs6000_expand_interleave): Likewise.
	* config/sparc/sparc.md (vec_perm_constv8qi): Delete in favor of...
	* config/sparc/sparc.c (sparc_vectorize_vec_perm_const): ...this
	new function.
	(TARGET_VECTORIZE_VEC_PERM_CONST): Redefine.

From-SVN: r256093

1 files changed, 83 insertions, 78 deletions

diff --git a/gcc/optabs.c b/gcc/optabs.c
index 3549b4a8495..9099ba29143 100644
--- a/gcc/optabs.c
+++ b/gcc/optabs.c
@@ -5387,35 +5387,33 @@ vector_compare_rtx (machine_mode cmp_mode, enum tree_code tcode,
   return gen_rtx_fmt_ee (rcode, cmp_mode, ops[0].value, ops[1].value);
 }
 
-/* Checks if vec_perm mask SEL is a constant equivalent to a shift of the first
-   vec_perm operand, assuming the second operand is a constant vector of zeroes.
-   Return the shift distance in bits if so, or NULL_RTX if the vec_perm is not a
-   shift.  */
+/* Check if vec_perm mask SEL is a constant equivalent to a shift of
+   the first vec_perm operand, assuming the second operand is a constant
+   vector of zeros.  Return the shift distance in bits if so, or NULL_RTX
+   if the vec_perm is not a shift.  MODE is the mode of the value being
+   shifted.  */
 static rtx
-shift_amt_for_vec_perm_mask (rtx sel)
+shift_amt_for_vec_perm_mask (machine_mode mode, const vec_perm_indices &sel)
 {
-  unsigned int i, first, nelt = GET_MODE_NUNITS (GET_MODE (sel));
-  unsigned int bitsize = GET_MODE_UNIT_BITSIZE (GET_MODE (sel));
+  unsigned int i, first, nelt = GET_MODE_NUNITS (mode);
+  unsigned int bitsize = GET_MODE_UNIT_BITSIZE (mode);
 
-  if (GET_CODE (sel) != CONST_VECTOR)
-    return NULL_RTX;
-
-  first = INTVAL (CONST_VECTOR_ELT (sel, 0));
+  first = sel[0];
   if (first >= nelt)
     return NULL_RTX;
   for (i = 1; i < nelt; i++)
     {
-      int idx = INTVAL (CONST_VECTOR_ELT (sel, i));
+      int idx = sel[i];
       unsigned int expected = i + first;
       /* Indices into the second vector are all equivalent.  */
       if (idx < 0 || (MIN (nelt, (unsigned) idx) != MIN (nelt, expected)))
 	return NULL_RTX;
     }
 
-  return gen_int_shift_amount (GET_MODE (sel), first * bitsize);
+  return gen_int_shift_amount (mode, first * bitsize);
 }
 
-/* A subroutine of expand_vec_perm for expanding one vec_perm insn.  */
+/* A subroutine of expand_vec_perm_var for expanding one vec_perm insn.  */
 
 static rtx
 expand_vec_perm_1 (enum insn_code icode, rtx target,
@@ -5453,38 +5451,32 @@ expand_vec_perm_1 (enum insn_code icode, rtx target,
   return NULL_RTX;
 }
 
-static rtx expand_vec_perm_var (machine_mode, rtx, rtx, rtx, rtx);
-
 /* Implement a permutation of vectors v0 and v1 using the permutation
    vector in SEL and return the result.  Use TARGET to hold the result
    if nonnull and convenient.
 
-   MODE is the mode of the vectors being permuted (V0 and V1).  */
+   MODE is the mode of the vectors being permuted (V0 and V1).  SEL_MODE
+   is the TYPE_MODE associated with SEL, or BLKmode if SEL isn't known
+   to have a particular mode.  */
 
 rtx
-expand_vec_perm (machine_mode mode, rtx v0, rtx v1, rtx sel, rtx target)
+expand_vec_perm_const (machine_mode mode, rtx v0, rtx v1,
+		       const vec_perm_builder &sel, machine_mode sel_mode,
+		       rtx target)
 {
-  enum insn_code icode;
-  machine_mode qimode;
-  unsigned int i, w, e, u;
-  rtx tmp, sel_qi = NULL;
-  rtvec vec;
-
-  if (GET_CODE (sel) != CONST_VECTOR)
-    return expand_vec_perm_var (mode, v0, v1, sel, target);
-
-  if (!target || GET_MODE (target) != mode)
+  if (!target || !register_operand (target, mode))
     target = gen_reg_rtx (mode);
 
-  w = GET_MODE_SIZE (mode);
-  e = GET_MODE_NUNITS (mode);
-  u = GET_MODE_UNIT_SIZE (mode);
-
   /* Set QIMODE to a different vector mode with byte elements.
      If no such mode, or if MODE already has byte elements, use VOIDmode.  */
+  machine_mode qimode;
   if (!qimode_for_vec_perm (mode).exists (&qimode))
     qimode = VOIDmode;
 
+  rtx_insn *last = get_last_insn ();
+
+  bool single_arg_p = rtx_equal_p (v0, v1);
+
   /* See if this can be handled with a vec_shr.  We only do this if the
      second vector is all zeroes.  */
   insn_code shift_code = optab_handler (vec_shr_optab, mode);
@@ -5496,7 +5488,7 @@ expand_vec_perm (machine_mode mode, rtx v0, rtx v1, rtx sel, rtx target)
       && (shift_code != CODE_FOR_nothing
 	  || shift_code_qi != CODE_FOR_nothing))
     {
-      rtx shift_amt = shift_amt_for_vec_perm_mask (sel);
+      rtx shift_amt = shift_amt_for_vec_perm_mask (mode, sel);
       if (shift_amt)
 	{
 	  struct expand_operand ops[3];
@@ -5520,65 +5512,81 @@ expand_vec_perm (machine_mode mode, rtx v0, rtx v1, rtx sel, rtx target)
 	}
     }
 
-  icode = direct_optab_handler (vec_perm_const_optab, mode);
-  if (icode != CODE_FOR_nothing)
+  if (targetm.vectorize.vec_perm_const != NULL)
     {
-      tmp = expand_vec_perm_1 (icode, target, v0, v1, sel);
-      if (tmp)
-	return tmp;
+      v0 = force_reg (mode, v0);
+      if (single_arg_p)
+	v1 = v0;
+      else
+	v1 = force_reg (mode, v1);
+
+      if (targetm.vectorize.vec_perm_const (mode, target, v0, v1, sel))
+	return target;
     }
 
   /* Fall back to a constant byte-based permutation.  */
+  vec_perm_indices qimode_indices;
+  rtx target_qi = NULL_RTX, v0_qi = NULL_RTX, v1_qi = NULL_RTX;
   if (qimode != VOIDmode)
     {
-      vec = rtvec_alloc (w);
-      for (i = 0; i < e; ++i)
-	{
-	  unsigned int j, this_e;
+      qimode_indices.new_expanded_vector (sel, GET_MODE_UNIT_SIZE (mode));
+      target_qi = gen_reg_rtx (qimode);
+      v0_qi = gen_lowpart (qimode, v0);
+      v1_qi = gen_lowpart (qimode, v1);
+      if (targetm.vectorize.vec_perm_const != NULL
+	  && targetm.vectorize.vec_perm_const (qimode, target_qi, v0_qi,
+					       v1_qi, qimode_indices))
+	return gen_lowpart (mode, target_qi);
+    }
 
-	  this_e = INTVAL (CONST_VECTOR_ELT (sel, i));
-	  this_e &= 2 * e - 1;
-	  this_e *= u;
+  /* Otherwise expand as a fully variable permuation.  */
 
-	  for (j = 0; j < u; ++j)
-	    RTVEC_ELT (vec, i * u + j) = GEN_INT (this_e + j);
-	}
-      sel_qi = gen_rtx_CONST_VECTOR (qimode, vec);
+  /* The optabs are only defined for selectors with the same width
+     as the values being permuted.  */
+  machine_mode required_sel_mode;
+  if (!mode_for_int_vector (mode).exists (&required_sel_mode)
+      || !VECTOR_MODE_P (required_sel_mode))
+    {
+      delete_insns_since (last);
+      return NULL_RTX;
+    }
 
-      icode = direct_optab_handler (vec_perm_const_optab, qimode);
-      if (icode != CODE_FOR_nothing)
+  /* We know that it is semantically valid to treat SEL as having SEL_MODE.
+     If that isn't the mode we want then we need to prove that using
+     REQUIRED_SEL_MODE is OK.  */
+  if (sel_mode != required_sel_mode)
+    {
+      if (!selector_fits_mode_p (required_sel_mode, sel))
 	{
-	  tmp = gen_reg_rtx (qimode);
-	  tmp = expand_vec_perm_1 (icode, tmp, gen_lowpart (qimode, v0),
-				   gen_lowpart (qimode, v1), sel_qi);
-	  if (tmp)
-	    return gen_lowpart (mode, tmp);
+	  delete_insns_since (last);
+	  return NULL_RTX;
 	}
+      sel_mode = required_sel_mode;
     }
 
-  /* Otherwise expand as a fully variable permuation.  */
-
-  icode = direct_optab_handler (vec_perm_optab, mode);
+  insn_code icode = direct_optab_handler (vec_perm_optab, mode);
   if (icode != CODE_FOR_nothing)
     {
-      rtx tmp = expand_vec_perm_1 (icode, target, v0, v1, sel);
+      rtx sel_rtx = vec_perm_indices_to_rtx (sel_mode, sel);
+      rtx tmp = expand_vec_perm_1 (icode, target, v0, v1, sel_rtx);
       if (tmp)
 	return tmp;
     }
 
-  if (qimode != VOIDmode)
+  if (qimode != VOIDmode
+      && selector_fits_mode_p (qimode, qimode_indices))
     {
       icode = direct_optab_handler (vec_perm_optab, qimode);
       if (icode != CODE_FOR_nothing)
 	{
-	  rtx tmp = gen_reg_rtx (qimode);
-	  tmp = expand_vec_perm_1 (icode, tmp, gen_lowpart (qimode, v0),
-				   gen_lowpart (qimode, v1), sel_qi);
+	  rtx sel_qi = vec_perm_indices_to_rtx (qimode, qimode_indices);
+	  rtx tmp = expand_vec_perm_1 (icode, target_qi, v0_qi, v1_qi, sel_qi);
 	  if (tmp)
 	    return gen_lowpart (mode, tmp);
 	}
     }
 
+  delete_insns_since (last);
   return NULL_RTX;
 }
 
@@ -5590,7 +5598,7 @@ expand_vec_perm (machine_mode mode, rtx v0, rtx v1, rtx sel, rtx target)
    SEL must have the integer equivalent of MODE and is known to be
    unsuitable for permutes with a constant permutation vector.  */
 
-static rtx
+rtx
 expand_vec_perm_var (machine_mode mode, rtx v0, rtx v1, rtx sel, rtx target)
 {
   enum insn_code icode;
@@ -5633,17 +5641,16 @@ expand_vec_perm_var (machine_mode mode, rtx v0, rtx v1, rtx sel, rtx target)
   gcc_assert (sel != NULL);
 
   /* Broadcast the low byte each element into each of its bytes.  */
-  vec = rtvec_alloc (w);
+  vec_perm_builder const_sel (w);
   for (i = 0; i < w; ++i)
     {
       int this_e = i / u * u;
       if (BYTES_BIG_ENDIAN)
 	this_e += u - 1;
-      RTVEC_ELT (vec, i) = GEN_INT (this_e);
+      const_sel.quick_push (this_e);
     }
-  tmp = gen_rtx_CONST_VECTOR (qimode, vec);
   sel = gen_lowpart (qimode, sel);
-  sel = expand_vec_perm (qimode, sel, sel, tmp, NULL);
+  sel = expand_vec_perm_const (qimode, sel, sel, const_sel, qimode, NULL);
   gcc_assert (sel != NULL);
 
   /* Add the byte offset to each byte element.  */
@@ -5838,9 +5845,8 @@ expand_mult_highpart (machine_mode mode, rtx op0, rtx op1,
   enum insn_code icode;
   int method, i, nunits;
   machine_mode wmode;
-  rtx m1, m2, perm;
+  rtx m1, m2;
   optab tab1, tab2;
-  rtvec v;
 
   method = can_mult_highpart_p (mode, uns_p);
   switch (method)
@@ -5883,21 +5889,20 @@ expand_mult_highpart (machine_mode mode, rtx op0, rtx op1,
   expand_insn (optab_handler (tab2, mode), 3, eops);
   m2 = gen_lowpart (mode, eops[0].value);
 
-  v = rtvec_alloc (nunits);
+  auto_vec_perm_indices sel (nunits);
   if (method == 2)
     {
       for (i = 0; i < nunits; ++i)
-	RTVEC_ELT (v, i) = GEN_INT (!BYTES_BIG_ENDIAN + (i & ~1)
-				    + ((i & 1) ? nunits : 0));
-      perm = gen_rtx_CONST_VECTOR (mode, v);
+	sel.quick_push (!BYTES_BIG_ENDIAN + (i & ~1)
+			+ ((i & 1) ? nunits : 0));
     }
   else
     {
-      int base = BYTES_BIG_ENDIAN ? 0 : 1;
-      perm = gen_const_vec_series (mode, GEN_INT (base), GEN_INT (2));
+      for (i = 0; i < nunits; ++i)
+	sel.quick_push (2 * i + (BYTES_BIG_ENDIAN ? 0 : 1));
     }
 
-  return expand_vec_perm (mode, m1, m2, perm, target);
+  return expand_vec_perm_const (mode, m1, m2, sel, BLKmode, target);
 }
 
 /* Helper function to find the MODE_CC set in a sync_compare_and_swap