x86-64: mingw: Pass and return _Float16 in vector registers [PR115054]

For MinGW on x86-64, GCC currently passes and returns `_Float16` in
GPRs. Microsoft does not specify an ABI for the type so this is purely
an extension; however, there are a few reasons the current ABI is not
ideal:

1. `float` and `double` are both passed and returned in xmm registers
   under the MSVC ABI, there isn't any reason for `_Float16` to deviate.
2. `_Float16` is returned in xmm0 on Windows x86-32 by both GCC and
   Clang.
3. There is a platform-natural ABI with AVX512-FP16, which requires
   half-precision operands to be in vector registers.
4. System V uses vector registers for `_Float16`.

Thus, update the `HFmode` ABI to both pass and return in vector
registers, meaning its ABI is now identical to `float` and `double`.
This is already Clang's behavior on both its x64 MSVC and MinGW targets,
so the change here also resolves an ABI incompatibility (originally
reported in linked issue).

The results can be verified by evaluating the change in assembly output
with this source:

    void pass_f16(_Float16 x, _Float16 *dst) {
        *dst = x;
    }

    void callee_f16(_Float16);
    void call_f16() {
        callee_f16(1.0);
    }

    _Float16 ret_f16(_Float16 *x) {
        return *x;
    }

    /* Check libcall ABI */

    void extend_f16(_Float16 *x, _Float32 *dst) {
        *dst = (_Float32)*x;
    }

    void trunc_f16(_Float32 *x, _Float16 *dst) {
        *dst = (_Float16)*x;
    }

    /* Float varargs should be in vregs with a zeroed shadow GPR */

    void va(_Float16, ...);
    void va_f16() {
        va(1.0f16, 2.0f16, 3.0f16, 4.0f16, 5.0f16);
    }

While modifying the `function_value_ms_64` `switch` statement, a
redundant condition and trailing whitespace in the 16-byte case is
cleaned up.

2025-09-13  Trevor Gross  <tmgross@umich.edu>

gcc:
	PR target/115054
	* config/i386/i386.cc (function_arg_ms_64,
	function_value_ms_64): Pass and return _Float16 in vector
	registers on Windows.

Signed-off-by: Trevor Gross <tmgross@umich.edu>
Signed-off-by: Jonathan Yong <10walls@gmail.com>

Author: tgross35

gcc/config/i386/i386.cc

@@ -3493,8 +3493,10 @@ function_arg_ms_64 (const CUMULATIVE_ARGS *cum, machine_mode mode,
 
   regno = x86_64_ms_abi_int_parameter_registers[cum->regno];
 
-  /* Only floating point modes are passed in anything but integer regs.  */
-  if (TARGET_SSE && (mode == SFmode || mode == DFmode))
+  /* Only floating point modes less than 64 bits are passed in anything but
+     integer regs.  Larger floating point types are excluded as the Windows
+     ABI requires vreg args can be shadowed in GPRs (for red zone / varargs). */
+  if (TARGET_SSE && (mode == HFmode || mode == SFmode || mode == DFmode))
     {
       if (named)
 	{
@@ -4316,7 +4318,6 @@ function_value_ms_64 (machine_mode orig_mode, machine_mode mode,
 	{
 	case 16:
 	  if (valtype != NULL_TREE
-	      && !VECTOR_INTEGER_TYPE_P (valtype)
 	      && !VECTOR_INTEGER_TYPE_P (valtype)
 	      && !INTEGRAL_TYPE_P (valtype)
 	      && !VECTOR_FLOAT_TYPE_P (valtype))
@@ -4327,9 +4328,10 @@ function_value_ms_64 (machine_mode orig_mode, machine_mode mode,
 	  break;
 	case 8:
 	case 4:
+	case 2:
 	  if (valtype != NULL_TREE && AGGREGATE_TYPE_P (valtype))
 	    break;
-	  if (mode == SFmode || mode == DFmode)
+	  if (mode == HFmode || mode == SFmode || mode == DFmode)
 	    regno = FIRST_SSE_REG;
 	  break;
 	default: