What is min of System.Numerics.Vector<T>.Count on any supported os+hw combination for net core?

[BreyerW]

anyone know? im simdifying some code but i dont have many machines and i need guarantee that i can store at least 6 floats in single Vec. On my machine count reports 8*float but architecturally is only less than 2 years old. I tried reading a bit about registers but it is useless since most sources talk about generic registers (8 for 32 bit, double that for 64 bit) not simd registers

[danmoseley]

@tannergooding

[gfoidl]

128 bits is the min vector register size.
A float consists of 4 bytes, so 32 bits, hence Vector<float> has 128 / 32 = 4 elements.

For the (current) max the vector register size is 256 bits, so Vector<float> has 256 / 32 = 8 elements.

Note: Vector<T> does not have a "fixed" size, as it depends on the max. available register size on the executing hardware.
In contrast Vector128 and Vector256 are fix in the size of 128 bit and 256 bit respectively. So for

at least 6 floats in single Vec

you'd need at least 256 bits, that's AVX and you can guard your algorithm with

if (Avx.IsSupported)
{
    // vectorized code path that operates on 8 floats at once
}
else
{
    // software fallback for scalar path
}

[BreyerW]

i see. Im stuck with net standard 2.0 in this particular code so cant use Vector256. BTW does Vector256 abstract away differences in SIMD support? eg if i used Vector256 on SSE (128 max if memory serves right) will this struct fallback to 2x 128 SIMD or i have to guard it with feature guard?

[gfoidl]

this struct fallback to 2x 128 SIMD or i have to guard it with feature guard?

It does not fallback. Vector256 is always 1x256 bit, so you have to take care about this. It's low level "primitive" for direct operation with intrinsics.

Vector<T> is kind of an abstraction over Vector128 and Vector256 in .NET Core / .NET 5+.

[tannergooding]

We do not have an officially documented minimum or maximum and it ultimately depends on the underlying platform you are targeting.

Today, the minimum is 16-bytes and this is what Vector<T> is for the software fallback.
The maximum today is 32-bytes and is what Vector<T> is on most modern x86/x64 (Intel or AMD) hardware.

ARM64 however supports Vector64<T> and so it could theoretically support an 8-byte minimum (possibly as an explicit opt-in on startup).
Likewise, certain modern x86/x64 machines support Vector512<T> and ARM64 has a new SVE instruction set that goes up to Vector2048<T>, so the maximum could be 64 or even 256-bytes.

If you need to enforce your own minimum for a code-path, I'd recommend a simple if (Vector<float>.Count >= 6) check to help enforce this. The JIT will treat this check as a constant and will not emit any actual codegen for it.

[BreyerW]

@tannergooding i was thinking wouldnt it be okay to document that min is 16 bytes and maximum is open ended? i dont think you will ever go "backward" in os/hardware support and i cant imagine any new hardware having smaller simd than 16 bytes. i think it is more likely that such device will have 0 simd.

EDIT ah i reread your comment and see that arm64 could report 8 byte then documented minimum would be 8 bytes?

[tannergooding]

Correct. ARM64 supports Vector64<T> and Vector128<T>, while x64/x86 supports Vector128<T> and Vector256<T> today.

You could imagine, for example, us enabling Vector<T> to be 8-bytes on an opt-in basis (such as via an AppConfig switch) to provide better power efficiency on mobile devices.
Likewise, you could imagine us extending Vector<T> to support larger sizes (such as Vector512<T> on newer x64/x86 boxes) for more performance oriented scenarios.

[BreyerW]

@tannergooding @gfoidl BTW what happens to Vec if used on hardware that has Vector.IsHardwareAccelerated=false or 0 SIMD? Does Count still report 4? from what i saw in source code Register is defined as always having at least 128 bits and i want to use Vector.GreaterThan & Vector.ConditionalSelect potentially when SIMD is disabled to simplify code (from what i saw these specific methods have software impl on top of [Intrisic] attribute). Can i rely on that?

[gfoidl]

If not supported by hardware directly, a software-fallback will be used.

Vector<T>'s only field is Register

runtime/src/libraries/System.Private.CoreLib/src/System/Numerics/Register.cs

Lines 153 to 161 in 82ca681

	// Internal System.Single Fields
	[FieldOffset(0)]
	internal float single_0;
	[FieldOffset(4)]
	internal float single_1;
	[FieldOffset(8)]
	internal float single_2;
	[FieldOffset(12)]
	internal float single_3;

So you get 4 floats here.

Roughly speaking for [Intrinsic] the JIT checks if hardware support is given, and if so produce specialized code. If not use the software-fallback. So you can rely on the fact that the software-fallback will be used / is present iif no hardware support is given.

[BreyerW]

yeah i saw that and that prompted the question bc i was worried NET might be doing some rewriting even when hardware is not accelerated disabling Vector<T> uses for such situation. This is good i can avoid duplicating code for non accelerated hardware

[tannergooding]

That is largely an implementation detail and you should rely on Vector<T>.Count as in any other scenario to determine how big it is and how many elements it can handle.

As mentioned before, some hardware does support vectors with less than 128-bits and so you could imagine us exposing a config switch in the future to customize the behavior of this for when things like battery life are important.

Not saying that will happen, simply that it has been raised and discussed previously, same as with how we would support vectors larger than 256-bits.