SVE Backend

[jeremylt]

Closes #631

[jeremylt]

This now compiles and passes the t3 tests on Ookami.

ToDo:

• Performance comparisons
• Makefile flag fix (not sure how the AVX one even works)
• Improved vectorization instructions?
• Add SVE backend to README
• Do we want manual unrolling for opt? The unrolling is pretty straightforward

[laylagi]

I remember from Ookami's talk that performance with GCC was the worst among all compilers. I did a brief experiment before losing access to Ookami.
sve/blocked with armclang: DoFs/Sec in CG: 1.43289 (1.43289) million
opt/blocked with gcc: DoFs/Sec in CG: 0.729637 (0.729637) million

I am not sure why these numbers are so small compared to what we had Friday but I think the poor performance with sve might partly have something to do with compiler, maybe?! I have to apply for an account to do more experiments though.

[jedbrown]

Is this ready for review? Should we include it in v0.11?

[jeremylt]

The two big todos are fixing the makefile magic and seeing if this actually does anything different than OPT in terms of performance.

[jedbrown]

I noticed that libxsmm contains aarch64/SVE code and it's announced as supported for the next release.

[jedbrown]

Do we have a place where we can measure performance? There is a machine at Sandia that you can access if you put in a Sarape request and AWS c7g also has SVE. JLSE also has a system that I could try requesting.

[jeremylt]

Ookami will reach its end of life at the end of this calendar year.

[jeremylt]

Side note - finally got the compile flags right to test in GitHub Actions. Yay. But the test is failing. Boo.

[jeremylt]

Given that XSMM targets this hardware and already gives us better performance on AVX hardware, I propose we close this. Any objections?

[jrwrigh]

This was well before my time, so I'm not 100% on the motivation and scope of the PR. But that logic makes sense to me.

[jedbrown]

I think XSMM did not target SVE at the time when this work was started. If XSMM covers the scope of what we might use SVE intrinsics for, I don't have a problem sticking with that and just closing this.