Don't get memcpy, memmove, memcmp, memset, and strlen from msvcrt.dll

[alexchandel]

The memcpy, memmove, memcmp, strlen, and memset symbols in msvcrt are just shims for implementations in ntdll.dll, which have existed since Windows 3.1.

If possible, we should link against these directly, rather than going through msvcrt. This would have the effect of eliminating all of our dependencies on msvcrt except for math functions and the entry point.

cc @retep998

[retep998]

These functions are not shims as I have just disassembled and verified this. Also ntdll.dll is an explicitly unstable API that should not be relied on directly so I don't recommend linking to it directly.

[alexchandel]

@retep998 At least on very old systems, msvcrt.dll did call into ntdll.dll for them. That's rather surprising that they would be called unstable, being there unchanged for 25 years. Can you find something on MSDN marking these symbols as unstable?

[retep998]

@alexchandel ntdll.dll is the interface to the NT kernel. The NT kernel is free to add or remove functions with each version of NT. Just because something hasn't changed does not mean Microsoft provides any sort of promise of stability. ntdll.dll has functions like memcpy and friends because it is providing them for other system libraries. It's not intended to be directly used by user-mode applications.

[alexchandel]

@retep998 I know it has all the syscalls, and that their syscalls are unstable, but I can't find any documentation at all on what symbols are in the current version. Which is uncharacteristic of MS.

But that's too bad if we can't, because unless we provide our own implementations of them (alongside an allocator like jemalloc and a math lib like openlibm), we're still tied to MS's CRT>

[retep998]

@alexchandel We don't rely on the CRT for the allocator, even without jemalloc. Replacing the math memory functions with our own optimized versions should be quite doable with the new simd stuff that @huonw is working on.

[huonw]

(I don't think implementing our own libm-replacement is a great idea---using a preexisting one like openlibm seems better for various reasons---and, even then, SIMD isn't so useful for scalar functions like those in libm.)

[abonander]

@huonw Can you elaborate on a few of these reasons? From what I understand, memcpy, memmove and memset have relatively trivial implementations. I realize there's probably loads of platform- and architecture-specific optimizations in the big libs but can't we still get reasonably fast with our own implementation? If it eliminates a problematic dependency I'm all for it. At the very least, it could be a viable alternative.

[alexchandel]

@cybergeek94 I think he meant implementing math functions. @huonw you're working on SIMD stuff for memory?

[huonw]

No, I'm not. I was replying to @retep998's original comment which had a typo: it said "math" instead of "memory".

In any case, rlibc implements the memory functions Rust needs.

[alexchandel]

@huonw how does one have rustc link against rlibc instead of msvcrt?

Combined with openlibm and an entry point, that would simplify a cross-compile to windows to LIB=$HOME/i686-pc-windows-msvc-w10/lib rustc --target i686-pc-windows-msvc test.rs, since LLD plays nicely now (#29126)

[briansmith]

Why not just remove all uses of memcpy, memmove, memcmp, strlen, and memset from the Rust code completely, writing them all in Rust? Not only would that be better than relying on an unstable Windows API, but also that effort would help other efforts for libc-less Rust.

[huonw]

LLVM inserts calls to (some of) those functions during optimisation passes.

[briansmith]

Why not just turn off those optimizations in LLVM? That would be useful for some users of #![no_std] too.

[retep998]

Because then you'd be harming performance for very little gain.