Skip to content

Implement on top of posix API for emscripten - #233

Closed
guusw wants to merge 4 commits into
boostorg:developfrom
guusw:emscripten
Closed

Implement on top of posix API for emscripten#233
guusw wants to merge 4 commits into
boostorg:developfrom
guusw:emscripten

Conversation

@guusw

@guusw guusw commented Apr 25, 2022

Copy link
Copy Markdown
Contributor

I want to add a set of defines that correctly implement this library when compiling on top of emscripten.

Some explanation about the checked defines:

  • The EMSCRIPTEN_STANDALONE_WASM is defined when you build with -s STANDALONE_WASM so that case should still be ignored/left unimplemented

  • When the above is not set and __EMSCRIPTEN__ is defined, it's safe to assume that the js library is bundled that implemented the posix filesystem API.

Apparently EMSCRIPTEN_STANDALONE_WASM is only set during linking so it's not useful in this case

I saw #230 , which seems to want to do the same but this should work for both the standalone/emscripten case

Comment thread src/operations.cpp Outdated
@Lastique

Copy link
Copy Markdown
Member

Also, can you confirm that this change will work with WASI?

- Add cmake option to turn WASI API on/off
- Rename BOOST_STANDALONE_WASM => BOOST_FILESYSTEM_STANDALONE_WASM
@guusw

guusw commented Apr 25, 2022

Copy link
Copy Markdown
Contributor Author

Apparently EMSCRIPTEN_STANDALONE_WASM is only set during linking so it's not useful in this case.

I added an alternative mechanism where configuring with BOOST_FILESYSTEM_DISABLE_EMSCRIPTEN_WASI=ON will use the posix API, otherwise it keeps working as it is now.

I would prefer it to be the other way around though since it's an opt-in from emscripten's side too (-sSTANDALONE_WASM during linking) although current WASI users will have to add this option.

@guusw
guusw requested a review from Lastique April 27, 2022 07:53
Comment thread src/operations.cpp Outdated
@Lastique

Copy link
Copy Markdown
Member

I have next to zero knowledge about WebAssembly and related ecosystem, so may I ask a few questions for my own education to better understand the PR?

I don't quite understand where Emscripten is wrt. WASI. Is WASI using Emscripten as a compiler or is there a different compiler that happen to define __wasm?

If Emscripten is the only compiler, and WASI is just one alternative of the system API, along with what Emscripten provides (what you called "JavaScript API"), which one is the default? Can we assume the default at all?

The reason I'm asking is I'd like to know which is the most reasonable default target to compile for. In other words, is the full POSIX API "the norm" or not in WebAssembly world? Or are Emscripten and WASI completely independent and should be considered separate target platforms?

@guusw

guusw commented Apr 27, 2022

Copy link
Copy Markdown
Contributor Author

Out of the box you use the Emscripten compiler to generate a set of Wasm/JS files for running in a browser.
The C++ code is compiled into the wasm module and imports some functionality defined in the JavaScript file (for example filesystem).

In addition to building for the web, Emscripten also has an -sSTANDALONE_WASM compiler flag which causes it to not generate JavaScript code an instead build just a wasm module.

Emscripten tries to use WASI for some of the basic functionality (open/read/write) so if you build using STANDALONE_WASM you could run it on any WASI runtime.

Additionaly the WASI-SDK exists - another compiler/SDK - which builds purely against WASI

I'm not to familiar with the current state of WASI but currently Emscripten implements most of the filesystem API through the POSIX functions (getcwd/stat/etc.)

@Lastique

Copy link
Copy Markdown
Member

Out of the box you use the Emscripten compiler to generate a set of Wasm/JS files for running in a browser. The C++ code is compiled into the wasm module and imports some functionality defined in the JavaScript file (for example filesystem).

In addition to building for the web, Emscripten also has an -sSTANDALONE_WASM compiler flag which causes it to not generate JavaScript code an instead build just a wasm module.

Emscripten tries to use WASI for some of the basic functionality (open/read/write) so if you build using STANDALONE_WASM you could run it on any WASI runtime.

So, if I understand it right, by default Emscripten targets JavaScript API that supports POSIX. STANDALONE_WASM is an option that the user has to manually select.

Is it possible to follow this convention in Boost.Filesystem? I.e. when Emscripten is used, by default assume full POSIX compliance, and add a user-defined macro to enable targeting STANDALONE_WASM. And while doing this, keep WASI-SDK (with its own compiler) functional by default.

Basically, can we separate Emscripten and WASI-SDK compiler and use different defaults for them?

@guusw

guusw commented Apr 27, 2022

Copy link
Copy Markdown
Contributor Author

So, if I understand it right, by default Emscripten targets JavaScript API that supports POSIX. STANDALONE_WASM is an option that the user has to manually select.

Yes

Basically, can we separate Emscripten and WASI-SDK compiler and use different defaults for them?

Pretty much what I wanted to do. I will update the PR and ping you when it's done

When using Emscripten filesystem now links against the POSIX functions
@guusw
guusw requested a review from Lastique May 4, 2022 16:52
@guusw

guusw commented May 4, 2022

Copy link
Copy Markdown
Contributor Author

Updated it.
Wasn't sure what to name it so I went with BOOST_FILESYSTEM_EMSCRIPTEN_USE_WASI for the cmake option

Comment thread CMakeLists.txt
if(BOOST_FILESYSTEM_DISABLE_BCRYPT)
target_compile_definitions(boost_filesystem PRIVATE BOOST_FILESYSTEM_DISABLE_BCRYPT)
endif()
if(BOOST_FILESYSTEM_DISABLE_EMSCRIPTEN_WASI)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block looks outdated.

Comment thread CMakeLists.txt
set(BOOST_FILESYSTEM_DISABLE_GETRANDOM OFF CACHE BOOL "Disable usage of getrandom API in Boost.Filesystem")
set(BOOST_FILESYSTEM_DISABLE_ARC4RANDOM OFF CACHE BOOL "Disable usage of arc4random API in Boost.Filesystem")
set(BOOST_FILESYSTEM_DISABLE_BCRYPT OFF CACHE BOOL "Disable usage of BCrypt API in Boost.Filesystem")
set(BOOST_FILESYSTEM_EMSCRIPTEN_USE_WASI OFF CACHE BOOL "Use WASI under emscripten in Boost.Filesystem")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't "Emscripten" start with a capital E?

Lastique added a commit that referenced this pull request May 15, 2022
This allows to enable more POSIX APIs when building with Emscripten
by default. Users may switch to WASI API by defining
BOOST_FILESYSTEM_EMSCRIPTEN_USE_WASI config macro.

Originally implemented in and based on
#233.
@Lastique

Copy link
Copy Markdown
Member

Thanks. I've merged a modified version of this PR.

@Lastique Lastique closed this May 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants