Display description text in docs groups by lud · Pull Request #2113 · elixir-lang/ex_doc

lud · 2025-04-09T09:05:54Z

Related to #2104

Still a work in progress. If we go in this direction we should add docs as well.

There are some limits with this solution:

Since we want the description from default_group_for_docs, it means that each doc node can return a different description for a given group. I made the choice to ignore those descriptions based on the group title.
As we want to keep the group order defined in moduledoc, the retriever will fetch those, possibly containing descriptions. I made the choice to not replace descriptions from moduledoc by descriptions returned by the config callback.
Node groups will temporarily contain a title/description map, only to be replaced with just the title once the groups are built in the modulenode. This avoids a lot of "when is_binary() ... when is_map()" stuff and avoids to refactor many tests. This also allows to have each group rendered only once. A drawback is that in the module_summary function we are reconciliating the groups once again where each doc node could already bear the full group. Not a big deal imho.
Since those groups are rendered, they must behave like most nodes. But they are not nodes for now. I guess we could add a struct for those, but it's not the real problem. What is more important is that I had to hardcode "text/markdown" and wrap the text in %{"en" => text }, as module and doc nodes are extracted from documentation. I could have written a specialized doc_ast render function though.
I added some classes on the html elements which may not be needed. Same for the IDs. This was mostly to test the rendering.

Besides that, it works.

josevalim · 2025-04-11T10:20:18Z

  def module_summary(module_node) do
    # TODO: Maybe it should be moved to retriever and it already returned grouped metadata
-    ExDoc.GroupMatcher.group_by(module_node.docs_groups, module_node.docs, & &1.group)
+


The TODO above is precisely about moving this logic to the retriever. Given we are already changing the group structure, maybe now is a good time to go ahead and do it? Basically, by adding a group column to the groups upfront?

Yeah I saw this TODO and in my first attempt I wanted to provide docs_groups with all the docs inside.

But to avoid having duplicate data, to me it means that we basically remove the :docs key from the ModuleNode struct. And that means refactoring a lot of tests to look for expected data in a list of groups instead of just matching on the content of the docs key. Lot more code to change and review. But I can do it if you think it's alright.

ah sorry I missed the part about a group column. Should be simple enough.

Alright, please see that last commit: aed460e

(There were other commits before since you last replied).

I'm not sure this is what you expect. The retriever is computing the groups, so the renderer part is only fetching that, which is what we wanted.

But,

There is duplicated data. The ModuleNode still has its :docs key with the DocNode list because many tests look into that for assertions. We can update those tests to look into the groups instead, with a lot of Enum.find calls :) and then delete the ModuleNode :docs key.

Those DocNodes in the :docs key have nil :rendered_doc because we only render the copy of the doc nodes that are inside group nodes.

Or maybe it is something else you had in mind?

The code looks great, thank you ! 🤩

There is duplicated data. The ModuleNode still has its :docs key with the DocNode list because many tests look into that for assertions. We can update those tests to look into the groups instead, with a lot of Enum.find calls :) and then delete the ModuleNode :docs key.

I think it is worth giving this a try! Perhaps we could have a helper function in tests, called find_doc(...) or find_group(...) to help make life easier. Could you please try that as well? I understand it is a lot of work though, but this is some of the refactoring we have been wanting to do for quite some time (basically do less work on the formatters and more on the retrievers).

lud · 2025-04-28T12:54:52Z

Hey @josevalim , have you had time to review my comments?

josevalim · 2025-05-05T08:07:38Z

Sorry for the delay, I am on holidays so I have limited time to stay on top of my inbox. I am reviewing this now but probably merge it only when I am back. :)

lud · 2025-05-05T09:41:45Z

Hello @josevalim no problem, it was just to be sure this was not forgotten. Take your time. We probably need to add some docs before mergeing.

@garazdawi if you have time can you tell me if it works as you would expect?

Thank you both.

garazdawi · 2025-05-08T08:56:53Z

I did a quick test and it works as expected. I'm wondering if maybe something could be done about the styling to make it more obvious that the text belongs to the entire group of functions... not sure what that would be though.

lud · 2025-05-10T18:34:18Z

@garazdawi thank you !

I think we can just add some whitespace between the description and the first function signature block. The same as between the title and the description. I guess that would be enough to show that the text belongs to the section.

@josevalim I see a lot have changed about the heading and html ID generation. I have rebased my branch and made sure that the new groups descriptions go through add_fancy_anchors. I also updated the templates to use the new render_doc helper instead of having a .rendered_doc key in the nodes.

I'll tackle removing the ModuleNode.docs key and updating the tests soon.

lud · 2025-05-10T19:01:40Z

@garazdawi I think whitespace is enough, what do you think?

josevalim · 2025-05-10T19:15:49Z

@lud, apologies for the large changes that had happened. But we had bugs and a lot of duplication, and the new version should make things more accessible and easier to follow.

lud · 2025-05-10T21:46:18Z

@josevalim No worries :)

lud · 2025-05-14T10:46:56Z

Hello,

I removed the .docs field from the ModuleNode struct and fixed all the tests accordingly. It was actually easy since most tests generate only one group. Other tests were straightforward too.

So I left the helper function directly in elixir_test.exs since this is the only place it is used.

A test failed because this function was receiving a list of 2 elements (and it is matching on a list of 1 element only)

defp extract_filename!([{location, _}]), do: location

I may have done something wrong when building the assets when changing whitespace for the groups.

I ran the build command (npm run --prefix assets build) once more and that fixed it, only a single file was present. I guess it was just because of the git rebase.

lud · 2025-05-14T10:48:16Z

@josevalim if it is ok to merge, would you like me to squash the branch ?

Also to prevent further rebasing issues I suggest I'd write the docs in another PR.

lud · 2025-05-28T01:10:22Z

@josevalim if it is ok to merge, would you like me to squash the branch ?

Also to prevent further rebasing issues I suggest I'd write the docs in another PR.

Hello,

I just rebased the branch on the latest release. Do you think we can merge or are there some remaining things to fix?

josevalim · 2025-05-28T06:04:08Z

My plan is to review it this week as I am finally getting caught up with work :)

lud · 2025-05-28T09:18:26Z

Perfect, thank you!

josevalim · 2025-06-10T10:04:53Z

    |> Enum.map(fn path ->
      Module.put_attribute(__CALLER__.module, :external_resource, path)
-      {Path.basename(path), File.read!(path)}
+      {Path.basename(path), "File.read!(#{path})"}


This seems to be a mistake? :)

Gosh ... fortunately you spotted that. Yeah I was debugging assets generation at some point.

josevalim · 2025-06-10T10:05:17Z

-    # TODO: Maybe it should be moved to retriever and it already returned grouped metadata
-    ExDoc.GroupMatcher.group_by(module_node.docs_groups, module_node.docs, & &1.group)
-  end
+  def module_summary(module_node), do: module_node.docs_groups


Let's remove this function and update its callers?

Done,

I took the liberty to update the templates to directly use module.docs_groups in there, I think it's clearer that we are dealing directly with the content of %ModuleNode{}.docs_groups`.

josevalim

I am sorry for the delay! I have added one nit and one comment, and we can ship it! ❤️

lud · 2025-06-11T10:25:46Z

Hello,

I fixed your last comments, thank you for the review :)

github-actions · 2025-06-11T16:27:56Z

📦 Docs artifacts are ready: https://github.com/elixir-lang/ex_doc/actions/runs/15582422419/artifacts/3307261911

josevalim · 2025-06-11T16:31:57Z

💚 💙 💜 💛 ❤️

lud mentioned this pull request Apr 9, 2025

Defining a text bloc for group_for_docs sections #2104

Closed

josevalim reviewed Apr 11, 2025

View reviewed changes

Comment thread lib/ex_doc/nodes.ex Outdated

josevalim reviewed Apr 11, 2025

View reviewed changes

Comment thread lib/ex_doc/retriever.ex Outdated

josevalim reviewed Apr 11, 2025

View reviewed changes

Comment thread lib/ex_doc/retriever.ex

lud force-pushed the group-header branch 2 times, most recently from 882493f to 8db2c39 Compare April 17, 2025 05:43

lud force-pushed the group-header branch from aed460e to 36eb55f Compare May 10, 2025 17:21

lud force-pushed the group-header branch from cba4ae9 to 5538373 Compare May 14, 2025 10:00

lud marked this pull request as ready for review May 14, 2025 10:42

lud force-pushed the group-header branch from 3442d79 to 9e2e011 Compare May 28, 2025 01:07

lud force-pushed the group-header branch from 9e2e011 to dfd8f71 Compare June 4, 2025 20:32

josevalim reviewed Jun 10, 2025

View reviewed changes

josevalim approved these changes Jun 10, 2025

View reviewed changes

Allow groups to carry descriptions

02a3ec5

lud added 13 commits June 11, 2025 12:26

Accept groups descriptions from callback

9ad748f

Render groups descriptions

3a4d609

refactor

84aebce

use new struct for node group

e0dcc22

enforce group titles as binaries

25521da

do not store groups in DocNode temporarily

6d9a795

compute grouped docs summary in retriever

53f7024

wip rebasing

666d857

fix rebasing, missing ids

ed26585

(no changes) verified anchors are added to section description headings

22fb2b7

add more whitespace below group descriptions

a76591b

removed ModuleNode.docs struct field, updated tests

c0a3031

review fixes

b1ac484

lud force-pushed the group-header branch from de7e6c0 to b1ac484 Compare June 11, 2025 10:26

josevalim merged commit e85d853 into elixir-lang:main Jun 11, 2025

rhcarvalho mentioned this pull request Jul 28, 2025

Provide function group descriptions in Ecto.Repo elixir-ecto/ecto#4643

Merged

Uh oh!

Conversation

lud commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lud commented Apr 28, 2025

Uh oh!

josevalim commented May 5, 2025

Uh oh!

lud commented May 5, 2025

Uh oh!

garazdawi commented May 8, 2025

Uh oh!

lud commented May 10, 2025

Uh oh!

lud commented May 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josevalim commented May 10, 2025

Uh oh!

lud commented May 10, 2025

Uh oh!

lud commented May 14, 2025

Uh oh!

lud commented May 14, 2025

Uh oh!

lud commented May 28, 2025

Uh oh!

josevalim commented May 28, 2025

Uh oh!

lud commented May 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

josevalim left a comment

Choose a reason for hiding this comment

Uh oh!

lud commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 11, 2025

Uh oh!

josevalim commented Jun 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

lud commented Apr 9, 2025 •

edited

Loading

lud commented May 10, 2025 •

edited

Loading

lud commented Jun 11, 2025 •

edited

Loading