Proposal to remove the JuMP Extension

[kellertuer]

Edit: during this discussion the title and intention of the PR changed.

I gave the idea of non-array types another try, this is a very first sketch, me diving into an archaeological excatvation in paths through undocumented JuMP internals.

@blegat would you maybe help me in two things?

1. How would a non-array variable be added to JuMP? I did not find any example nor was I able to specify that in @variable myself. An example call would be neat to further play Tomb Raider through JuMP code with functions I would need
2. I renamed the second variable in
   

   Manopt.jl/ext/ManoptJuMPExt.jl

   Lines 613 to 619 in 2fdb644

   	function JuMP.build_variable(::Function, point, m::ManifoldsBase.AbstractManifold)
   	@info point
   	shape = _shape(m)
   	return JuMP.VariablesConstrainedOnCreation(
   	JuMP.vectorize(point, shape), ManifoldSet(m), shape
   	)
   	end

   
   from func to point, but otherwise it is often documented as info. What is this, what does it contain? Maybe magically some variable info where I can get the point type again from?
   I currently have an @info in the code to get more of this, but it looks very very very very long and to me pure gibberish:

[ Info: ScalarVariable{Float64, Float64, Float64, Float64}[ScalarVariable{Float64, Float64, Float64, Float64}(VariableInfo{Float64, Float64, Float64, Float64}(false, NaN, false, NaN, false, NaN, true, 0.2672612419124244, false, false)), ScalarVariable{Float64, Float64, Float64, Float64}(VariableInfo{Float64, Float64, Float64, Float64}(false, NaN, false, NaN, false, NaN, true, 0.5345224838248488, false, false)), ScalarVariable{Float64, Float64, Float64, Float64}(VariableInfo{Float64, Float64, Float64, Float64}(false, NaN, false, NaN, false, NaN, true, 0.8017837257372732, false, false))]

One further thing that is totally unclear to me is whether you ever store tangent vectors (gradients for example). The challenge is then, that this might (in Manifolds) even be of different “shape” than points. Example

• SVDMPoint (for fixed rank matrices, the example I would love to be able to provide) – stores an SVD, so U, S, V, where S is a vector (in data structure, mathematically more a diagonal matrix, but stored as a vector)
• corresponding tangents are UMVTangentvectors, similar structures for U and V but M is now indeed a n-by-n matrix.

Well, hopefully JuMP never cares and this part just works ;)

[kellertuer]

Ah, for the first I found an example call in

#491

will continue with

@variable(model, x in Hyperbolic(2), start = HyperboloidPoint([1, 2]))

and try to understand how that has to be stored then. But it seems to also do a scalar variable? But it is not?

[odow]

What's the simplest example you want to be able to build? No JuMP. Pure Manopt.jl

[kellertuer]

The very simplest example is the Riemannian center of mass. Given 3 points compute the minimiser of the squared distances.

Hm, but in my first 5 minute hack I also ran into problems there. But as I wrote in slack, last time I gave up on this after 4 months, so this is not meant to be finished fast and I would prefer to maybe at some point at least understand a little bit the code that now lives in my code base.
Currently I do not at all, because you scheme (as nice and helpful as it is) is to ask for an example, hack something, of which I do not understand anything and there we are.

This aims to be an extension where at least I can understand the code. I am aware that this might take much longer than 4 months of me - every now and then - diving somewhere into JuMP code.

You hope it fulfils enough other cases as well, I do not understand any of the code - I feel that is not a very sustainable variant of your code living in my package.
I will try to come up with an example the next few days.

[odow]

The very simplest example is the Riemannian center of mass

Code?

At the moment I'm not even sure if JuMP can support your problem types. We really assume that variables belong to R^n, and that there is such a thing as a scalar variable in R.

[kellertuer]

Here is some code

using Manopt, Manifolds
using ManifoldDiff: grad_distance
M = Hyperbolic(2)
data = PoincareBallPoint.([ [0.1, 0.2], [0.3,0.25], [0.35,0.4] ])
n = length(data)
f(M, p) = sum(1 / (2 * n) * distance.(Ref(M), Ref(p), data) .^ 2)
grad_f(M, p) = sum(1 / n * grad_distance.(Ref(M), data, Ref(p)));
m1 = gradient_descent(M, f, grad_f, data[1]; stepsize=ConstantLength(0.2))

Concerning the Rn thing. Sure we can open that discussion as well. About a year or a year and a half ago I was nearly convinced to totally delete this extension, because also then we discussed whether that is useful. Shall we open that again?

In the end, none of my problems live in Rn - they can be represented in a set of real numbers usually (like unit norm vectors in R3 are a two-dimensional space represented in R3 but it is not R3)...

I am open to deleting the extension, exactly due to the reason that Manopt gos far beyond things in Rn, but @blegat was against that.
I thought in the beginning, it would be a nice thing and maybe a bit of visibility, but maybe also really no one uses this extension anyways (even I don't, I am just curious whether it would work maybe).

For the example above, on Hyperbolic space there are three different ways to represent points: one the Hyperboloid (or array default, but also has a type) poincare ball and poincare half plane. All 3 do have a similar interpretation, just because the hyperboloid is the most common, we used that for arrays, the other are distinguished.
For The Fixed Rand Manifold, the SVDPoint represents a point on that manifold by storing U S V from and svd; this can be vectorized in principle...

With variables in R and the total thing in Rn we only exactly have Euclidean(n), even already Euclidean(n,m) (optimization on the space of matrices) would be beyond that. For just that one Manifold, this extension would be not worth having.

But again. If all this does not fit, then we should probably really delete this extension. I am totally open to that as well.

[kellertuer]

To not only have the pessimistic side: reasons why it might fit JuMP

• several manifolds can be seen as constrains (the sphere is a unit norm constraint) - though not all, while Stiefel is unit norm vectors, Grassmann is a quotient manifold of that, and these do usually not easily fit into a “constraint thinking” - Grassmann is, however, already possible in the current Array-variant
• Some types points are just wrappers (like for Hyperbolic), all others we currently have can at least be “vectorized” (squeezed into an 1D array), though that is an per-point-type manual thing to do then.
• Even fixed rank with its representation in 3 factors (by the way for numerical stability and ease of implementation) is a constrained thingy: n-by-m matrices of rank p. Turning it into a manifold makes the Riemannian problem unconstrained, so that is maybe nice to have (at the cost of using Manifolds.jl / differential geometry tools)

But again, if it does not fit, I am totally fine deleting the extension as well. I am not here to try to sneak this package into something where it does not belong.

[odow]

I'll discuss this with @blegat next week at https://jump.dev/meetings/jumpdev2025/

[kellertuer]

Ok. Enjoy JumpDev 2025!
Auckland, that sounds very nice.

[odow]

We discussed this at JuMP-dev. A summary:

• Matrix-based manifolds that use algebraic functions are a good fit for Manopt+JuMP. This is pretty much what the current extension does.
• More complicated manifolds are much trickier to implement
• If the manifolds require ManifoldDiff or user-defined operators (like distance), then the answer is a strong "no", this cannot be done in JuMP. The main reason is that we don't have good ways to support vector-valued nonlinear operators, and custom gradients are also tricky, etc.
• We agree that the MOI interface is large, quite verbose, and poorly documented. It probably isn't a good use of your time to spend a month trying to understand JuMP and MOI to implement the extension.
• You could either give @blegat small examples, and he can see if it is possible to implement (it might not be, see points above).
• Alternatively, maybe a Manopt extension is the wrong place for the wrapper. @blegat can remove the wrapper into a separate package, either under his personal GitHub account or under jump-dev. Then you're not responsible for developing and maintaining it.

Benoît might have some other comments, but those were my notes.

[kellertuer]

Thanks for the detailed discussion points, that I feel are indeed very central to how to continue here.

• Matrix-based manifolds that use algebraic functions are a good fit for Manopt+JuMP. This is pretty much what the current extension does.

Currently (and for the foreseeable future) all manifolds are finite dimensional and matrix based.
In the two examples from above,

• the Hyperbolic manifold has three different representations, but all 3 are “just” vectors. To avoid reimplementations of elements that are independent of the representation, we have one manifold type, that is representation independent. We then distinguish representations by point type and their corresponding tangent vectors also get a type. But these are all “vector wrappers”
• FixedRankMatrices already tell that in their name, are n-by-m matrices of rank p, also matrix -based. Still, if we would just store that matrix it would take much more space than the reduced SVD and in every step we would have to first compete the SVD. This is inefficients and unstable. So we store the SVD (U, V, and the vector of diagonal entries) But it is still a matrix-based thingy.
• We have a few that are tensors (Segre, Tucker, Euclidean - possibly) but that is just higher dimensional arrays than matrices.

Do all. these not fit? That would be very unfortunate, since just because it came first in our implementation, the only “nonwrapped” (to be precise not necessarily wrapped) would be the hyperbolic representation.
So if it does not fit, we should not have an extension at all, I feel, because the wrapping was more like random and I do not want to make JuMP available for just some random manifolds.

So a main question for me: does matrix-based here refer to mathematically based on a matrix? Then all manifold we have to fit into that.
If it is meant as the type of(2D array and only exactly that type) based - then I would prefer not to have an extension.

• If the manifolds require ManifoldDiff or user-defined operators (like distance), then the answer is a strong "no", this cannot be done in JuMP. The main reason is that we don't have good ways to support vector-valued nonlinear operators, and custom gradients are also tricky, etc.

This is one further point where I would say, then it really does not fit. We nearly always have manual / custom gradients that have to be provided.
One simple reason is, there is a difference between the Euclidean gradient and the Riemannian one. In the current sketch we do that transform, but usually Euclidean (maybe AD) & convert is usually relatively inefficient. At least until someone comes along and writes bot the theory and a package for AD (directly) on manifolds.

• We agree that the MOI interface is large, quite verbose, and poorly documented. It probably isn't a good use of your time to spend a month trying to understand JuMP and MOI to implement the extension.

I do see, that for you the documentation for a user is of course more central than for a developer, because they are In the vast majority for you.

• You could either give @blegat small examples, and he can see if it is possible to implement (it might not be, see points above).

I tried that the last few times already.

• Alternatively, maybe a Manopt extension is the wrong place for the wrapper. @blegat can remove the wrapper into a separate package, either under his personal GitHub account or under jump-dev. Then you're not responsible for developing and maintaining it.

My only point here is, if code lives in “my” repository, I want to be able to provide support to some extend and at least understand it roughly myself. For now I do not do that (see missing documentation above). So maybe the inverse idea of having (if at all) an extension in your responsibility is the better idea.
If you need anything from the manifold side, that is hopefully documented in detail – every function in the ManifoldsBase API is documented in detail and all functions in Manifolds hopefully have even their formula documented.

So my conclusion would be to move it to your responsibility, but if the first question is that only arrays should be used, then I would prefer to both delete it here and not do it at all, since the ideas of JuMP and Manopt are just too different in how to approach things
(and surely none of them is better per se)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (dev-0.6@33c090d). Learn more about missing BASE report.

Additional details and impacted files

@@             Coverage Diff             @@
##             dev-0.6      #532   +/-   ##
===========================================
  Coverage           ?   100.00%           
===========================================
  Files              ?        90           
  Lines              ?     10897           
  Branches           ?         0           
===========================================
  Hits               ?     10897           
  Misses             ?         0           
  Partials           ?         0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kellertuer]

I gave this a careful thought again, and with all the arguments stated and discussion we had, I think it is best to remove the JuMP extension for now.

I thought for a while it was cool for visibility, since you really have a large user base, but as a mere mortal it feels impossible to me to maintain such an extension. Furthermore

• manifolds as a constraints are a bit of a stretch modelling wise – though true in several cases.
• matrices are already a stretch, since all JuMP variables are vectors
• typed points and especially also distinguishing points and tangent vectors seems to be a huge effort and is not really supported nor something that seems to be wishful to have in JuMP

All these in mind – I am open to have such a “bridge” between Manopt and JuMP somewhere, but better in an own package somewhere in the “JuMP universe”, I am happy to help with manifold-related things. This way around, the parts that are internal and not (yet) so well documented are “closer” to the package. I would carefully claim that both Manifolds.jl and Manopt.jl are reasonably documented – but of course I am also available to help if someone wants to give this connection a new try.

I will keep this PR open maybe until somewhen in January 2026 and then merge it.

Thanks for all the discussions and efforts.

[blegat]

If the manifolds require ManifoldDiff or user-defined operators (like distance), then the answer is a strong "no", this cannot be done in JuMP. The main reason is that we don't have good ways to support vector-valued nonlinear operators, and custom gradients are also tricky, etc.

Thinking more about this, it may be possible with Riemannian objective (added in #448). But with classical objective for which JuMP is responsible to do the AD, it might be quite complicated.

So my conclusion would be to move it to your responsibility, but if the first question is that only arrays should be used, then I would prefer to both delete it here and not do it at all, since the ideas of JuMP and Manopt are just too different in how to approach things

Moving it to a package either under my personal account or under jump-dev sounds good to me. From my perspective, maintaining it here as a package extension or maintain it in a separate package is mostly the same amount of work so I'm fine with anything.

[kellertuer]

Then it would be great to have it as a package either on your account or the jump namespace.
For the manifold part I can still help of course, if there are questions.

[kellertuer]

I diverted the PR to be collected on a dev-0.6 branch, since we plan to collect a few forthcoming breaking changes for that release. Will merge this when the tests have completed.

[odow]

Probably for the best. @blegat and I may revisit if we get around to adding true vector-valued nonlinear operators.

(Un)related: this PR is a great example of why I'm very hesitant to add new extensions in JuMP-related packages. Linking here so I have a back link if I ever need to find this in future: jump-dev/MathOptInterface.jl#2972 and jump-dev/MathOptInterface.jl#2967

[kellertuer]

As I mentioned, I am happy to help from the Manopt-side, if you give it a new try. I personally felt too often lost within JuMP. And that is not meant as negative as it sounds. I think the docs for users of JuMP are great and you do a very good job on the development and commenting (even now here on such short note!), just that from the development side, I often was diving deep into the source code of JuMP itself.

To the “vector valued nonlinear operators” – I fear we would especially need variables (and corresponding costs) beyond vectors – into matrices and even arbitrary structs (wrappers of representations or even structured data like for fixed rank an SVD or so).

But maybe some day we continue this adventure then :)

[blegat]

Sounds good, I am planning to experiment with vector-valued nonlinear operators in https://github.com/blegat/ArrayDiff.jl/
Now, I need to choose a good name, do you have any policy on the names used by packages outside of JuliaManifolds ? We request that packages outside of jump-dev don't use JuMP in their names for instance.
One possibility is JuMPonManifolds which has a bouncy feel

[kellertuer]

I always jump around on manifolds. It indeed sounds a bit bouncy. If so you should name all packages like that (or even use TranpolinX.jl and such).

Joke aside (had a long day already since I am currently in JST time zone). Great that you reach out about a name.

I am fine with either JuMPManoptExt.jl or ManoptJuMPExt.jl (inspired by e.g. OptimizationManopt.jl within SciML, which they even chose without asking), and we do not have so much of a Scheme. If it is within the JuMP-org, maybe putting JuMP first is reasonable? On the other hand, the capital letters grouping in the middle also looks a bit strange.

Back to the start … since if you actually do jump on a trampoline, the surface does form a manifold and it looks even a bit hyperbolic...
So indeed also just Trampolin.jl is fine this me, though maybe a bit harder to discover for users.

edit: Trampolin.jl is indeed a name that is available in the General Registry and though that started as a joke when I started writing it, it is somehow nice. relates to JuMP (is a bit bouncy, yes) and curved.

[mateuszbaran]

BouncyCastle.jl would also be a fun name 🙂 . But watch out for sharp corners we're going to add soon with #554 😉 .

More seriously, regarding Ext names, I think it's reserved for actual package extensions? If a separate package is required then something like JuMPManopt or JuMPManifolds sounds good. I think it's fine to use words "Manopt"/"Manifolds" in packages outside of JuliaManifolds, though I'd prefer if the first word in the name wasn't something already existing in JuliaManifolds.