JSONObject.toString() throws StackOverflowError (not JSONException) on self-referential cycles

[zhangjiashuo-cs]

Description

JSONObject.toString() (and write()) recurse into nested JSONObject values without any cycle detection. If a JSONObject contains itself (directly or transitively), serialization recurses indefinitely and the JVM throws StackOverflowError.

The parsing path is protected by JSONParserConfiguration.getMaxNestingDepth(), but cycles created via put() programmatically bypass it because the cycle was never parsed.

Reproducer (org.json 20240303)

import org.json.JSONObject;

public class Repro {
    public static void main(String[] args) {
        JSONObject jo = new JSONObject();
        jo.put("key", "value");
        jo.put("self", jo);    // direct self-reference
        jo.toString();         // -> StackOverflowError
    }
}

Indirect cycles also trigger:

JSONObject a = new JSONObject(), b = new JSONObject();
a.put("b", b);
b.put("a", a);
a.toString();  // -> StackOverflowError

JSONArray containing itself triggers the same:

JSONArray arr = new JSONArray();
arr.put("x");
arr.put(arr);
arr.toString();   // -> StackOverflowError

Why this is more than "don't construct cycles"

• Code that takes user input and walks it into a JSONObject model (deserializers, GraphQL resolvers, ORM emitters) may produce a cycle without realizing it (object graph derived from a database join, a mutually-referencing config).
• The current contract is that JSONObject.toString() returns a String or throws a checked JSONException. A StackOverflowError is an Error, not an Exception, so application try/catch blocks targeting Exception (or even JSONException) won't catch it. The JVM thread crashes.
• A library used in a hot serialization path that crashes on Error rather than throwing a typed exception is a DoS / availability issue for any process that lets this be reached.

Root cause

JSONObject.writeValue(Writer, Object, …) and JSONArray.write(…) recurse on nested values without maintaining a "seen" set. The fix is to either:

1. Pass an IdentityHashMap<Object, Boolean> of currently-being-serialized objects down through write()/writeValue() and throw JSONException on a cycle.
2. Use the same maxNestingDepth limit on serialization that already exists on parsing.

(1) is more precise; (2) is simpler and matches the parsing-side mitigation.

Suggested patch sketch

public Writer writeValue(Writer writer, Object value, int indentFactor, int indent,
                         Set<Object> seen) throws JSONException, IOException {
    if (value instanceof JSONObject || value instanceof JSONArray) {
        if (!seen.add(System.identityHashCode(value))) {
            throw new JSONException("Cyclic reference detected during serialization");
        }
        try {
            // existing logic, threading `seen` into recursive calls
        } finally {
            seen.remove(System.identityHashCode(value));
        }
    } else { /* unchanged */ }
}

Environment

• org.json: 20240303 (latest at time of writing)
• JDK: 21

Discovered via jqwik property-based testing on the invariant toString() either returns a String or throws JSONException (never Error). Happy to PR.