adding constraints to disjunctions forces the solver to always pick the constrained route even if suboptimal

[tueboesen]

The fundamental problem I'm trying to solve is that I want to insert the option of adding a transit hub to a pickup/delivery, meaning that the solver should have the option of either doing a pickup/delivery a->b, or a pickup delivery a->c followed by another pickup delivery c->b (not necessarily with the same vehicle).

The way I am thinking of solving this problem is by making all of these options different nodes (at least for now). Thus my problem is that I have the following pickup/deliveries 0->1,2->3,4->5 (where 0->1 is the direct pickup/delivery, 2->3,4->5 represent an intermediate drop-off at a transit hub.)

To force the solver to pick one of these two options I use the following disjunctions:

    # Force the solver to either go to node [{0,1}] or to [{2,3},{4,5}], but not both
    routing.AddDisjunction([manager.NodeToIndex(0), manager.NodeToIndex(2)])
    routing.AddDisjunction([manager.NodeToIndex(0), manager.NodeToIndex(3)])
    routing.AddDisjunction([manager.NodeToIndex(1), manager.NodeToIndex(4)])
    routing.AddDisjunction([manager.NodeToIndex(1), manager.NodeToIndex(5)])

    routing.AddDisjunction([manager.NodeToIndex(0), manager.NodeToIndex(4)])
    routing.AddDisjunction([manager.NodeToIndex(0), manager.NodeToIndex(5)])
    routing.AddDisjunction([manager.NodeToIndex(1), manager.NodeToIndex(2)])
    routing.AddDisjunction([manager.NodeToIndex(1), manager.NodeToIndex(3)])

Next I create the pickup/deliveries:

    solver = routing.solver()

    # Make {0,1} into a pickup delivery
    pickup_index = manager.NodeToIndex(0)
    delivery_index = manager.NodeToIndex(1)
    routing.AddPickupAndDelivery(pickup_index, delivery_index)
    solver.Add(routing.VehicleVar(pickup_index) == routing.VehicleVar(delivery_index))

    # Make {2,3} into a pickup delivery
    pickup_index = manager.NodeToIndex(2)
    delivery_index = manager.NodeToIndex(3)
    routing.AddPickupAndDelivery(pickup_index, delivery_index)
    solver.Add(routing.VehicleVar(pickup_index) == routing.VehicleVar(delivery_index))

    # Make {4,5} into a pickup delivery
    pickup_index = manager.NodeToIndex(4)
    delivery_index = manager.NodeToIndex(5)
    routing.AddPickupAndDelivery(pickup_index, delivery_index)
    solver.Add(routing.VehicleVar(pickup_index) == routing.VehicleVar(delivery_index))

All of this works fine and does what I expect as far as I can see.
But next I need to add the constraint that (2->3) is done before (4->5) is started. Initially I figured I could just do
routing.AddPickupAndDelivery(manager.NodeToIndex(3), manager.NodeToIndex(4)), but that didn't seem to work, and also it wasn't entirely clear to me that it should, since a normal pickup/delivery assumes a single vehicle is used, and also because both of these were already part of another pickup/delivery.

So instead I decided to create a time dimension and then enforce the constraint that way:

    # Add time dimension. We need this to ensure that {2,3} is completed before {4,5} is started, if the solver selects to go that route.
    def time_callback(from_index, to_index):
        """Returns the distance between the two nodes."""
        from_node = manager.IndexToNode(from_index)
        to_node = manager.IndexToNode(to_index)
        return M[from_node][to_node]
    transit_callback_index = routing.RegisterTransitCallback(time_callback)

    dimension_name = "Time"
    routing.AddDimension(
        transit_callback_index,
        max_slack_time,  # no slack
        30000,  # vehicle maximum travel distance
        True,  # start cumul to zero
        dimension_name,
    )
    time_dimension = routing.GetDimensionOrDie(dimension_name)

    # We want to enforce that {2,3} is completed before {4,5} is started. Note that if we remove this constraint, then the code behaves as expected.
    pickup_index = manager.NodeToIndex(3)
    delivery_index = manager.NodeToIndex(4)
    solver.Add(time_dimension.CumulVar(pickup_index) <= time_dimension.CumulVar(delivery_index)) # This enforces the constraint, but also seems to make the solver always pick this solution if possible.
    # solver.Add(routing.ActiveVar(pickup_index) * time_dimension.CumulVar(pickup_index) <= routing.ActiveVar(delivery_index) *
[model.txt](https://github.com/user-attachments/files/16281011/model.txt)
 time_dimension.CumulVar(delivery_index)) # This doesn't help

So this adds the constraint, but unfortunately it seems that by adding this constraint the solver is now no longer free to pick between the two different disjunction routes and is instead forced to pick the transit stop, even when this is sub-optimal.
I tried to use routing.ActiveVar(pickup_index) in the constraint as suggested in other similar problems, but it seems to have no effect in this case.
I believe the likely reason that this constraint is still affecting the solver and forcing it to choose this route is because it is still forcing some internal calculation of time_dimension.CumulVar(pickup_index) which might not be well defined if pickup_index is dropped?

I have attached the full code for running this example.

For transparency I should mention that I have already posted about this issue here:
https://groups.google.com/g/or-tools-discuss/c/1q7BRIvayDs
and I also talked about it here:
#4284