Dynamic creation of Arbitraries on demand

[dgrozenok]

I would like to be able to dynamically generate arbitraries for classes that are not know to FsCheck or not registered yet. Is it possible to register such a class in a similar to Json.NET custom converters manner? I would then have a class with the method

bool CanHandle(Type type)

that FsCheck would call when encounters an unknown type. And then I would have to define the two methods for returning the corresponding Arbitrary and Shrinker based on the type parameter.

Having this functionality it would be possible then to register a class that would create generators for LanguageExt NewType<> based classes in C#. For example for the class like this:

public class String32 : NewType<String32, string, StrLen<I1, I32>> { internal String32(string value) : base(value) { } }

It would be possible to generate the arbitrary based on the generic type parameters as

private static Gen<string> GenLimitedString(int min, int max) =>
    Arb.Default.NonEmptyString().Filter(s => s.Item.Length >= min && s.Item.Length <= max)
        .Generator.Select(s => s.Item);

public static Arbitrary<String32> String32Arb() => GenLimitedString(1, 32).Select(String32.New).ToArbitrary();

Now I have to define the arbitraries for every such type, while I think it could be done dynamically.

[kurtschelfthout]

Yes, it should be possible. The trick is to register a completely generic Arbitrary, and delegate to the default Arb.Default.Derive() for any type you don't want to handle. You'll need to use reflection though to inspect the generic type parameter, as well as to generate instances. That's essentially what Arb.Default.Derive() does as well.

I haven't tested this, but something like:

type MyArb =
  static member MyDerive<'T>() : Arbitrary<'T> =
    if CanHandle(typeof<'T>) then
      // create an instance of 'Arbitrary<'T>
   else
     Arb.Default.Derive<'T>()

Arb.register<MyArb>()

should be possible to get working. Let me know if you get stuck.

[dgrozenok]

Great! Works exactly as I needed. Thanks!

[dgrozenok]

One question though. Is it possible to create an arbitrary using reflection given the Type? Something like this would be helpful:

Arb.Default.Derive(Type type)

[dgrozenok]

I can call the Derive function using reflection given the valueType:

var deriveMethod = typeof(Arb)
    .GetMethod(nameof(Arb.Default.Derive))?
    .MakeGenericMethod(valueType);
var valueTypeArb = deriveMethod?.Invoke(null, null);

But now I would like to call Filter() and get the Generator from that arbitrary to convert the internal value type into the higher level one with Select(). I was trying to cast to Arbitrary<object>, but it fails. There is IArbitrary interface that would help, but it's internal. Any suggestions?

[kurtschelfthout]

Don't think that's currently exposed - do I understand correctly that you don't really want to generate any new types reflectively, but that the types you generate will impose filters/other constraint on existing types like string etc?

[dgrozenok]

Yes, that was the idea. Though I was thinking about that as a last resort for the types I don't have covered yet. The types I would like to handle automatically are the wrappers around primitive types. This is the way to implement single case unions in C#. Every such type has the three elements:

• a value property with the primitive type from which I can figure the type valueType and create the initial arbitrary for that type
• a predicate function predicateFunc, that I can use to filter the arbitrary
• a new function newFunc, that I can use to construct the arbitrary for the wrapper type

Something like this:

Arb.From(valueType)
    .Filter(predicateFunc)
    .Generator
    .Select(newFunc)
    .ToArbitrary();

It works fine for simple scenarios with insignificant limitations like a string of specific size. For the case of a type that represents an MongoDB ObjectId which is strictly a string with 24 hex characters the approach doesn't work. Very tiny chance of fscheck generating those. So I'm now thinking that I would have to abandon that initial idea and implement specific types of predicates like this would be for the ObjectId type:

if (default(PRED) is StrObjectId)
{
    return Gen.Elements(Enumerable.Range('0', 10).Concat(Enumerable.Range('a', 6)))
        .Select(Convert.ToByte)
        .ArrayOf(24)
        .Select(Encoding.ASCII.GetString)
        .Select(NewType<NEWTYPE, string, PRED>.New)
        .ToArbitrary();
}

I will have it all done in a generic class, so I will know the types while constructing the arbitraries. Then based on the specific types I will construct that helper class using reflection and MakeGeneric.

[kurtschelfthout]

I think for now it's easiest to have your own Arbitrary instances implement a type like IArbitrary so you can cast them in a similar way. But I'll think about how to handle this sort of scenario going forward - probably exposing Derive(Type) and IArbitrary or something like it makes sense.

[dgrozenok]

Just tried the version 3.0.0-beta1 and couldn't figure out how to implement the scenario with the dynamic arbitraries anymore for the case of nested types. Here is an example:

using System;
using FsCheck;
using FsCheck.Fluent;
using FsCheck.Xunit;

namespace ArbMapTests
{
    public class PersonId
    {
        internal PersonId(string value) => Value = value ?? throw new ArgumentNullException(nameof(value));
        public string Value { get; }
        public static PersonId New(string value) => new(value);
    }

    public class Person
    {
        public PersonId PersonId { get; set; }
    }
    
    public class MyArbitraries
    {
        public static Arbitrary<T> GenericArb<T>() where T : class
        {
            if (typeof(T).Name != "PersonId") return ArbMap.Default.ArbFor<T>();
            return ArbMap.Default
                .GeneratorFor<StringNoNulls>()
                .Select(s => (T)(object) PersonId.New(s.Item))
                .ToArbitrary();
        }
    }
    
    public class ArbMapTests
    {
        [Property(Arbitrary = new [] {typeof(MyArbitraries)})]
        public void PersonTest(Person person)
        {
        }
    }
}

I think the problem is in the line when I call ArbMap.Default.ArbFor<T>(). The Default map doesn't know anything about MyArbitraries. If I change it to ArbMap.Default.Merge<MyArbitraries>().ArbFor<T>() I'm getting the infinite recursion.

[kurtschelfthout]

Yes, it's no longer possible because I hid the Arb.Default.Derive which you need to make this work.

I'm considering what to do about this; for the moment I prefer your initial suggestion of allowing dynamic type tests somehow, with a boolean handler, to register these types. Otherwise the self-referential aspects are pretty confusing.

[dgrozenok]

Thank you. On a related note, when I have MyArbitraries class describing all the domain types, would it be possible to have it provided to all Xunit Property attributes on the test class level, so that I don't have to repeat it for every test case like this:

[Property(Arbitrary = new [] { typeof(MyArbitraries) })]
public void TestCase(Person person)
{
}

If it's not supported yet, I will create another issue. Maybe having another attribute for providing it on the class level.

[kurtschelfthout]

On a related note, when I have MyArbitraries class describing all the domain types, would it be possible to have it provided to all Xunit Property attributes on the test class level, so that I don't have to repeat it for every test case like this:

Yes, check out the Properties attribute: https://fscheck.github.io/FsCheck/RunningTests.html#Using-FsCheck-Xunit

[dgrozenok]

Perfect! Exactly what I needed. Thank you!

[bartelink]

@dgrozenok do you feel there's an outstanding issue here, or should it be closed?
Potentially semi-related: #644 (comment)

[kurtschelfthout]

I think we're all good here (but documentation....)