Magic of F# Type Providers

Are you watching closely?

The first time I saw a type provider I did not actually get what all the fuss was about. I mean, I have used code generation before, and while type providers seemed a little bit more convenient, I was not immediately sold on them.

But then I found different type providers, which changed my mind. Using them was like using dark magic (I’ll give one example shortly). And although I was excited, something wasn’t right. I could not for the life of me conceive of how they could do what they do. I knew I had to get to the bottom of that. If you know that too, you have come to the right place.

Let us begin with a code snippet below:

SqlCommandProvider signals error

What is going on there? I pass 2 static parameters to SqlCommandProvider, namely query string and connection string and get an error Invalid object name 'TableThatIsNotThere'.

My first question was how? This is just a simple .net string there, and somehow the compiler knows that it contains an error before I even try to compile anything. I mean, I know that F# is a strongly typed language and that the F# compiler can infer almost any type, but I would never dream of it checking even my strings for me.

Second question — what’s with these angle brackets? Aren’t they for type parameters in generics?

In the case of type providers angle brackets are used to supply static parameters, meaning parameters that are known at compile time. Which of course makes perfect sense if you know that type providers provide their types at compile time. Additionally, type providers can use regular dynamic parameters, provided in parenthesis. For example, I supply SqlCommandProvider with one dynamic parameter for the connection string that should be used at run time.

Static parameters typically give type providers necessary information to access some external information source. The connection string in SqlCommandProvider is a good example of that. But in some cases, a static parameter can be an information source in itself. The SQL string parameter gives SqlCommandProvider information about the query we would like to execute at run time.

Now SqlCommandProvider has two information sources to work with. It will try to create some type representing the SQL query and it will check this query for correctness. SqlCommandProvider will see that the query is used to select from TableThatIsNotThere, so it will check the DB schema for an object with this name. And if at any time in this process SqlCommandProvider encounters some problem it will throw an exception and the compiler will catch it. Modern compilers work not only when you explicitly try to compile something. They analyze your code as you type. So, the compiler will report an error to you via your IDE.

What this means effectively is that type providers are compiler plugins. They can be used not only to provide types but also to analyze information sources for correctness at compile time. SqlCommandProvider augments F# compiler so that it can “compile” SQL strings into some .net representation.

More formally, type providers are a form of compile time meta-programming.

Would you be surprised to decompile your assembly, or try to use reflection of it, and find no trace of types that were provided by some (helpful) type provider? I certainly would have been. As a matter of fact, this is exactly the case with most of them, because this is the default when you are writing type providers.

Type providers can give you two very different species of types — generated types and erased types.

Generated types are regular .net types. You can easily use them in other .net languages or use reflection on them and what not. One thing you should be aware of is that F# type providers are designed specifically for the F# compiler, and would not work with other compilers. So, if you want to use generated types in other .net languages you have to generate them with the F# compiler in some separate assembly.

Erased types, on the other hand, are no regular types. In fact, even in F#, the only way you can get them is with type providers.

Erased types behave as regular types when you work in your IDE, but they will be “erased” after compilation. The term “erased” may lead you to believe that you will lose your code, but in fact, erased types will be replaced by something else. Designers of type provider libraries have to specify an erasure for each erased type, that is — a regular type that will take take the place of the erased type in runtime. That could be some standard .net type (e.g. Dictionary) or some custom type designed specifically for the type provider. As for methods and properties of erased types, their code will be simply inlined.

This is all fine and well, if a little bit weird, but what purpose could it serve?

Erased types can contain delayed members (nested types and properties). What that gives you is the ability to get a type that could potentially represent infinite information space. Surely there is no infinite information source out there (or is there)? But some data sources are so big, interconnected and complicated that generating their full type (or hosting such a type in runtime) just is not feasible. Some type providers can give you an erased type for every single row in your database; can you imagine what generated code for that would look like?

For that reason, erased provided types have their rightful place in F# today.

I hope that this article was helpful in understanding the “magic” of type providers, and with that understanding, you will know what to expect from them at compile time as well as at run time and will use them to your advantage.

If you want to go deeper down that rabbit hole, here are some links for further reading:

  • Paper by Don Syme and others on the idea of F# type providers and some use cases for them: link
  • Tutorial on creating type providers by Microsoft: link
  • Several other articles on creating type providers: link
  • GitHub repo of type provider’s SDK for creating type providers: link

Software engineer at Kaspersky Lab, Moscow. Opinions are my own.