Prashant Sridharan
Microsoft Corporation
March 2003
Applies to:
Microsoft® Visual C#™
Summary: The Microsoft Corporation is in the process of developing the next major version of the C# language. This article contains information regarding four key new features, including generics, iterators, anonymous methods, and partial types. (15 printed pages.)
Contents
Introduction
Generics
Iterators
Anonymous Methods
Partial Types
Standards Compliance
Availability
More Information
Introduction
C# is a modern and innovative programming language that carefully incorporates features found in the most common industry and research languages. In keeping with the design philosophy of C#, Microsoft has introduced several potential new features to the C# language that increase developer productivity with language constructs.
Microsoft C#
Since its introduction in February 2001, a number of developers have begun building software using the C# programming language. Even within Microsoft, C# has been used to build several shipping applications, including the .NET Framework, MSN Web properties, and the Tablet PC SDK. As such, C# has proven itself as a language suitable for the construction of high-quality commercial software.
Many of the features in the C# language were created with four different design goals in mind:
- A unified type system and simplifying the way that value and reference types are used by the language.
- A component-based design established through features such as XML comments, attributes, properties, events and delegates.
- Practical developer headroom established through the unique capabilities of the C# language, including safe pointer manipulation, overflow checking, and more.
- Pragmatic language constructs, such as the foreach and using statements, which improve developer productivity.
In the "Visual Studio for Yukon" version of the C# language, Microsoft plans to build on an already elegant and expressive syntax by incorporating a variety of features across a broad spectrum of research and industry languages. Included among these language features are generics, iterators, anonymous methods and partial types.
Potential Future Features
Indeed, it is on the pillars of a unified type system, component-based development, developer headroom, and pragmatic language constructs that future innovation in C# is based. The following summarizes four key new features that Microsoft is planning on delivering in the next major version of the C# language. The design of these features has not yet been finalized and the Microsoft Corporation invites comment on them from the developer community.
Generics
As their projects get more sophisticated, programmers increasingly need a means to better reuse and customize their existing component-based software. To achieve such a high level of code reuse in other languages, programmers typically employ a feature called generics. C# will include a type-safe, high-performance version of generics which differs slightly in syntax and greatly in implementation from templates as found in C++ and generics as proposed for the Java language.
Building Generic Classes Today
In C# today, programmers can create a limited version of true generic types by storing data in instances of the base object type. Since every object in C# inherits from the base object type, and because of the boxing and unboxing features of the unified .NET type system, programmers can store both reference and value types into a variable whose type is object. However, there are performance penalties for converting between reference types and value types and the base object type.
To illustrate, the following code sample creates a simple Stack type with two actions, "Push" and "Pop". The Stack class stores its data in an array of object types, and the Push and Pop methods use the base object type to accept and return data, respectively:
public class Stack
{
private object[] items = new object[100];
public void Push(object data)
{
...
}
public object Pop()
{
...
}
}Then, a custom type can be pushed — a Customer type, for example — onto the stack. However, if your program needs to retrieve the data, it needs to explicitly cast the result of the Pop method, a base object type, into a Customer type.
Stack s = new Stack();
s.Push(new Customer());
Customer c = (Customer) s.Pop();
If a value type is passed, such as an integer, to the Push method, the run-time automatically converts it into a reference type — a process known as boxing — and then stores it in the internal data structure. Similarly, if you want your program to retrieve a value type, such as an integer, from the stack, your program needs to explicitly cast the object type obtained from the Pop method into a value type, a process known as unboxing:
Stack s = new Stack();
s.Push(3);
int i = (int) s.Pop();
The boxing and unboxing operations between value and reference types can be particularly onerous.
Furthermore, in the current implementation, it is not possible to enforce the kind of data placed in the stack. Indeed, a stack could be created and then a Customer type could be pushed onto it. Later, the same stack could be used and attempt to pop data off it and cast it into a different type, as in the following example:
Stack s = new Stack();
s.Push(new Customer());
Employee e = (Employee) s.Pop();
However, while the previous code sample is an improper use of the single type Stack class that is wanted for implementation and should be an error, it is actually legal code and the compiler would not have a problem with it. At run-time, however, the program would fail due to an invalid cast operation.
Creating and Consuming Generics
Generics in C# provide a facility for creating high-performance data structures that are specialized by the compiler based on the types that they use. These so-called parameterized types are created so that their internal algorithms remain the same, but also so that the types of their internal data can differ, based on end user preference.
In order to minimize the learning curve for developers, generics in C# are declared in much the same way as they are in C++. Programmers can create classes and structures just as they normally have, and by using the angle bracket notation (< and >) they can specify type parameters. When the class is used, each parameter must be replaced by an actual type that the user of the class supplies.
In the following example, create a Stack class where a type parameter, called ItemType, is specified and declared in angle brackets after the class declaration. Rather than forcing conversions to and from the base object type, instances of the generic Stack class will accept the type for which they are created and stores data of that type natively. The type parameter, ItemType, acts as a proxy until that type is specified during instantiation and is used as the type for the internal items array—the type for the parameter to the Push method, and the return type for the Pop method:
public class Stack<ItemType>
{
private ItemType[] items;
public void Push(ItemType data)
{
...
}
public ItemType Pop()
{
...
}
}When your program uses the Stack class, as in the following example, you can specify the actual type to be used by the generic class. In this case, you instruct the Stack class to use a primitive integer type by specifying it as a parameter using the angle notation in the instantiation statement:
Stack<int> stack = new Stack<int>();
stack.Push(3);
int x = stack.Pop();
In so doing, your program creates a new instance of the Stack class, for which every ItemType is replaced with the supplied integer parameter. Indeed, when your program creates the new instance of the Stack class with an integer parameter, the native storage of the items array inside the Stack class is now an integer rather than an object. Additionally, your program has eliminated the boxing penalty associated with pushing an integer onto the stack. Furthermore, when your program pops an item off the stack, you no longer need to explicitly cast it to the appropriate type because this particular instance of the Stack class natively stores an integer in its data structure.
If you want your program to store items other than an integer into a Stack class, you have to create a new instance of the Stack class, specifying the new type as the parameter. Suppose you have a simple Customer type and you wanted your program to use a Stack object to store it. To do so, you simply instantiate the Stack classes with the Customer object as the type parameter and easily reuse your program code:
Stack<Customer> stack = new Stack<Customer>();
stack.Push(new Customer());
Customer c = stack.Pop();
Of course, once your program creates a Stack class with a Customer type as its parameter, it is now limited to storing only Customer types in the stack. Indeed, generics in C# are strongly typed, meaning you can no longer improperly store an integer into the stack, such as in the following example:
Stack<Customer> stack = new Stack<Customer>();
stack.Push(new Customer());
stack.Push(3) // compile-time error
Customer c = stack.Pop(); // no cast required.
Benefits of Generics
Generics allow programmers to author, test, and deploy code once, and then reuse that code for a variety of different data types. While this is true of the first Stack example as well, the second Stack example allows your program to reuse code with a negligible performance impact to its applications. For value types, the first Stack example imposes a significant performance penalty, whereas the second Stack example eliminates the penalty entirely because you have eliminated boxing and downcasts.
In addition, generics are checked at compile-time. When your program instantiates a generic class with a supplied type parameter, the type parameter can only be of the type your program specified in the class definition. For example, when your program created a Stack of Customer objects, it was no longer able to push an integer onto the stack. By enforcing such behavior, you can build code that is more reliable.
Furthermore, the C# implementation of generics reduces code bloat when compared to other strongly-typed implementations. By creating typed collections with generics, you can avoid the need to create specific versions of each class while retaining the performance benefits of doing so. For example, your program can create one parameterized Stack class and avoid having to create an IntegerStack to store integers, a StringStack to store strings, or a CustomerStack to store Customer types.
This, of course, leads to code that is more readable. By creating one Stack class, your program can encapsulate all behavior associated with a stack in one convenient class. Then, when you create a Stack of Customer types, it is still obvious that your program is using a stack data structure, albeit with Customer types stored within it.
Multiple Type Parameters
Generic types can employ any number of parameter types. In the previous Stack example, only one type was used. Suppose you created a simple Dictionary class that stored values alongside keys. Your program could define a generic version of a Dictionary class by declaring two parameters, separated by commas within the angle brackets of the class definition:
public class Dictionary<KeyType, ValType>
{
public void Add(KeyType key, ValType val)
{
...
}
public ValType this[KeyType key]
{
...
}
}When using the Dictionary class, you need to supply multiple parameters within the angle brackets of the instantiation statement, again separated by commas, and supply the right types to the parameters of the Add function and indexer:
Dictionary<int, Customer> dict = new Dictionary<int, Customer>();
dict.Add(3, new Customer());
Customer c = dict.Get[3];
Constraints
Commonly, your program will do more than store data based on a given type parameter. Often, you will want your program to use members of the type parameter to execute statements within your program's generic type.
Why we need constraints
For example, suppose in the Add method of the Dictionary class you want to compare items using the CompareTo method of the supplied key, for example:
public class Dictionary<KeyType, ValType>
{
public void Add(KeyType key, ValType val)
{
...
switch(key.CompareTo(x))
{
}
...
}
}Unfortunately, at compile-time the type parameter KeyType is, as expected, generic. Written as-is, the compiler assumes that only the operations available to a base object type, such as ToString, are available to the key instance of the type parameter, KeyType. As a result, the compiler displays a compilation error because the CompareTo method is undefined. However, your program can cast the key variable to an object that does contain a CompareTo method, such as an IComparable interface. In the following example, your program explicitly casts the instance of the parameter type KeyType called key to an IComparable interface, allowing the program to compile:
public class Dictionary<KeyType, ValType>
{
public void Add(KeyType key, ValType val)
{
...
switch(((IComparable) key).CompareTo(x))
{
}
...
}
}However, if you now instantiate a Dictionary class and supply a type parameter that does not implement the IComparable interface, your program will encounter a run-time error, specifically an InvalidCastException.
Declaring Constraints
In C#, your program can supply an optional list of constraints for each type parameter declared in your generic class. A constraint indicates a requirement that a type must fulfill in order to construct a generic type. Constraints are declared using the where keyword, followed by a "parameter-requirement" pair where the "parameter" must be one of those defined in the generic type, and the "requirement" must be a class or interface.
In order to satisfy the need to use the CompareTo method in the Dictionary class, your program can impose a constraint on the KeyType type parameter, requiring any type passed to the first parameter of the Dictionary class to implement the IComparable interface, for example:
public class Dictionary<KeyType, ValType> where KeyType : IComparable
{
public void Add(KeyType key, ValType val)
{
...
switch(key.CompareTo(x))
{
}
...
}
}Now when compiled, your code will be checked to ensure that each time your program uses the Dictionary class, it is passing as a first parameter a type that implements the IComparable interface. Further, your program no longer has to explicitly cast the variable to an IComparable interface before calling the CompareTo method.
Multiple Constraints
For any given type parameter, your program can specify any number of interfaces as constraints, but no more than one class. Each new constraint is declared as another parameter-requirement pair, with each constraint for a given generic type separated by commas. In the following example, the Dictionary class contains two types of parameters, KeyType and ValType. The KeyType type parameter has two interface constraints, while the ValType type parameter has one class constraint:
public class Dictionary<KeyType, ValType> where
KeyType : IComparable,
KeyType : IEnumerable,
ValType : Customer
{
public void Add(KeyType key, ValType val)
{
...
switch(key.CompareTo(x))
{
}
...
}
}Generics in the Runtime
When a generic class is compiled, there is actually nothing different between it and a regular class. Indeed, the result of the compilation is nothing but metadata and intermediate language (IL). The IL is, of course, parameterized to accept a user-supplied type somewhere in code. How the IL for a generic type is used differs based on whether or not the supplied type parameter is a value or reference type.
When a generic type is first constructed with a value type as a parameter, the run-time creates a specialized generic type with the supplied parameter (or parameters) substituted in the appropriate places in the IL. Specialized generic types are created once for each unique value type used as a parameter.
For example, suppose your program code declared a Stack constructed of integers:
At this point, the run-time generates a specialized version of the Stack class with the integer substituted appropriately for its parameter. Now, whenever your program code uses a Stack of integers, the run-time reuses the generated specialized Stack class. In the following example, two instances of a Stack of integers are created, both using the code already generated by the run-time for a Stack of integers:
Stack<int> stackOne = new Stack<int>();
Stack<int> stackTwo = new Stack<int>();
However, if at another point in your program code another Stack class is created, this time with a different value type, such as a long or a user-defined structure as its parameter, the run-time generates another version of the generic type, this time substituting a long in the appropriate places in IL. The benefit of creating specialized classes for generics constructed with value types is better performance. After all, conversions are no longer necessary because each specialized generic class "natively" contains the value type.
Generics work slightly different for reference types. The first time a generic type is constructed with any reference type, the run-time creates a specialized generic type with object references substituted for the parameters in the IL. Then, each time a constructed type is instantiated with a reference type as its parameter, regardless of what type it is, the run-time reuses the previously created specialized version of the generic type.
For example, suppose you had two reference types, a Customer class and an Order class, and further suppose that you created a Stack of Customer types:
Stack<Customer> customers;
At this point, the run-time generates a specialized version of the Stack class that, instead of storing data, stores object references that will be filled in later. Suppose the next line of code creates a Stack of another reference type, called Order:
Stack<Order> orders = new Stack<Order>();
Unlike with value types, another specialized version of the Stack class is not created for the Order type. Rather, an instance of the specialized version of the Stack class is created and the orders variable is set to reference it. For each object reference that was substituted in place of the type parameter, an area of memory the size of an Order type is allocated and the pointer is set to reference that memory location. Suppose you then encountered a line of code to create a Stack of a Customer type:
customers = new Stack<Customer>();
As with the previous use of the Stack class created with the Order type, another instance of the specialized Stack class is created, and the pointers contained therein are set to reference an area of memory the size of a Customer type. Because the number of reference types can vary wildly from program to program, the C# implementation of generics greatly reduces code bloat by reducing to one the number of specialized classes created by the compiler for generic classes of reference types.
Moreover, when a generic C# class is instantiated with a type parameter, be it a value or reference type, it can be queried at runtime using reflection and both its actual type, as well as its type parameter, can be ascertained.
Differences Between C# Generics and Other Implementations
C++ templates differ significantly from C# generics. Where C# generics are compiled into IL, causing specialization to occur intelligently at runtime for each value type and once only for reference types, C++ templates are essentially code expansion macros that generate a specialized type for each type parameter supplied to a template. Therefore, when the C++ compiler encounters a template, such as a Stack of integers, it expands the template code into a Stack class that contains integers internally as its native type. Regardless of whether the type parameter is a value or reference type, unless the linker is specifically designed to reduce code bloat, the C++ compiler creates a specialized class each time, resulting in a significant increase in code bloat over C# generics.
Moreover, C++ templates cannot define constraints. C++ templates can only define constraints implicitly by simply using a member that might or might not belong to the type parameter. If the member does exist in the type parameter that is eventually passed to the generic class, the program will work properly. If the member does not exist in the type parameter, the program will fail and a cryptic error message will likely be returned. Because C# generics can declare constraints and are strongly typed, these potential errors do not exist.
Meanwhile, Sun Microsystems® has proposed the addition of generics in the next version of the Java language, code named "Tiger". Sun has chosen an implementation that does not require modifying the Java Virtual Machine. As such, Sun is faced with implementing generics on an unmodified virtual machine.
The proposed Java implementation uses similar syntax to templates in C++ and generics in C#, including type parameters and constraints. However, because it treats value types differently than reference types, the unmodified Java Virtual Machine will not be able to support generics for value types. As such, generics in Java will gain no execution efficiency. Indeed, the Java compiler will inject automatic downcasts from the specified constraint, if one is declared, or the base Object type, if a constraint is not declared, whenever it needs to return data. Furthermore, the Java compiler will generate a single specialized type at compile-time that it will then use to instantiate any constructed type. Finally, because the Java Virtual Machine will not support generics natively, there will be no way to ascertain the type parameter for an instance of a generic type at runtime and other uses of reflection will be severely limited.
Generics Support in Other Languages
It is Microsoft's intent to support the consumption and creation of generic types in Visual J#™, Visual C++, and Visual Basic. While some languages may implement this feature earlier than others, all of Microsoft's other three languages will contain support for generics. Meanwhile, the C# team is laying the groundwork for multiple language support by incorporating facilities in the underlying runtime for generics. Microsoft is working closely with third party language partners to ensure the creation and consumption of generics across .NET-based languages.
Iterators
An iterator is a language construct based on similar features in research languages such as CLU, Sather and Icon. Simply put, iterators make it easy for types to declare how the foreach statement will iterate over their elements.
Why Iterators are Needed
Today, if classes want to support iteration using the foreach loop construct, they must implement the "enumerator pattern". For example, the foreach loop construct on the left is expanded by the compiler into the while loop construct on the right:
List list = ...foreach(object obj in list)
{
DoSomething(obj);
}
| Enumerator e = list.GetEnumerator()while(e.MoveNext())
{
object obj = e.Current;
DoSomething(obj);
|
Notice that the List data structure—the instance being iterated—must support the GetEnumerator function in order for the foreach loop to work. Now that the List data structure is created, the GetEnumerator function must be implemented, which then returns a ListEnumerator object:
public class List
{
internal object[] elements;
internal int count;
public ListEnumerator GetEnumerator()
{
return new ListEnumerator(this);
}
}The ListEnumerator object that was created must not only implement the Current property and the MoveNext method, but it must also maintain its internal state so that your program can move to the next item each time around the loop. This internal state machine can be simple for the List data structure, but for data structures that require recursive traversal, such as binary trees, the state machine can be quite complicated.
Because implementing this enumerator pattern could require a great deal of effort and code on the developer's part, C# will include a new construct that will make it easier for a class to dictate how the foreach loop will iterate over its contents.
Defining an iterator
As an iterator is the logical counterpart of the foreach loop construct, it is defined similarly to a function using the foreach keyword, followed by an open and closed parenthesis pair. In the following example, your program will declare an iterator for the List type. The return type of the iterator is determined by the user, but since the List class stores an object type internally, the return type of the following iterator example will be an object:
public class List
{
internal object[] elements;
internal int count;
public object foreach()
{
}
}Notice that once the enumerator pattern is implemented, your program needs to maintain an internal state machine in order to keep track of where the program is in the data structure. Iterators have built-in state machines. Using the new yield keyword, your program can return values back to the foreach statement that called the iterator. The next time the foreach statement loops and calls the iterator again, the iterator begins its execution where the previous yield statement left off. In the following example, your program yields three string types:
public class List
{
internal object[] elements;
internal int count;
public string foreach()
{
yield "microsoft";
yield "corporation";
yield "developer division";
}
}In the following example, the foreach loop that calls this iterator will execute three times, each time receiving the strings in the order specified by the previous three yield statements:
List list = new List();
foreach(string s in list)
{
Console.WriteLine(s);
}If you want the program to implement the iterator to traverse the elements in the list, you would modify the iterator to step across the array of elements using a foreach loop, yielding each item in the array in every iteration:
public class List
{
internal object[] elements;
internal int count;
public object foreach()
{
foreach(object o in elements)
{
yield o;
}
}
}How Iterators Work
Iterators handle the messy chore of implementing the enumerator pattern on behalf of your program. Rather than having to create the classes and build the state machine, the C# compiler translates the code you have written in your iterator into the appropriate classes and code using the enumerator pattern. In so doing, iterators provide a significant increase in developer productivity.
Anonymous Methods
Anonymous methods are another practical language construct that allows programmers to create code blocks that can be encapsulated in a delegate and executed at a later time. They are based on a language concept called a lambda function and are similar to those found in Lisp and Python.
Creating Delegate Code
A delegate is an object that references a method. Whenever the delegate is invoked, the method that it references is called. In the following example, a simple form is illustrated with three controls—a list box, a text box and a button. When the button is initialized, the program instructs its Click delegate to reference the AddClick method stored elsewhere in the object. In the AddClick method, the value of the text box is stored into the list box. By virtue of being added to the button instance's Click delegate, the AddClick method is called whenever the button is clicked.
public class MyForm
{
ListBox listBox;
TextBox textBox;
Button button;
public MyForm()
{
listBox = new ListBox(...);
textBox = new TextBox(...);
button = new Button(...);
button.Click += new EventHandler(AddClick);
}
void AddClick(object sender, EventArgs e)
{
listBox.Items.Add(textBox.Text);
}
}Using Anonymous Methods
The previous example is fairly straightforward. A separate function is created, it is referenced by the delegate, and whenever the delegate is invoked, the program calls the function. Inside the function, a series of executable steps are performed. With the addition of anonymous methods, your program could forego creating an entirely new method for the class and instead directly reference the executable steps contained therein from the delegate. Anonymous methods are declared by instantiating a delegate, then following the instantiation statement with a curly brace pair that denotes a scope of execution, and a semicolon to terminate the statement.
In the following example, your program modifies the delegate creation statement to directly modify the list box rather than referencing a function that modifies the list box on behalf of your program. The code is stored to modify the list box within the scope of execution immediately following the delegate creation statement.
public class MyForm
{
ListBox listBox;
TextBox textBox;
Button button;
public MyForm()
{
listBox = new ListBox(...);
textBox = new TextBox(...);
button = new Button(...);
button.Click += new EventHandler(sender, e)
{
listBox.Items.Add(textBox.Text);
};
}
}Notice how code within the Anonymous method can access and manipulate variables declared outside its scope. Indeed, Anonymous methods can reference variables declared by the class as well as by parameters or local variables declared in the method in which it resides.
Passing Parameters to Anonymous Methods
Curiously, the Anonymous method statement includes two parameters, named sender and e. Looking at the definition of the Button class' Click delegate, you find that any function referenced by the delegate must include two parameters; the first of type object, the second of type EventArgs. In the first example, without using Anonymous methods, your program passed two parameters to the AddClick method, an object and an EventArgs.
Even though the code is written inline, the delegate must still receive two parameters. In the Anonymous method, the names of the two parameters must be declared so that the associated code block can use them. When the Click event on the button is triggered, the Anonymous method is invoked and the appropriate parameters are passed to it.
How Anonymous Methods Work
When an Anonymous delegate is encountered, the C# compiler automatically converts the code in its scope of execution into a uniquely named function within a uniquely named class. The delegate in which the code block is stored is then set to reference the compiler-generated object and method. When the delegate is invoked, the Anonymous method block is executed via the compiler-generated method.
Partial Types
While it is good object-oriented programming practice to maintain all source code for a type in a single file, sometimes performance constraints force types to be large. Further, the cost of splitting the type into subtypes is not acceptable for all situations. Moreover, programmers often create or use applications that emit source code, modifying the resulting code. Unfortunately, when source code is emitted again sometime in the future, all of the existing source code modifications are overwritten.
Partial types allow you to break up types consisting of a large amount of source code into several different source files for easier development and maintenance. Furthermore, partial types allow you to separate machine-generated and user-written parts of your types so that it is easier to supplement or modify code generated by a tool.
In the following example, two C# code files, File1.cs and File2.cs, both contain definitions for a class named Foo. Without partial types, this would represent a compiler error since both classes exist in the same namespace. With the partial keyword, we can indicate to the compiler that this class might have other definitions elsewhere.
| File1.cs | File2.cs |
public partial class Fo{
public void MyFunction()
{
// do something here
}
}
| public partial class Fo{
public void MyOtherFunction()
{
// do something here
}
}
|
Upon compilation, the C# compiler gathers all definitions of a partial type and combines them. The resulting IL generated by the compiler shows a single combined class, rather than several constituent classes.
Standards Compliance
In December 2001 the C# programming language was ratified by the European Computer Manufacturer's Association (ECMA) as a standard (ECMA 334). Soon thereafter, the C# standard was put on the "fast-track" process to the International Organization for Standards (ISO), where it is expected to be ratified shortly. A significant milestone in the evolution of a new programming language, the creation of a C# standard brought with it the promise of multiple implementations on a variety of operating system platforms. Indeed, as has been seen during its brief history, a number of third-party compiler vendors and researchers have implemented the standard and created their own versions of the C# compiler.
With these proposed features, Microsoft invites customers to provide their feedback on their addition to the C# language and intends to submit these features to the ongoing standards process for the language.
Availability
The features discussed here will be available in a future release of the C# compiler. In early 2003, the "Everett" release of Visual Studio .NET will contain a version of C# slightly modified for full ECMA compliance. This version will not include the features discussed here. Microsoft's hope is that these features will be included in the "VS for Yukon" release of Visual Studio, the date for which has not yet been determined.
Over the next several months, Microsoft will be releasing more information on these features, including full specifications. The programmer and language design community is invited to offer their opinion and feedback on these and any other language features they may find interesting. The C# language designers can be reached by sending email to the mailto:sharp@microsoft.com e-mail alias.
More Information
The C# Community Web site: http://www.csharp.net
The Visual C#™ Product Web site: http://msdn.microsoft.com/vcsharp